stack manipulation update

[John Rose]

[CC-ing mlvm-dev with my reply, since your note is of great general  
interest.  -- John]

On Apr 22, 2008, at 11:22 AM, Lukas Stadler wrote:

> Hi!
>> If you have a login for that wiki, send me your login identifier,  
>> and I'll add you to the author list.  ('ChristianWimmer' already  
>> has author access, and every JVM developer should have it, though  
>> I have to enter the names one at a time for now.)
> I've created a wikis.sun.com user named "LukasStadler". It would be  
> great if you could add that account to the author list.

Done this morning...  Please feel free to create or edit pages in  
HotSpotInternals, especially notes on implementation as you learn  
your way around.

> Building: Maybe it should be mentioned that the mercurial mq  
> extension needs to be enabled. It took me a while to figure out  
> where the "qselect" command comes from...

Thanks; I'll fix it in the README.

>> Are the continuations really bounded?  (Looks like the framework's  
>> main method, as reflected, serves as a boundary token.)  What  
>> happens when you try to return from a bounded continuation?
> Good question - right now it is silently assumed that the  
> continuation is reinstated at a correct position (in this case the  
> stackframes are removed up to and including the main method, and  
> then the main method is the first stackframe to be installed again)

I suppose when a continuation is resumed all frames up to its bound  
should be overwritten.

>> I thought there was some serialization story there.  The source  
>> code mentions a need to setAccessible(true).  (Another potential  
>> security challenge, out of many in this work.)
> Hm... I don't think that there's a way - Method doesn't implement  
> Serializable and there's no special code in the  
> ObjectOutputStream... But it's not that big a problem anyway, I  
> just don't see why Class is serializable and Method is not.

Here is a guess:  There's a security hole if you allow arbitrary  
methods to be created from raw bytes with their "accessible" bit set  
to true.  But the fix would be to do the same security checks that  
setAccessible does in the first place.

>>>      - Sounds like a reorganization of the code may allow nearly  
>>> complete reuse of the deopt blob
>> I hope so.  There are not that many degrees of freedom in  
>> interpreter frames, so it should not need fundamentally new  
>> techniques.
> I added an entry point into the deopt_blob that can be passed an  
> UnrollInfo object (and that doesn't call fetch_unroll_info). It's  
> only ~10 new lines of code. One problem that I see is that it may  
> be necessary to remove the stackframe of an inlined method from a  
> compiled frame to reach the boundary method, which means that the  
> topmost frame may need to be deoptimized. This might prove, uhm,  
> interesting. (because it shouldn't return to the Interpreter after  
> it's done)

That's one reason I made the boundary method be a fixed native method  
doCopyStackContext.  All those edge cases are easier to control that  
way.  I don't see (yet) that it's a significant loss of generality.

> I also added code to vframeArray and Deoptimization that creates a  
> vframeArray from a StackSerializer. So far it works like a charm,  
> although I'm ignoring monitors for now. (I don't really know how to  
> create or acquire the BasicLock objects for monitors,  
> ObjectSynchronizer::inflate?)

I don't remember off hand, but I'd look at how the current deopt. or  
OSR code does the trick.

> I have created Java classes for the stack state (see the attached  
> files) and I've written native code that performs the stack copy  
> operation into these objects. I'd like to work with this  
> representation for the time being. I don't think that this is much  
> less efficient than the StackSerializer (apart from the obviously  
> larger memory consumption). Currently I'm writing code to create  
> the vframeArray from this representation and I'll send you an  
> overview of all the changes I made as soon as I'm finished with that.
You probably need more random access than the serialized form can  
give you.  Hmmm...  The array-based representation is fine for  
prototyping but for production use is too transparent to optimize or  
secure.  It's not a short term problem.  What we need is a sort of  
union-type list abstraction that can equally easily store references  
and primitives (at least ints; would like IJFD), but without exposing  
its internals too much.

> interface MachineWordList extends List<Object> {
>   int size();
>   byte type(int i);
>   Object get(int i);
>   int getInt(int i);
>   long getLong(int i);
>   // setters?
>   void set(int i, Object x);
>   void setInt(int i, int x);
>   // arrays?
>   byte[] typeArray();
>   Object[] toArray();
>   int[] toIntArray();
> }

But this is a cleanup; you don't have to turn aside from the main work.

>
>> Generators:  That's an important use case for "small scale"  
>> continuations.  Open question:  Does the existing design (a  
>> serialized blob of data) give enough visibility to the compiler to  
>> allow it to optimize across continuation invocations (thus  
>> inlining generator patterns)?  Probably not, unless the unsafe  
>> copyStack/resumeStack structures are made more stateless and  
>> immutable, so they can be examined by the compiler.
> An ImmutableContinuation object, which allows no access to its  
> contents whatsoever, would fit this purpose, I think. The compiler  
> could even detect cases where the continuation is used as a  
> nonlocal return in inlined methods and create no continuation at  
> all...
Yes, that's the end-to-end optimization that we want for things like  
generators.

Great work!

-- John

> package javax.stack;
>
> import java.lang.reflect.Method;
>
> /**
>  * Contains the defining state of one stack frame activation. Read  
> access provides access to private data, unverified
>  * write access may allow code to escape security constraints,  
> violate JVM invariants or even break the JVM.
>  *
>  * Data within this object is somewhat packed: the values and  
> objects of local variables and expressions are
>  * sequentially written into valueSlots and objectSlots. After that  
> the monitors are written into objectSlots.
>  */
> public class Stackframe {
> 	private Method method;
> 	private int bci;
>
> 	private boolean[] localIsObject;
> 	private boolean[] expressionIsObject;
> 	private int[] valueSlots;
> 	private Object[] objectSlots;
>
> 	public Method getMethod() {
> 		return method;
> 	}
>
> 	public void setMethod(Method method) {
> 		this.method = method;
> 	}
>
> 	public int getBci() {
> 		return bci;
> 	}
>
> 	public void setBci(int bci) {
> 		this.bci = bci;
> 	}
>
> 	public int getLocalCount() {
> 		return localIsObject == null ? 0 : localIsObject.length;
> 	}
>
> 	public int getExpressionCount() {
> 		return expressionIsObject == null ? 0 : expressionIsObject.length;
> 	}
>
> 	public int getMonitorCount() {
> 		return (valueSlots == null ? 0 : valueSlots.length) +  
> (objectSlots == null ? 0 : objectSlots.length)
> 				- getLocalCount() - getExpressionCount();
> 	}
>
> 	public int getValueSlot(int index) {
> 		return valueSlots[index];
> 	}
>
> 	public Object getObjectSlot(int index) {
> 		return objectSlots[index];
> 	}
>
> 	public void setValueSlots(int[] valueSlots) {
> 		this.valueSlots = valueSlots;
> 	}
>
> 	public void setObjectSlots(Object[] objectSlots) {
> 		this.objectSlots = objectSlots;
> 	}
>
> 	public boolean localIsObject(int index) {
> 		return localIsObject[index];
> 	}
>
> 	public boolean expressionIsObject(int index) {
> 		return expressionIsObject[index];
> 	}
>
> 	public void setLocalIsObject(boolean[] localIsObject) {
> 		this.localIsObject = localIsObject;
> 	}
>
> 	public void setExpressionIsObject(boolean[] expressionIsObject) {
> 		this.expressionIsObject = expressionIsObject;
> 	}
>
> }
> package javax.stack;
>
> import java.lang.reflect.Method;
>
> /**
>  * Allows to store stack frames so that they can be inspected,  
> modified and reinstated. Any access beyond querying the
>  * stack frame count and the methods poses a potential security risk.
>  */
> public class Continuation {
> 	private Stackframe[] stackframes;
>
> 	public int getStackframeCount() {
> 		return stackframes == null ? 0 : stackframes.length;
> 	}
>
> 	public Method getMethod(int index) {
> 		return stackframes[index].getMethod();
> 	}
>
> 	public Stackframe getStackframe(int index) {
> 		// TODO: security check ...
> 		return stackframes[index];
> 	}
>
> 	public Stackframe[] getStackframes() {
> 		// TODO: security check ...
> 		return stackframes;
> 	}
>
> 	public void setStackframes(Stackframe[] stackframes) {
> 		// TODO: security check
> 		this.stackframes = stackframes;
> 	}
> }


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20080422/caa4b4ff/attachment.html