From steshaw at gmail.com Mon Jul 7 03:42:59 2008 From: steshaw at gmail.com (Steven Shaw) Date: Mon, 7 Jul 2008 20:42:59 +1000 Subject: Time to reconsider m:n or green threading options? In-Reply-To: References: <485CCCBF.1040409@sun.com> <64efa1ba0806210318m63546c5ambee9118b5db22133@mail.gmail.com> Message-ID: <1ba9d4a00807070342x43832e38i16584499e2d2179a@mail.gmail.com> 2008/6/24 John Rose : > It would be great if we had really lightweight continuations, with a JVM > scheduler (Scheme calls them engines, I think) which keeps running the next > one. The part I can't see yet is how to make stack-based and heap-based > activation records play together efficiently. (Maybe you JIT two versions > of every method, with inlining to remove overheads as usual?) > Is it disastrous for performance to heap allocated all activations? Perhaps it could be an JVM option. Perhaps activations could be abstractly considered "heap allocated" and have the JVM stack allocated when possible. Is the much research in this area? Cheers, Steve. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20080707/c0956a3d/attachment.html From John.Rose at Sun.COM Mon Jul 7 11:38:41 2008 From: John.Rose at Sun.COM (John Rose) Date: Mon, 07 Jul 2008 11:38:41 -0700 Subject: Time to reconsider m:n or green threading options? In-Reply-To: <1ba9d4a00807070342x43832e38i16584499e2d2179a@mail.gmail.com> References: <485CCCBF.1040409@sun.com> <64efa1ba0806210318m63546c5ambee9118b5db22133@mail.gmail.com> <1ba9d4a00807070342x43832e38i16584499e2d2179a@mail.gmail.com> Message-ID: <9FA626F2-BD7F-481B-A76A-F70D79677834@sun.com> On Jul 7, 2008, at 3:42 AM, Steven Shaw wrote: > 2008/6/24 John Rose : > It would be great if we had really lightweight continuations, with > a JVM scheduler (Scheme calls them engines, I think) which keeps > running the next one. The part I can't see yet is how to make > stack-based and heap-based activation records play together > efficiently. (Maybe you JIT two versions of every method, with > inlining to remove overheads as usual?) > > Is it disastrous for performance to heap allocated all activations? It would be a long, hard job to make up the performance loss. Not all JVMs (especially the small ones) could follow. The immediate reuse of memory that stacks enable is very cache- friendly, and we have to be friends with caches these days. > Perhaps it could be an JVM option. Perhaps activations could be > abstractly considered "heap allocated" and have the JVM stack > allocated when possible. If you could get the escape events to happen infrequently, you could do this, with stack-to-heap copying to occur only when required. In such a design the eager allocation (if any) could be reduced to a one- word handle object. Stack frames involving generators or closures could skip directly into the heap. That suggests a "try to keep me on the stack" bit in the bytecode. You might want to combine groups of tightly-coupled methods into stacklets served by one allocation. A system which bounces things between stack and heap probably requires lots of global analysis. To make that part of the JVM architecture probably requires a new set of verifiable assertions which a low-end JVM could use directly. (It would verify the assertions transmitted in the bytecodes, rather than re-derive them from scratch.) I don't think such a thing has been investigated yet. > Is the much research in this area? I'm not aware of anything related to the JVM, but there's some good fundamental stuff by Appel and Shao: http://flint.cs.yale.edu/flint/publications/stack.html -- John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20080707/f861e8fe/attachment.html From steshaw at gmail.com Wed Jul 9 02:51:38 2008 From: steshaw at gmail.com (Steven Shaw) Date: Wed, 9 Jul 2008 19:51:38 +1000 Subject: Time to reconsider m:n or green threading options? In-Reply-To: <9FA626F2-BD7F-481B-A76A-F70D79677834@sun.com> References: <485CCCBF.1040409@sun.com> <64efa1ba0806210318m63546c5ambee9118b5db22133@mail.gmail.com> <1ba9d4a00807070342x43832e38i16584499e2d2179a@mail.gmail.com> <9FA626F2-BD7F-481B-A76A-F70D79677834@sun.com> Message-ID: <1ba9d4a00807090251l5c144a91vd1c021b1942f9c55@mail.gmail.com> 2008/7/8 John Rose : > I'm not aware of anything related to the JVM, but there's some good > fundamental stuff by Appel and Shao: > http://flint.cs.yale.edu/flint/publications/stack.html Thanks for the reply John! I haven't seen this paper before and I'm really enjoying it :) Cheers, Steve. From john.rose at sun.com Tue Jul 15 00:19:00 2008 From: john.rose at sun.com (john.rose at sun.com) Date: Tue, 15 Jul 2008 07:19:00 +0000 Subject: hg: mlvm/mlvm/hotspot: 2 new changesets Message-ID: <20080715071900.4B0E8D11C@hg.openjdk.java.net> Changeset: c0db1bded354 Author: jrose Date: 2008-07-15 00:03 -0700 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/c0db1bded354 anonk: rebase to jdk7-b30 ! anonk.patch ! series Changeset: 99e6c3830f6d Author: jrose Date: 2008-07-15 00:12 -0700 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/99e6c3830f6d callcc: resume works better, stack frame API Contributed-By: Lukas Stadler rebase to jdk7-b30 ! callcc.patch ! callcc.txt ! series From john.rose at sun.com Tue Jul 15 00:19:40 2008 From: john.rose at sun.com (john.rose at sun.com) Date: Tue, 15 Jul 2008 07:19:40 +0000 Subject: hg: mlvm/mlvm/jdk: 2 new changesets Message-ID: <20080715071940.E26E4D121@hg.openjdk.java.net> Changeset: 43e41568eedf Author: jrose Date: 2008-07-15 00:03 -0700 URL: http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/43e41568eedf anonk: rebase to jdk7-b30 ! series Changeset: 524d5cf342c5 Author: jrose Date: 2008-07-15 00:17 -0700 URL: http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/524d5cf342c5 callcc: resume works better, stack frame API Contributed-By: Lukas Stadler rebase to jdk7-b30 ! callcc.patch ! series From john.rose at sun.com Tue Jul 15 00:44:56 2008 From: john.rose at sun.com (john.rose at sun.com) Date: Tue, 15 Jul 2008 07:44:56 +0000 Subject: hg: mlvm/mlvm/hotspot: anonk: fix two build problems Message-ID: <20080715074456.755D2D126@hg.openjdk.java.net> Changeset: 886f9ddefb15 Author: jrose Date: 2008-07-15 00:43 -0700 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/886f9ddefb15 anonk: fix two build problems callcc: extract misplaced bug fix Contributed-By: jrose, Lukas Stadler ! anonk.patch ! callcc.patch From John.Rose at Sun.COM Tue Jul 15 00:50:01 2008 From: John.Rose at Sun.COM (John Rose) Date: Tue, 15 Jul 2008 00:50:01 -0700 Subject: new callcc.patch In-Reply-To: <4863AAB2.8070501@jku.at> References: <4863AAB2.8070501@jku.at> Message-ID: <2AB7390A-A38C-436A-B23C-C51238E715CC@Sun.COM> On Jun 26, 2008, at 7:41 AM, Lukas Stadler wrote: > This is my current version of the callcc.patch ... working with the > callcc code the last few weeks brought up a few problems (safepoint- > safety, ...) for which I made some changes. > > * the deopt_blob isn't patched into the return pc any more because > the vframearray already exists when the resumeStack method returns > and thus the oops that it contains could be collected. the > deopt_blob is now called like a method and thus the continuation > entry point now behaves like a method entry point. I've pushed your most recent changes. I tweaked it a bit to get it to build and run on Solaris. (My development platform is Solaris / VMware / MacBook Pro.) I also changed stackframe to stackFrame or stack_frame, as appropriate to language and context. The "smoke test" (i.e., does the widget emit puffs of smoke when power is applied?) runs as before. I rebased everything to the current build, b30. There was a puzzle with MonitorInfo structs; they grew another field (called "eliminated") because of escape analysis, and I am not sure I passed the values through correctly: Please give it a look: http://hg.openjdk.java.net/mlvm/mlvm/hotspot http://hg.openjdk.java.net/mlvm/mlvm/jdk > * (this is part of anonk.patch) I've changed line 387 of > constantPoolOop.cpp to "assert(pseudo_ok || > java_lang_String::is_instance(entry), "must be string"); " because > of a compiler warning that prohibited product builds. I pushed that separately, along with an anonk fix that sneaked into the callcc patch. Regarding your earlier comments: > * there are two copyStack methods, but the resumeStack method can > be used for both code paths! You'll have noticed that the serialization-based POC was simpler to code up, but of course the data-structure-based one has the potential to be faster (and to allow data structure reuse for related continuations). Let's make sure the extra structure pays for itself. > * the new continuation entry point in the deopt_blob is only > implemented in the x86 version - it gets the UnrollBlock via rax That's OK for now. It will have to be ported to SPARC at some point, but we're not there yet. > * I have added code to javaClasses.* for my Continuation and > Stackframe classes Right. See comment above. > * doCopyStackContext is now an intrinsic Java method that will > always be interpreted, this allows for nested doCopyStackContext calls > * I made some initial performance improvements > * I was surprised how costly it is to create the reflection method > object Maybe method handles can bail us out here. (I put a comment in to that effect.) The idea is that a method handle is to the methodOop as a Class object is to a klassOop: It's a small Java handle to the ineffable metaobject. Reflective methods are more like symbol table entries; they are intrinsically bulky. > * there are lots of things that could be cached, like oop maps, etc > * I think that it should be allowed to reuse a Continuation > object, this would save allocation time Yes; they should probably present an immutable interface also. > * I think that in the long run there should be a special deoptblob/ > UnrollBlock implementation for continuations, for performance reasons > * the return value and exception for the continuations are now > stored in UnrollBlock, I'm not sure if this is ok In general, if you do it the way deopt does, you'll be safest. Thanks for the good work! Best wishes, -- John From John.Rose at Sun.COM Wed Jul 16 21:09:40 2008 From: John.Rose at Sun.COM (John Rose) Date: Wed, 16 Jul 2008 21:09:40 -0700 Subject: Interface injection In-Reply-To: <9997d5e60807141007n6203e2a2hc4718bd45edb292c@mail.gmail.com> References: <9997d5e60807141007n6203e2a2hc4718bd45edb292c@mail.gmail.com> Message-ID: On Jul 14, 2008, at 10:07 AM, Tobias Ivarsson wrote: > I've done a fair amount of thinking about interface injection: > http://journal.thobe.org/2008/07/my-jvm-whishlist-pt-1-interface.html > I would really like to see this happening. Is anybody working on > it? What would it take to bring interface injection to hotspot? Is > there any way I can help? > > best wishes, > Tobias Ivarsson I'm excited that you are excited! I have dumped a few more notes online, and you are welcome to do so also: http://wikis.sun.com/display/mlvm/InterfaceInjection (Make a wiki ID and send it to me and I will give you edit access. Same offer goes to any mlvm contributor.) I'm working hard on method handles this month, so I don't have much time for anything else. But I (and people on this mlvm-dev list) would be interested to talk more about concrete implementation details. The dense answer to your question is that it would take the following items to bring interface injection to HotSpot: - method handles as an enabling technology (to simplify certain details of implementation and specification) - a proof of concept coded (as inti.patch) in the Da Vinci Machine project - a clean, clear, precise specification, independent of HotSpot - a few exciting uses of it (i.e., some wins to motivate adoption) - eventually, incorporation in JSR 292 or a successor JSR You raised the question on your blog about the interactions with reflection, esp. Class.getInterfaces. The question of reflection always comes up with such extensions. My answer is usually that it is OK to surface the new functionality as new methods in the Core Reflection API. So you might have Class.getInjectedInterfaces, as a changing quantity. You would also have Class.getRejectedInterfaces; both sets would never shrink over time. I had a further thought today about this: Interface injection must integrate with subtyping properly, but perhaps there is a clean way to get a separate injection decision for each class in a hierarchy, rather than just one for the top. Of course interface methods need to override each other; maybe the injection decision point can declare that an injection is _final_ for a particular class (meaning every subclass reuses that injection). But if an injection is not final (w.r.t. the class hierarchy is what I mean), the each subclass also gets an injection event. This can be useful as a way to hand customized per-class state on classes. The most obvious use (for dynamic languages) is to associate a metaobject with each class encountered by the dynamic language runtime, but there are other uses too, such as associating counters with classes, etc. By using interface injection, such appended state would be just a method call away. (This also raises the question of an API for getting injected properties from the class itself, even if you don't have an instance to invoke on.) -- John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20080716/36c131d0/attachment.html From John.Rose at Sun.COM Sat Jul 19 14:45:52 2008 From: John.Rose at Sun.COM (John Rose) Date: Sat, 19 Jul 2008 14:45:52 -0700 Subject: what's John doing? Message-ID: Hello, colleagues. This month I'm hammering on method handle code in the JVM. I'll do a checkpoint (push of partially working code) to the repo. in a few days. (As a second priority, I'm also working on a mixed array data structure. The refactoring part of that code is out for review here: http://webrev.invokedynamic.info/jrose/6711911.layout/ ) The JSR 292 EDR period has quieted down, but we should get another wave of comments when the method handles code goes public, because people will want to at least read the javadoc for the Java APIs and help make it better. I have taken to heart the suggestions that MethodHandle and MethodType be concrete classes instead of interfaces. I'm using NetBeans to develop the Java code that works with the JVM (java.dyn.MethodHandle, etc.) Since this code must go on the boot class path, I had to convince NetBeans to run its JUnit testing with funny JVM arguments. The tips are here: http://wikis.sun.com/display/mlvm/BootClassPath If you come up with similar workaround on Eclipse, please add it to the wiki there. (Wiki registration is free; write access is available on request, because an administrator--yours truly--has to upgrade your wiki id to editor.) As I relearn all the calling sequence paths in HotSpot, I'm putting out notes on what I find. You might enjoy the HotSpotInternals wiki pages I've added recently: http://wikis.sun.com/display/HotSpotInternals/CallingSequences http://wikis.sun.com/display/HotSpotInternals/StaticCalls http://wikis.sun.com/display/HotSpotInternals/VirtualCalls http://wikis.sun.com/display/HotSpotInternals/InterfaceCalls The Da Vinci Machine Project wiki has (or should have) a page for each current or proposed subproject. I just dumped some design notes into the method handles page: http://wikis.sun.com/display/mlvm/MethodHandles If you are working on another subproject (Lukas, Arnold?) please use the wiki page like you use that never-fully-erased whiteboard in your office. You know, the one with all the interesting design notes, the one that gets added to every time you have an interested colleague come by on the way to the coffee pot. In general, as you learn your way around the code, please (!) add your discoveries to the wiki, while they are fresh to you. Best summer wishes, -- John From yozh at mx1.ru Mon Jul 21 07:30:47 2008 From: yozh at mx1.ru (Stepan Koltsov) Date: Mon, 21 Jul 2008 18:30:47 +0400 Subject: what's John doing? In-Reply-To: References: Message-ID: <7c1bea4b0807210730m3e0ed9c4le486010d54b4888e@mail.gmail.com> Shouldn't java.dyn be java.lang.dyn? For symmtry with java.lang.reflect. S. On Sun, Jul 20, 2008 at 01:45, John Rose wrote: > Hello, colleagues. This month I'm hammering on method handle code in > the JVM. I'll do a checkpoint (push of partially working code) to > the repo. in a few days. > > (As a second priority, I'm also working on a mixed array data > structure. The refactoring part of that code is out for review here: > http://webrev.invokedynamic.info/jrose/6711911.layout/ ) > > The JSR 292 EDR period has quieted down, but we should get another > wave of comments when the method handles code goes public, because > people will want to at least read the javadoc for the Java APIs and > help make it better. I have taken to heart the suggestions that > MethodHandle and MethodType be concrete classes instead of interfaces. > > I'm using NetBeans to develop the Java code that works with the JVM > (java.dyn.MethodHandle, etc.) Since this code must go on the boot > class path, I had to convince NetBeans to run its JUnit testing with > funny JVM arguments. The tips are here: > http://wikis.sun.com/display/mlvm/BootClassPath > > If you come up with similar workaround on Eclipse, please add it to > the wiki there. (Wiki registration is free; write access is > available on request, because an administrator--yours truly--has to > upgrade your wiki id to editor.) > > As I relearn all the calling sequence paths in HotSpot, I'm putting > out notes on what I find. You might enjoy the HotSpotInternals wiki > pages I've added recently: > http://wikis.sun.com/display/HotSpotInternals/CallingSequences > http://wikis.sun.com/display/HotSpotInternals/StaticCalls > http://wikis.sun.com/display/HotSpotInternals/VirtualCalls > http://wikis.sun.com/display/HotSpotInternals/InterfaceCalls > > The Da Vinci Machine Project wiki has (or should have) a page for > each current or proposed subproject. I just dumped some design notes > into the method handles page: > http://wikis.sun.com/display/mlvm/MethodHandles > > If you are working on another subproject (Lukas, Arnold?) please use > the wiki page like you use that never-fully-erased whiteboard in your > office. You know, the one with all the interesting design notes, the > one that gets added to every time you have an interested colleague > come by on the way to the coffee pot. > > In general, as you learn your way around the code, please (!) add > your discoveries to the wiki, while they are fresh to you. > > Best summer wishes, > -- John > _______________________________________________ > mlvm-dev mailing list > mlvm-dev at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev > From John.Rose at Sun.COM Mon Jul 21 11:35:29 2008 From: John.Rose at Sun.COM (John Rose) Date: Mon, 21 Jul 2008 11:35:29 -0700 Subject: what's John doing? In-Reply-To: <7c1bea4b0807210730m3e0ed9c4le486010d54b4888e@mail.gmail.com> References: <7c1bea4b0807210730m3e0ed9c4le486010d54b4888e@mail.gmail.com> Message-ID: On Jul 21, 2008, at 7:30 AM, Stepan Koltsov wrote: > Shouldn't java.dyn be java.lang.dyn? For symmtry with > java.lang.reflect. Could be, but the main function of java.lang is the implicit import for all Java programs. Since this stuff has nothing to do with the Java language per se, java.lang is a red herring. -- John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20080721/4eef3aad/attachment.html From arnold.schwaighofer at gmail.com Tue Jul 22 10:35:58 2008 From: arnold.schwaighofer at gmail.com (Arnold Schwaighofer) Date: Tue, 22 Jul 2008 19:35:58 +0200 Subject: what's John doing? In-Reply-To: References: Message-ID: > If you are working on another subproject (Lukas, Arnold?) please use Hi John, hi my fellow jvm workers. Yes i am still alive. Some small status report on my work on tail calls. Tail calls in the interpreter (only) work so far. (was easiest part :). Static sibling calls in c1 also work. Currently i am on vacation so work on it will be stalled for another few weeks. regards and have a nice summer time arnold From john.rose at sun.com Thu Jul 31 04:00:04 2008 From: john.rose at sun.com (john.rose at sun.com) Date: Thu, 31 Jul 2008 11:00:04 +0000 Subject: hg: mlvm/mlvm/hotspot: meth: 6655638: dynamic languages need method handles Message-ID: <20080731110004.56E98DCC5@hg.openjdk.java.net> Changeset: ad4e40228a6a Author: jrose Date: 2008-07-31 03:58 -0700 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/ad4e40228a6a meth: 6655638: dynamic languages need method handles Summary: MethodHandle and related types, precursor to invokedynamic Includes a standalone project of associated JDK classes. Reviewed-by: jrose + indy.txt + meth.patch + meth.proj.patch + meth.txt ! series From John.Rose at Sun.COM Thu Jul 31 12:28:24 2008 From: John.Rose at Sun.COM (John Rose) Date: Thu, 31 Jul 2008 12:28:24 -0700 Subject: what's John doing? In-Reply-To: <7c1bea4b0807210730m3e0ed9c4le486010d54b4888e@mail.gmail.com> References: <7c1bea4b0807210730m3e0ed9c4le486010d54b4888e@mail.gmail.com> Message-ID: Last Saturday around midnight the first method handle invocation went through. It was a virtual call, equivalent to: Object x = "foo"; foo.toString(); See MethodHandleBytecodeTest.testToString in the push I just did: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/ Next up, adapter method handles. -- John From charles.nutter at sun.com Thu Jul 31 14:35:47 2008 From: charles.nutter at sun.com (Charles Oliver Nutter) Date: Thu, 31 Jul 2008 16:35:47 -0500 Subject: what's John doing? In-Reply-To: References: <7c1bea4b0807210730m3e0ed9c4le486010d54b4888e@mail.gmail.com> Message-ID: <48923033.2050405@sun.com> John Rose wrote: > Last Saturday around midnight the first method handle invocation went > through. It was a virtual call, equivalent to: > Object x = "foo"; > foo.toString(); > > See MethodHandleBytecodeTest.testToString in the push I just did: > http://hg.openjdk.java.net/mlvm/mlvm/hotspot/ > > Next up, adapter method handles. Very nice. I saw the commit come in and tweeted it. I'm going to try to get my MLVM builds back up and going again. At what point do you think it would be worth trying to do an experimental test with e.g. JRuby, wiring it in where reflection was being used previously? - Charlie