Lazy Method Handle update

[John Rose]

On May 9, 2012, at 8:30 PM, Charles Oliver Nutter wrote:

> Thanks for the update, John! Comments below...
> 
> On Wed, May 9, 2012 at 2:34 PM, John Rose <john.r.rose at oracle.com> wrote:
>> In JDK 7 FCS a method handle is represented as a chain of argument transformation blocks, ending in a pointer to a methodOop.  The argument transformations are assembly coded and work in the interpreter stack.  The reason this is not outrageously slow is that we vigorously inline method handle calls whenever we can.  But there is a performance cliff you can drop off of, when you are working with non-constant MHs.  (BTW, invokedynamic almost always inlines its target.)  Project Lambda needs us not to drop off of this cliff.
> 
> And I need you to not drop off that cliff too! It's very easy to
> trigger...just make a method big enough, and AAAAAAAARRGH into the pit
> you go.

Adapting my favorite grook (Piet Hein):
  There's an art to knowing when:
  Never try to guess.
  Code until it stalls and then,
  Twenty methods less.
(Ref: http://www.archimedes-lab.org/grooks.html )

Seriously, we are adjusting inlining strategy, and using JRuby benchmarks to check.

> Luckily, for the ambitious early-access JRuby users running JRuby
> master + Java 7u2+ in production, the code they're hitting is all
> small enough to avoid the cliff, but with JRuby 1.7 preview release
> coming out in a couple weeks more people are going to start trying
> things out.

Is "richards" big enough to be an interesting benchmark?

> It also creates some *epic* stack traces when it blows up. Will those
> fold away in the future?

Yes, by a combination of hiding and tail-calling.
> 
> I have been tossing numbers and benchmarks back and forth with
> Christian, and now testing a local build of the meth-lazy stuff
> myself. Numbers haven't been great, but I think Christian made great
> progress today (based on an email showing C1 + indy beating C1 without
> indy and drastically beating C1 + indy in a stock u6 build that falls
> off the cliff). It's very exciting!

Thanks for rejoining the adventure.  We're doing our best to make it even more exciting.

> I assume this is the problem Christian described to me, where it was
> calling back into the interpreter to fix up the arguments?

Yes.  Problem in a nutshell:  When dropping from MH.invokeExact(a,b,c) on a direct method handle down to target(a,b,c) (i.e., the target method of the MH), we need to delete MH from the argument list, leaving all other arguments unchanged.  This is easy in the interpreter:  Just blindly jump into the target, ignoring the MH deep in the JVM argument stack.  It is hard in the compiled code; as with C calling sequences, leading arguments are given registers and extra trailing arguments can be safely ignored.  Removing the leading MH from the compiled calling sequence is tricky, and we were bailing out of this problem by calling a C2I adapter, ignoring the MH (deep in the unpacked interpreter stack), and going back to the I2C adapter of the target method.  This caused brutal data motion for many out-of-line MH calls.  We have a fix now, which is to drop the MH in the compiled caller's outgoing argument list (a,b,c), in certain carefully controlled circumstances.  With care this can be done even when the target method is a variable.  I'll refer to the source code for further details. :-)

>> For B. we are currently working on bootstrap issues.  The idea here is that, while we can do escape analysis, etc., a cleaner data structure will make the compiler succeed more often.
> 
> I will be *thrilled* when EA works across indy call sites. We have
> started work on our new compiler, which uses a simpler intermediate
> representation and which will be indy-only from day 1. Already we're
> seeing gains since we don't have to hand-write all the different call
> paths we want to represent; we can wire up any combinations of
> arguments, handles, and target using only method handles. That means
> we do things that will be ripe for EA like:
> 
> * Allocating heap storage for closures right next to the closure creation
> * Passing closures as a handle rather than as an opaque, polymorphic structure
> * Specializing closure-receiving code in *our* compiler until Hotspot
> can specialize it for us

If there is a convenient moment to do so, write some micro-benchmarks that exercise these code patterns, and we can use them to test our inlining, EA, and constant propagation.

> I'd be very surprised if we can't approach Java performance for the
> *general* cases of Ruby code by end of year, and if we can specialize
> closure-receiving code *and* get EA, we might be able to compete with
> Java 8 lambda performance for Ruby's closures too.
> 
> We also have our own profiling, inlining, and so on...but that's all
> above the level of bytecode to work around as-yet-unoptimized patterns
> in Hotspot. :)

We'll get there…  My ideal is to have layer-appropriate profiling (not necessarily inlining) at each layer, with enough optimization so that up-stack profiling decisions are exploited as thoroughly as internal JVM decisions.

> I have not been able to stump Chris with any NCDFEs lately, so that's
> good. But I do have some hacks in place to prevent them I can't remove
> until the new logic solidifies a bit.

That's fair.  Those NCDFEs are kind of spooky; I will be glad when it's 1 year past the last known occurrence.

> Now that the logic has started to land, I'm going to do some
> benchmarking and assembly-reading of my own to help from my end. And
> hopefully there's a chance I'll be able to help more directly over the
> summer.

Great.

> Very exciting stuff...I'm thrilled that dynlangs and indy are being
> taken so seriously. I told a couple thousand people at JAX 2012 how
> strongly I believe that indy is the most important work happening on
> the JVM right now, and I'm looking forward to doing more and more with
> it :)

Your enthusiastic and patient adoption of this technology has helped to foster it; many thanks.

Best,
— John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20120510/b8f3d677/attachment.html