How high are he memory costs of polymorphic inline caches?

[Charles Oliver Nutter]

Hello, fellow implementer :-)

On Mon, Aug 18, 2014 at 6:01 AM, Raffaello Giulietti
<raffaello.giulietti at gmail.com> wrote:
> So, the question is whether some of you has experience with large scale
> projects written in a dynamic language implemented on the JVM, that makes
> heavy use of indy and PICs. I'm curious about the memory load for the PICs.
> I'm also interested whether the standard Oracle server JVM satisfactorily
> keeps up with the load.

JRuby has implemented call sites using this sort of PIC structure
since the beginning. I dare say we were the first.

Experimentally, I determined that the cost of a PIC became greater
than a non-indy, non-inlining monomorphic cache at about 5 deep. By
cost, I mean the overhead involved in dispatching fromthe caller to an
empty method....essentially just the cost of the plumbing.

Now of course that number's going to vary, but overall a small PIC
seems to have value...especially when you consider that the cost of
rebinding a call site is rather high.

We do have JRuby users using our indy logic in production, and none of
their concerns have had any relation to the PIC (usually, it's just
startup/warmup concerns).

> For example, we have a large Smalltalk application with about 50'000 classes
> and about 600'000 methods. In Smalltalk, almost everything in code is a
> method invocation, including operators like +, <=, etc. I estimate some 5-10
> millions method invocation sites. How many of them are active during a
> typical execution, I couldn't tell. But if the Smalltalk runtime were
> implemented on the JVM, PICs would quite certainly represent a formidable
> share of the memory footprint.

That's why you limit their size. If an inline cache behind a GWT takes
up N bytes in memory, a PIC based on the same invalidation logic
should just be N * X where X is the depth of your PIC.

Of course, you could do what we do in JRuby, and make the PIC depth
configurable to try out a few things.

> More generally, apart from toy examples, are there studies in real-world
> usage of indy and PICs in large applications?
> Perhaps some figures from the JRuby folks, or better, their users'
> applications would be interesting.

The only "studies" we have are the handful of JRuby users running with
indy enabled in production. They love the higher performance, hate the
startup/warmup time, and most of them had to either bump permgen up or
switch to Java 8 to handle the extra code generation happening in
indy.

I'm happy to answer any other questions.

- Charlie