AnonymousClassLoader in JRuby

[Charles Oliver Nutter]

Ok, reading back over some notes and anonk.txt I think I understand a 
bit better. Tell me if I'm understanding things right.

java.dyn.AnonymousClassLoader is not a java.lang.ClassLoader itself. It 
bypasses the normal classloading chain by calling the new Unsafe method 
that allocates and loads a class directly. Given a byte[] and a host 
class, you can use AnonymousClassLoader to simply load a class; you can 
also use it to produce patched versions of a class, reducing the 
metadata necessary for each new class loaded by only altering the 
constant pool.

The classes are not associated with a classloader, and so the normal 
overhead we face of having a classloader-per-class (aggravated by a 
class-per-method in JRuby) is reduced.

 From an API perspective, we've largely achieved the "GCable class 
loading" concept in JRuby out of necessity. But on JDK6 and lower we pay 
the cost of having all those classes holding on to a clasloader in memory.

I have been unable to see an improved memory cost, but it's possible the 
actual cost of those classloaders in this test is outweighed by other 
JRuby runtime metadata associated with a given JITed method. Perhaps a 
simple Java test that loads 10_000 really simple classes via a 
OneShotClassLoader and via AnonymousClassLoader would be a better way to 
measure this?

It's also possible that the ClassCache concept would be useful/important 
to include or publish along with AnonymousClassLoader, since I would 
expect that most people loading classes anonymously will want a way to 
cache and retrieve already-loaded versions of code they're trying to 
load. This also sees like it might be necessary since anonymous classes 
will be otherwise impossible to look up, forcing even the same code in 
different places to instantiate a new class. Any thoughts on this? 
ClassCache (unpatched) in its entirety is here:

http://svn.codehaus.org/jruby/trunk/jruby/src/org/jruby/util/ClassCache.java

FYI, in JRuby, the key we use is generated by rendering the Ruby AST 
into a string containing a sexp representation of the code. This allows 
multiple instances of JRuby using the same class cache to share each 
others anonymously-loaded method bodies.

- Charlie

Charles Oliver Nutter wrote:
> I've managed to wire the AnonymousClassLoader into JRuby. A patch is 
> attached, that will probably be rather confusing to anyone unfamiliar 
> with JRuby.
> 
> ClassCache is basically a smart class-loading utility that can be used 
> to share loaded classes across JRuby instances without keeping hard 
> references to them and preventing them from unloading.
> 
> AnonymousClassLoader is an all-static utility class that wraps the logic 
> of instantiating "one-shot" classloaders for loading bodies of code. In 
> this case, it either uses java.dyn.AnonymousClassLoader or our own 
> "OneShotClassLoader" which is not anonymous but which is used for only a 
> single class.
> 
> The code generally appears to work about the same with 
> java.dyn.AnonymousClassLoader available, which is good. But I'm having 
> trouble quantifying the benefit to JRuby. I'd like to be able to show 
> how it helps, but memory profiles look practically the same between the 
> version using an OneShotClassLoader per method body and the version 
> using AnonymousClassLoader per method body. I have not done a lot of 
> work to isolate the cost of loading, but it seems to be unnoticeable in 
> my simple benchmark which defines 100_000 Ruby methods and forces them 
> to JIT.
> 
> So the runtime benefits may not be so great. The practical benefits, 
> such as being able to chuck byte[] into AnonymousClassLoader without 
> decorating class names and ensuring uniqueness, I have not yet utilized 
> in this code. Those benefits may show how writing class caches like the 
> one in JRuby are made a lot easier. For the moment, in order to allow 
> JRuby to support both MLVM work and pre-JDK7 JVMs, this patch still does 
> name-mangling to ensure methods are unique.
> 
> I think part of my confusion is that originally I desired a class loader 
> for which I could have a single instance I would throw *many* byte[] at, 
> and they'd all be loaded using that classloader but without hard 
> references and without the overhead of a classloader-per-method. But the 
> AnonymousClassLoader interface appears to one exactly one byte[] per 
> instance, though it does have considerably reduced in-memory cost per 
> AnonymousClassLoader instance.
> 
> Basically, what I've been looking for to make my life easier and memory 
> costs lower is:
> 
> ClassLoader cl = someClassLoader....;
> Class first = cl.defineClass(firstClass);
> Class second = cl.defineClass(secondClass);
> first = null; // and at some point I would expect the class to GC
> 
> Is this the purpose of AnonymousClassLoader? Am I doing it wrong? I'll 
> be poking around for example code now.
> 
> - Charlie
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev