Fast class reloading

[David Holmes]

Igor Adam Klimer said the following on 06/01/10 18:13:
> I think updating existing instances won't be a problem - because we
> don't care for them :) Once we tested a particular mutation, we no
> longer need it and won't reference it again, so I think it doesn't
> matter if that instance gets into a "dirty" state. We want to
> "discard" that instance, reload the class with a new mutation and
> test the new instance. Am I thinking about this right or am I missing
> something? (of course, this is definitely not a safe approach, but
> some sacrifices are need for the speed gain we want to achieve)

I'm not sure how you can discard the existing instances easily, but if 
they are discard-able then that would make the job easier.

> About the modified JVM: would you reckon that the idea of using a
> Java Agent to modify the JVM "on the fly" would work (if it even
> makes sense :)), 

An agent can only use functionality exposed by the VM and there is no VM 
functionality in existence to do what you want. This has to be supported 
by the VM (after which it could be exposed to agents).

David

or would the extent of changes needed to be made to
> the JVM would be so large (or just beyond the capabilities of a Java
> Agent) that a custom build would be required? I understand that
> maintaning a custom VM would be easier than "patching" via a Java
> Agent, but the latter would be more user friendly. BTW, thanks for
> the tip about building custom VM from the OpenJDK sources :)
> 
> Best regards, Igor Klimer
> 
>> Hi Igor,
>> 
>> Gut feeling: you need a modified JVM to do this. If you only want
>> to add new methods then a mechanism for expanding the existing
>> vtable and codecache etc of the loaded class might not be too hard.
>> Adding fields is much harder as all existing objects need to be
>> modified in place to have the new field (or else you do some
>> serious augmentation to allow missing fields to be trapped and
>> handled in a different manner). Dealing with existing instances of
>> a class is what makes general hot-swapping very difficult.
>> 
>> By the way you can build any kind of custom VM you want from the 
>> OpenJDK sources.
>> 
>> Cheers, David Holmes
>> 
>> Igor Adam Klimer said the following on 06/01/10 10:37:
>>> Dear all, my name is Igor Klimer, I'm a 5th year student at the
>>> Wroclaw
>> University of Technology [1]. For my Master of Science thesis, I'm
>> working on improving the performance of a mutation testing tool my
>> colleagues are working on.
>>> Long story short - for each class you want to test, you end up
>> with hundreds (+/- it depends on the number of mutation operators
>> enabled) of "mutants" - slightly modified (by a mutation operator)
>> original class. This mutations (one mutation at a time, in one
>> place of the class at a time) range from simple arithmetic operator
>> changes ("-" becomes "+") to changing method signature. The problem
>> is that each of this mutation is just another version of the same
>> class, so I have to load each of them via a separate ClassLoader -
>> which has a very significant impact on the overall testing process.
>> 
>>> So my main job is to make the class (re)loading faster or even
>> better (?) take it out of the equation. Unfortunately, Java doesn't
>> support [2] dynamic class reloading which makes this problem
>> tougher (and more interesting ;)). I've looked into a number of
>> possible solutions:
>>> - HotSwap [3] offers great speed, but it can only modify the
>> body of a method. While that would speed up some/most of the 
>> mutation operators, the *most* interesting ones, the object- 
>> oriented ones, would be impossible with this method
>>> - many solutions involving proxy classes [4], etc. - while they
>> offer great flexibility, the speed penalty of modifying such a 
>> number of classes would probably nullify any speed improvements 
>> over normal classloading
>>> - creating a custom JVM - while this solution seems the most
>> elegant one (and offering the greatest possibilities), I'm afraid
>> it would void any chances of thie mutation testing tool becoming a
>> popular tool/practice. Another reason against this idea is that,
>> AFAIK, Sun does not allow creating custom JVM's based on their
>> code. A possible solution (and usually preferred for research
>> purposes) is to use the JikesRVM [5] - which is very
>> "research-friendly". However, since speed and compatibility is our
>> goal, JikesRVM is also not acceptable.
>>> One solution I came up with is to use a Java Agent [3] to
>> modify some key elements of the JVM so that class reloading would
>> be possible (and didn't have the restrictions of HotSwap). While I
>> understand that it's harder than it sounds, I believe (more like: I
>> pray :)) it is doable - of course, this solution would be highly
>> customized to our needs/situation - for example, we can be sure
>> that once a class/mutant is tested it can be safely
>> discarded/reloaded, we're sure no other part of the program is
>> going to reference it. It might come out a bit "hackish", but it's
>> the best I could come up with.
>>> So my question to you, dear Hotspot developers, is what are
>> your thoughts on this? Just your "gut feeling" is ok - does my idea
>> seem feasible? Any advice where to start hacking the JVM? I was
>> also wondering if you could point to some more information on the
>> internals of class loading - I found [6], [7] and [8] a very
>> interesting read (although they are too brief, IMHO), the source
>> code and comments there are also helpful, but I was wondering if
>> there's anything more you'd recommend :)
>>> Any thoughts on this matter would be greatly appreciated.
>>> 
>>> Best regards, Igor Klimer
>>> 
>>> [1] http://www.pwr.wroc.pl/en_main.xml [2]
>>> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4910812 [3]
>> http://java.sun.com/javase/6/docs/technotes/guides/instrumentation/index.html>[4]
>> M. Pukall et al, Flexible Runtime Program Adaptations in Java-A
>> Comparision,
>> http://www2.cs.uni-magdeburg.de/fin_media/downloads/forschung/technical_reports_und_preprints/2009/TechReport14-p-1591.pdf
>> 
>>> [5] http://jikesrvm.org/ [6]
>>> http://wikis.sun.com/display/HotSpotDev/Hom [7]
>>> http://wikis.sun.com/display/HotSpotInternals/Home [8]
>>> http://openjdk.java.net/groups/hotspot/docs/RuntimeOverview.html