G1 garbage collection Ext Root Scanning time increases due to Class.forName (Problem, Solution, and Patch)

[David Holmes]

Hi Ashley,

Is this Dictionary the systemDictionary in the VM or something to do 
with G1? If there is an issue with the VM's use of the systemDictionary 
then it needs to be fixed in the VM. If there is an issue in G1 then it 
needs to be fixed in G1. A change to the Java-level behaviour of 
Class.forName needs to pass extreme scrutiny and as Ioi points out it is 
not appropriate for forName to assume how a ClassLoader will load a 
class - it must defer to the classloader to do it. So the patch is not 
acceptable.

What exactly is G1 reporting and why do you think it is wrong? (I have 
no idea what "Ext root scanning" means.)

Thanks,
David

On 10/07/2013 8:17 AM, Ashley Taylor wrote:
> Hello
>
> We have an application that makes heavy use of an apache API that uses
> Class.forName.
>
> Using this method call wehave noticed is that over time the “Ext Root
> Scanning” aspect of the G1 garbage collector time increased linearly.
>
> Working with John Cuthbertson from the hotspot-gc mailing list we
> tracked this down to the Dictionary thread within the garbage collector.
>
> https://www.dropbox.com/s/7s7gishn8mxtote/G1Checker.java reproduces this
> issue, to see the issue run it with the following jvm options
>
> -XX:+PrintGCDetails -XX:+UseG1GC
>
> Watch the Ext Root Scanning time and a single thread will start to climb.
>
> We propose that when loading a class in a new classloader using
> Class.forName it will create a reference in the Dictionary table even
> when the class is present in the Dictionary table due to a parent
> classloader.
>
> We have created a patch
> https://www.dropbox.com/s/t7gepqsvksz77kv/Class.diff to the
> Class.forName method that fixes this bug by using the same approach as
> UrlClassLoader.loadClass
>
> which recursively tries to load the given class in its parent
> classloader first.
>
> The only potential change in behaviour that we could imagine is to how
> static objects within the class get referenced,
>
> have creating some unit tests have confirmed that the behaviour is
> identical after the patch.
>
> https://www.dropbox.com/s/wf8rfa53jl7o5zg/ClassLoaderTests.java
>
> https://www.dropbox.com/s/y5ir6x927niu2ax/Loaded.java
>
> We have been using this patch in production now for the last couple of
> month with no issues and would like to see it adopted so we can stop
> using a custom build.
>
> Cheers,
>
> *Ashley Taylor*
>
> Software Engineer
>
> Email:ashley.taylor at sli-systems.com <mailto:ashley.taylor at sli-systems.com>
>
> Website: www.sli-systems.com <http://www.sli-systems.com/>
>
> Blog: blog.sli-systems.com <http://blog.sli-systems.com/>
>
> Podcast: EcommercePodcast.com <http://ecommercepodcast.com/>
>
> Twitter: www.twitter.com/slisystems <http://www.twitter.com/slisystems>
>
> sli_logo_2011**
>