the amazing tales of the search for the invisible man! or, where's my gc root

[Tony Printezis]

OK, I'll bite.

When you say: "a large section of memory (a plugin framework)" do you 
mean only objects in the young / old gen, or also classes in the perm gen?

How do you know that said memory is not being reclaimed? Do you 
eventually get an OOM?

Given that it happens with two different JVMs (I assume you use HotSpot 
on Linux and Mac, as well as the IBM JDK), it's unlikely to be a GC bug, 
as both JVMs would need to have the same bug. Not impossible, but 
unlikely, IMHO.

Tony

Jed Wesley-Smith wrote:
> all,
>
> I am writing to this list in some desperation hoping for some expert 
> advice. We (the JIRA development team at Atlassian) have been hunting 
> memory leaks for some weeks and in the process have tracked down and 
> removed every possible reference to a large section of memory (a 
> plugin framework) that we could find. Starting with all strong 
> references and proceeding to remove soft and weak references - even 
> things like clearing the java.lang.reflect.Proxy cache - and even 
> Finalizer references until both YourKit, Eclipse MAT, JProfiler and 
> jhat all report that the memory in question is dead and should be 
> collectable, but inexplicably _the JVM still holds on to it_. There 
> are no JNI Global references either, yet this memory remains 
> uncollectable!
>
> This happens for the 1.5 and 1.6 JVMs on Linux and Mac, and the IBM 
> 1.6 JDK on Linux.
>
> So my question is, how on earth do I search for what is referencing 
> this uncollectable memory? Are there any other tools that can help 
> find why this memory is not collected? Can I query the VM directly 
> somehow?
>
> I fear this is a JVM GC bug as no known memory analysis tool can find 
> the heap root (i.e. according to "the rules" there is no heap root). 
> Are there any known GC memory leaks caused by ClassLoaders being 
> dropped for instance?
>
> The application is creating and disposing of a lot of ClassLoaders via 
> OSGi (Apache Felix) with Spring OSGi. It creates a lot of 
> java.lang.reflect.Proxy class instances.
>
> We have written this up and added an example heap dump here:
> http://jira.atlassian.com/browse/JRA-16932
>
> Having come to the end of our tethers here, if anyone can help in any 
> way it would be massively appreciated.
>
> cheers,
> Jed Wesley-Smith
> JIRA Team @ Atlassian

-- 
---------------------------------------------------------------------
| Tony Printezis, Staff Engineer   | Sun Microsystems Inc.          |
|                                  | MS UBUR02-311                  |
| e-mail: tony.printezis at sun.com   | 35 Network Drive               |
| office: +1 781 442 0998 (x20998) | Burlington, MA 01803-2756, USA |
---------------------------------------------------------------------
e-mail client: Thunderbird (Linux)