TCCL memory leak in RMI registry creation

[Mark Thomas]

On 21/04/2016 23:32, Stuart Marks wrote:
> Hi Mark,
> 
> Is it possible to save the reference to the registry that was created by
> the call to LocateRegistry.createRegistry()? If so, then it's possible
> to shut down that registry by exporting it:
> 
>     UnicastRemoteObject.unexportObject(registry, true);
> 
> Adding this to the "RegistryLeak" test program you posted changes its
> behavior to print "no leak". (I haven't looked at this too closely,
> though, so I may be misinterpreting what's going on.)

Excellent. Thank you very much for that. It does indeed work.

> It's probably necessary to unexport and maybe also unbind everything
> from this registry as well, as Roger had suggested previously.

For the record, if the module/application creates the registry,
unexporting the registry is sufficient. However, an additional GC cycle
is required to clean up the references if unexport and unbind are not
called.

The good news is that, with the information above, this leak is
something that modules/applications can and should do something about.

This moves the problem to how to detect when modules/applications fail
to clean up an RMI Registry they created. I'd like to be able to provide
nice loud error messages when this happens. Open questions are:
- How to get a list of RMI registries?
- How to determine the TCCL for a registry (held in the ccl field of the
  associated sun.rmi.transport.Target object) so I can figure out if a
  module/application created it?

The additional GC cycle required if objects in the registry are not
unexported has proven problematic in the past. Ideally, I'd like to
avoid that. The raises an other question.

Given a Registry where you haven't retained a reference to the exported
objects, is there any way to unexport those objects?

Thanks for all the great pointers provided so far in this thread.

I've updated and expanded the demonstration code in the GitHub repo
based on the new information.

Many thanks,

Mark

> 
> s'marks
> 
> On 4/21/16 6:53 AM, Mark Thomas wrote:
>> Continuing the previous thread but with a more precise subject.
>>
>> Calling
>>
>> LocateRegistry.createRegistry(Registry.REGISTRY_PORT);
>>
>> is sufficient to pin the Thread Context Class Loader (TCCL) in memory
>> with no public API available (that I can find) to remove the reference.
>>
>> This is a problem for modular environments like Java EE containers that
>> can frequently load and unload modules since it will trigger a memory
>> leak. I have made a simple Java class that demonstrates this issue
>> available on GitHub. [1]
>>
>> Up to and including Java 8, it is possible to use reflection tricks [2]
>> to a) identify these leaks and b) fix them. As of Java 9, these
>> reflection tricks will be blocked.
>>
>> Being able to identify and fix these memory leaks has been extremely
>> useful in resolving issues with memory leaks with multiple deployments
>> in Java EE environments.
>>
>> I believe that equivalent functionality needs to be available in Java 9
>> onwards. I have no strong views on what that functionality should look
>> like as long as it enables the leaks to be identified and, ideally,
>> resolved.
>>
>> Looking at the source code for sun.rmi.transport.Target, the ccl field
>> was added for good reason and needs to be retained. That suggests that
>> the solution is likely to involve destroying a currently existing
>> registry. While Registry.destroy() (or similar) would be sufficient for
>> application developers to do the 'right thing' it would not be
>> sufficient for container developers to identify that an application
>> running on the container had done the 'wrong thing' and needed to be
>> fixed. For that something along these lines would be required:
>>
>> LocateRegsitry.listRegistries()
>> Registry.getThreadContextClassLoader()
>> Registry.destroy()
>>
>> Assuming my analysis is correct, thoughts on the above solution or any
>> alternative solution welcome. For example, a way to hack around the Java
>> 9 restrictions would be sufficient.
>>
>> Thanks,
>>
>> Mark
>>
>>
>> [1]
>> https://github.com/markt-asf/memory-leaks/blob/master/src/org/apache/markt/leaks/rmi/RegistryLeak.java
>>
>>
>> [2]
>> http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/catalina/loader/WebappClassLoaderBase.java?view=annotate#l2214
>>
>>