Troublesome reflection-cached SoftReferences

[Charles Oliver Nutter]

Y. Srinivas Ramakrishna wrote:
> You can try -XX:SoftRefLRUPolicyMSPerMB=0 , which is kind of a sledgehammer
> to clear soft refs the first time GC sees them (making them behave, in 
> essence, like weak references)

Yes, definitely a sledgehammer. I knew about this flag, but since we're 
dealing with JRuby deploying to GlassFish it would be unwise for us to 
start monkeying with softref LRU policy.

> Could it be the case that your soft ref is accessed very frequently, so 
> never falls below the computed
> recency threshold?

In this case, no...I do see it eventually disappear, but it takes some 
time. We're talking about a JRuby applications deployed into GlassFish, 
so it's probably around 100MB of memory. I believe the default for 
SoftRefLRUPolicyMSPerMB is 1000ms, so we'd be talking about 100 seconds 
before the heap collects. However, if another undeploy/redeploy happens 
in those 100 seconds and we go up to, say, 150MB, does that mean all 
soft references would have to be unused for 150 seconds? And then if 
another undeploy/redeploy happens? I think this is the case we're 
running into with "agile" developers doing more rapid redeployments.

> In any event, the JVM spec guarantees that all soft refs that are not 
> strongly reachable
> will be cleared before an OOM is issued., I trust your issue is one of 
> performance badness
> on account of a large heap, not an OOM?

Actually it is an OOM problem.

The problem is that on repeated redeploys, the memory continues to grow 
because these SoftReferences are holding on to a large graph of data. 
Some of this data holds also to class references, and so rather than 
getting a general heap OOM we get a PermGen OOM. Should a PermGen OOM 
force softly-reachable objects to be collected as well?

- Charlie