Request for reviews (M): 7013538: Java memory leak with escape analysis

[Vladimir Kozlov]

Tom Rodriguez wrote:
> Any reason not to completely replace VectorSetI with your new implementation?  Otherwise I think it's ok.

I thought someone may want to keep old code and
if I remove it they will complain :)
I would be glad to remove it if nobody objects.

Thanks,
Vladimir

> 
> tom
> 
> On Feb 4, 2011, at 11:38 AM, Vladimir Kozlov wrote:
> 
>> http://cr.openjdk.java.net/~kvn/7013538/webrev
>>
>> Fixed 7013538: Java memory leak with escape analysis
>>
>> First, it is not leak. I instrumented VM around method
>> compilation call to see if there are allocations which
>> are not freed. There are none. I used mechanism similar
>> to NoHandleMark to catch direct calls to os::malloc()
>> (NoMallocMark). I found only one case (see below) but it
>> frees memory almost immediately. The rest (major part) of
>> allocations happened on thread Resource area and Compile
>> arena which return used Chunks to ChunkPool after each
>> compilation. But ChunkPool could become very large with EA.
>> It is cleaned each 5 sec by VM task but there could be
>> situations when 2 compilers threads request a lot of
>> memory during 5 sec period causing out of memory problems.
>>
>> So the problem is compilation with EA consumes C heap a lot
>> more than without it. Collected allocation statistics during
>> one hour of the test run (dacapo is running eclipse several
>> times) shows that used C Heap size with EA is around 500Mb
>> when without EA it is only 150Mb (not all of it is used,
>> it could be fragmented because of the problem 2 below).
>> Also numbers of calls to os::malloc rises from 2.5 millions
>> to 10.5 M.
>>
>> The main cause of native memory consumption is allocation
>> of GrowableArray and VectorSet in next very hot EA method:
>>
>> void ConnectionGraph::PointsTo(VectorSet &ptset, Node * n) {
>>  VectorSet visited(Thread::current()->resource_area());
>>  GrowableArray<uint>  worklist;
>>
>> And the cause of a lot calls to os::malloc() is VectorSetI
>> iterator which is used only by EA but it is used a lot,
>> it could also cause C heap fragmentation:
>>
>> SetI_ *VectorSet::iterate(uint &elem) const
>> {
>>  VSetI_ *foo = (new(ResourceObj::C_HEAP) VSetI_(this));
>>  elem = foo->next();
>>  return foo;
>> }
>>
>> The first problem is fixed by reusing structures in PointsTo().
>> I also added Reset() method to VectorSet to zero data array
>> instead of freeing it.
>>
>> The second problem is fixed by adding new simple VectorSetI
>> iterator (old iterator is renamed to VectorSetIter) which
>> is not based on SetI and doesn't need VSetI_.
>>
>> I also fixed compile time statistics for EA since it is now
>> part of Optimizer.
>