Request for reviews (M): 7013538: Java memory leak with escape analysis

[Vladimir Kozlov]

OK. Thanks, Tom

Vladimir

Tom Rodriguez wrote:
> On Feb 4, 2011, at 4:04 PM, Vladimir Kozlov wrote:
> 
>> Tom Rodriguez wrote:
>>> Any reason not to completely replace VectorSetI with your new implementation?  Otherwise I think it's ok.
>> I thought someone may want to keep old code and
>> if I remove it they will complain :)
>> I would be glad to remove it if nobody objects.
> 
> Who's going to object on Friday at 5?  ;)  I'd say remove it.  It doesn't really match hotspot style anyway which is why we got into trouble.
> 
> tom
> 
>> Thanks,
>> Vladimir
>>
>>> tom
>>> On Feb 4, 2011, at 11:38 AM, Vladimir Kozlov wrote:
>>>> http://cr.openjdk.java.net/~kvn/7013538/webrev
>>>>
>>>> Fixed 7013538: Java memory leak with escape analysis
>>>>
>>>> First, it is not leak. I instrumented VM around method
>>>> compilation call to see if there are allocations which
>>>> are not freed. There are none. I used mechanism similar
>>>> to NoHandleMark to catch direct calls to os::malloc()
>>>> (NoMallocMark). I found only one case (see below) but it
>>>> frees memory almost immediately. The rest (major part) of
>>>> allocations happened on thread Resource area and Compile
>>>> arena which return used Chunks to ChunkPool after each
>>>> compilation. But ChunkPool could become very large with EA.
>>>> It is cleaned each 5 sec by VM task but there could be
>>>> situations when 2 compilers threads request a lot of
>>>> memory during 5 sec period causing out of memory problems.
>>>>
>>>> So the problem is compilation with EA consumes C heap a lot
>>>> more than without it. Collected allocation statistics during
>>>> one hour of the test run (dacapo is running eclipse several
>>>> times) shows that used C Heap size with EA is around 500Mb
>>>> when without EA it is only 150Mb (not all of it is used,
>>>> it could be fragmented because of the problem 2 below).
>>>> Also numbers of calls to os::malloc rises from 2.5 millions
>>>> to 10.5 M.
>>>>
>>>> The main cause of native memory consumption is allocation
>>>> of GrowableArray and VectorSet in next very hot EA method:
>>>>
>>>> void ConnectionGraph::PointsTo(VectorSet &ptset, Node * n) {
>>>> VectorSet visited(Thread::current()->resource_area());
>>>> GrowableArray<uint>  worklist;
>>>>
>>>> And the cause of a lot calls to os::malloc() is VectorSetI
>>>> iterator which is used only by EA but it is used a lot,
>>>> it could also cause C heap fragmentation:
>>>>
>>>> SetI_ *VectorSet::iterate(uint &elem) const
>>>> {
>>>> VSetI_ *foo = (new(ResourceObj::C_HEAP) VSetI_(this));
>>>> elem = foo->next();
>>>> return foo;
>>>> }
>>>>
>>>> The first problem is fixed by reusing structures in PointsTo().
>>>> I also added Reset() method to VectorSet to zero data array
>>>> instead of freeing it.
>>>>
>>>> The second problem is fixed by adding new simple VectorSetI
>>>> iterator (old iterator is renamed to VectorSetIter) which
>>>> is not based on SetI and doesn't need VSetI_.
>>>>
>>>> I also fixed compile time statistics for EA since it is now
>>>> part of Optimizer.
>