RFE (m) (Prelminary): JDK-7197666: java -d64 -version core dumps in a box with lots of memory

[Coleen Phillimore]

Bengt,

This is a nice way to solve this, although don't know why you don't just 
use mmap for all the task queues.   You don't allocate and deallocate 
them a lot, do you?

You can also use the trick in globals.hpp:

*+   product(uintx, ArrayAllocatorMallocLimit, SOLARIS_ONLY(64K) NOT_SOLARIS(MAX_UINT),  \*
*+           "Allocation less than this value will be allocated "              \*
*+           "using malloc. Larger allocations will use mmap.")*


And include both solaris sparc and x86 in the size calculation (since 
sparc malloc has the same problem as x86 just not as bad), and save 
editing all the globals_<os_cpu>.hpp files.

Coleen

On 03/28/2013 06:09 PM, Bengt Rutisson wrote:
>
> Hi all,
>
> Sending this to both runtime and GC since I think it concerns both areas.
>
> I'd like some feedback on this preliminary change. I still want to do 
> some more testing and evaluation before I ask for final reviews:
>
> http://cr.openjdk.java.net/~brutisso/7197666/webrev.00/
>
> In particular I would like some feedback on these questions:
>
> - I am adding a flag that has the same value on all platforms except 
> Solaris x86. There is the product_pd flag macro to support this. But 
> there is no experimental_pd marcro. I would have preferred to make my 
> new flag experimental. Should I add experimental_pd or should I just 
> use a product flag?
>
> - Even with product_pd I think I still have to go in to all the 
> different platform files and add the exact same code to give the flag 
> a default value on all platforms. Is there a way to have a default 
> value and only override it on Solaris x86?
>
> - The class I am adding, ArrayAllocator, wants to choose between doing 
> malloc and mmap. Normally we use ReservedSpace and VirtualSpace to get 
> mapped memory. However, those classes are kind of clumsy when I just 
> want to allocate one chunk of memory. It is much simpler to use the 
> os::reserve_memory() and os::commit_memory() methods directly. I think 
> my use case here motivate using these methods directly, but is there 
> some reason not to do that?
>
> Some background on the change:
>
> The default implementation of malloc on Solaris has several limitation 
> compared to malloc on other platforms. One limitation is that it can 
> only use one consecutive chunk of memory. Another limitation is that 
> it always allocates in this single chunk of memory no matter how large 
> the requested amount of memory is. Other malloc implementations 
> normally use mapped memory for large allocations.
>
> The Java heap is mapped in memory and we try to pick a good address 
> for it. The lowest allowed address is controlled by 
> HeapBaseMinAddress. This is only 256 MB on Solaris x86 (other 
> platforms have at least 2 GB). Since the C heap ends up below the Java 
> heap it means that in some cases it is limited to 256 MB.
>
> When we run with ParallelOldGC we get three task queues per GC thread. 
> Each task queue takes mallocs 1MB. The failing machine in the bug 
> report has lots of CPUs and ends up with 83 GC threads. This is 249 
> MB, which is more than we can get out of the 256 MB limited C heap 
> considering that there are other things that get malloced too.
>
> So, the problems occur mostly on Solaris x86. My suggested fix tries 
> to address this by letting the task queues be mapped instead of 
> malloced on Solaris x86. Instead of inlining this logic in 
> taskqueue.cpp I added a more general class. The reason for this is 
> that I think we need to use the same logic in more places, especially 
> for G1, which is mallocing quite a lot.
>
> Since I think malloc on other platforms use mapped memory for large 
> malloc requests I think it is enough for this change to have effect on 
> Solaris. The other platforms probably have better heuristics than I 
> can come up with for which sizes should be mapped. On Sparc we have 
> the same limitation with malloc, but we have more memory available for 
> the C heap. This is why I have only enabled this for Solaris x86.
>
> Also, I will be on vacation for a few days. Back in the office 
> Thrusday April 4. I'm happy for any feedback on this, but if I don't 
> respond until next week you know why :)
>
> Thanks,
> Bengt

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/attachments/20130401/a87fe754/attachment.html