RFR(XS): 8193521: glibc wastes memory with default configuration

[Doerr, Martin]

Hi,

the behavior of the glibc can easily be traced by gdb.
I have set a breakpoint in "mmap" with cond "len>10000000" and then looked at the stack traces.

The first malloc in each new thread triggers a 128 MB mmap. This typically seems to be the initialization of thread local storage. (Other mallocs may be hidden in libraries.)

#0  __mmap (addr=addr at entry=0x0, len=len at entry=134217728, prot=prot at entry=0, flags=flags at entry=16418, fd=fd at entry=-1, offset=offset at entry=0) at ../sysdeps/unix/sysv/linux/wordsize-64/mmap.c:33
#1  0x00007ffff72403d1 in new_heap (size=135168, size at entry=2264, top_pad=<optimized out>) at arena.c:438
#2  0x00007ffff7240c21 in _int_new_arena (size=24) at arena.c:646
#3  arena_get2 (size=size at entry=24, avoid_arena=avoid_arena at entry=0x0) at arena.c:879
#4  0x00007ffff724724a in arena_get2 (avoid_arena=0x0, size=24) at malloc.c:2911
#5  __GI___libc_malloc (bytes=24) at malloc.c:2911
#6  0x00007ffff7de9ff8 in allocate_and_init (map=<optimized out>) at dl-tls.c:603
#7  tls_get_addr_tail (ti=0x7ffff713e100, dtv=0x7ffff0038890, the_map=0x6031a0) at dl-tls.c:791
#8  0x00007ffff6b596ac in Thread::initialize_thread_current() () from openjdk10/lib/server/libjvm.so

The VM typically starts more than 20 threads even when only using java -version so we mmap several useless GB.

Best regards,
Martin


-----Original Message-----
From: Andrew Dinn [mailto:adinn at redhat.com] 
Sent: Freitag, 15. Dezember 2017 10:56
To: Andrew Haley <aph at redhat.com>; Doerr, Martin <martin.doerr at sap.com>; hotspot-runtime-dev at openjdk.java.net
Subject: Re: RFR(XS): 8193521: glibc wastes memory with default configuration

On 14/12/17 19:06, Andrew Haley wrote:
> On 14/12/17 18:51, Doerr, Martin wrote:
>> And I also assume that most JVM users don't know that the JVM has own arena implementation etc. which makes glibc arenas under the hood useless for these kind of allocations. 
>> So I think it would be nice if we could help these people.
> 
> So, here's a radical thought: why are we using malloc() at all?
> Shouldn't we be calling mmap() and then basing our arena in that?
The JVM does that almost all of the time. My guess is that the
difference Martin is seeing is down to malloc operations done outside
libjvm.so

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander