Metaspace - memory allocation
Thomas Stüfe
thomas.stuefe at gmail.com
Fri Feb 5 08:24:48 UTC 2016
Hi Timur,
On Fri, Feb 5, 2016 at 7:53 AM, Timur Akhmadeev <timur.akhmadeev at gmail.com>
wrote:
> Thanks Thomas and Jon!
> I think I've opened the page Thomas has referenced more than once before
> asking my question but somehow I missed this paragraph which is the answer
> :)
>
> If UseCompressedOops is turned on and UseCompressedClassesPointers is
>> used, then two logically different areas of native memory are used for
>> class metadata. UseCompressedClassPointers uses a 32-bit offset to
>> represent the class pointer in a 64-bit process as does UseCompressedOops for
>> Java object references. A region is allocated for these compressed class
>> pointers (the 32-bit offsets). The size of the region can be set with
>> CompressedClassSpaceSize and is 1 gigabyte (GB) by default. The space
>> for the compressed class pointers is reserved as space allocated by mmap at
>> initialization and committed as needed. TheMaxMetaspaceSize applies to
>> the sum of the committed compressed class space and the space for the other
>> class metadata.
>
>
> It's a bit unnatural to call Compressed Class pointers part of the
> Metaspace and at the same time allow its part to be set bigger than the
> whole space while not allowing Compressed Class pointers (hopefully) to
> grow beyond MaxMetaspaceSize.
>
>
I agree. Maybe it is just me, but I also have a hard time understanding the
relationship between those two parameters.
See also the mail thread here:
http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2015-June/013797.html
.
I do not know the history, so I only can guess that the permgen removal
came first, and that at first it was just a "simple" metaspace. But with
moving out class data from the heap we lost the ability to address them
with compressed oops (32bit pointers) and this feature was reintroduced in
the form of compressed class pointers, which made the implementation more
complicated and now we have those two switches.
But this is only a guess from my side. Whatever the reasons, I think the
parameters are confusing, especially as the sizes have two different
meanings - reserved size in case of CompressedClassSpaceSize, committed
size in case of MaxMetaSpaceSize.
I also do not know if CompressedClassSpaceSize is a hard limit - what
happens when we fill compressed class space with more than that? Will it
expand or throw an OOM?
Kind Regards, Thomas
> On Thu, Feb 4, 2016 at 10:02 PM, Jon Masamitsu <jon.masamitsu at oracle.com>
> wrote:
>
>>
>>
>> On 02/04/2016 06:41 AM, Thomas Stüfe wrote:
>>
>> Hi Timur,
>>
>> I believe you see the space for the compressed class data (Compressed
>> class space), which is by default 1GB and controlled by
>> -XX:CompressedClassSpaceSize . I think this area is semantically one
>> subarea of the metaspace, the other being non-class data.
>>
>> I am unsure about the difference between MaxMetaSpaceSize and
>> CompressedClassSpaceSize, but what I believe is that:
>>
>> - MaxMetaSpaceSize limits the amount of total committed (not reserved)
>> space
>> - CompressedClassSpaceSize sets the amount of space to be reserved (not
>> committed) for compressed classes only.
>>
>> This would fit your NMT output, which says:
>>
>> "Class (reserved=1142808KB, committed=106520KB)"
>>
>> so, ~100M are committed. With a MaxMetaSpaceSize=256M - what you
>> specified - you would not be able to commit the whole 1GB, but hit the
>> limit far earlier.
>>
>> See also:
>> https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/considerations.htm
>> (the section about UseCompressedClassesPointers)
>>
>> Please take this with a grain of salt. I am not sure I understand this
>> correctly and would value input from other developers.
>>
>>
>> Thomas,
>>
>> No disagreement from me. I think you described it
>> correctly. Note that if you turn off UseCompressedClassPointers,
>> there will not be a separate space for the class metadata.
>>
>> Jon
>>
>>
>>
>> Kind Regards, Thomas
>>
>> On Thu, Feb 4, 2016 at 2:13 PM, Timur Akhmadeev <
>> timur.akhmadeev at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I'm trying to figure out which parts of memory Hotspot can allocate in
>>> HugePages on Linux (appears to be heap and code cache by default, plus
>>> Metaspace with -XX:+UseLargePagesInMetaspace) and noticed something I can't
>>> explain.
>>>
>>> There's 1G allocation on the OS level in 4K pages that is apparently
>>> attributed to Class metadata, although Metaspace is maxed at a value much
>>> lower than 1G. Question: why?
>>>
>>> [oracle at oel6u7 bin]$ grep -B 11 'KernelPageSize: 4 kB'
>>> /proc/18906/smaps | grep '^Size:' | head
>>> Size: 4 kB
>>> Size: 4 kB
>>> Size: 2764 kB
>>> Size: 12288 kB
>>> Size: 1036288 kB
>>> Size: 128 kB
>>> Size: 4 kB
>>> Size: 4 kB
>>> Size: 4 kB
>>> Size: 8 kB
>>>
>>> [oracle at oel6u7 bin]$ /home/oracle/jdk1.8.0_72/bin/jcmd 18906
>>> VM.native_memory summary
>>> 18906:
>>>
>>> Native Memory Tracking:
>>>
>>> Total: reserved=2069703KB, committed=1032559KB
>>> - Java Heap (reserved=786432KB, committed=786432KB)
>>> (mmap: reserved=786432KB,
>>> committed=786432KB)
>>>
>>> - Class (reserved=1142808KB, committed=106520KB)
>>> (classes #13624)
>>> (malloc=2072KB #22966)
>>> (mmap: reserved=1140736KB,
>>> committed=104448KB)
>>>
>>> - Thread (reserved=56009KB, committed=56009KB)
>>> (thread #55)
>>> (stack: reserved=55512KB, committed=55512KB)
>>> (malloc=178KB #272)
>>> (arena=319KB #108)
>>>
>>> - Code (reserved=55407KB, committed=55407KB)
>>> (malloc=5487KB #7167)
>>> (mmap: reserved=49920KB, committed=49920KB)
>>>
>>> - GC (reserved=2600KB, committed=1748KB)
>>> (malloc=32KB #355)
>>> (mmap: reserved=2568KB, committed=1716KB)
>>>
>>> - Compiler (reserved=213KB, committed=213KB)
>>> (malloc=82KB #331)
>>> (arena=131KB #3)
>>>
>>> - Internal (reserved=4921KB, committed=4917KB)
>>> (malloc=4885KB #16137)
>>> (mmap: reserved=36KB, committed=32KB)
>>>
>>> - Symbol (reserved=17518KB, committed=17518KB)
>>> (malloc=14377KB #172159)
>>> (arena=3141KB #1)
>>>
>>> - Native Memory Tracking (reserved=3617KB, committed=3617KB)
>>> (malloc=149KB #2329)
>>> (tracking overhead=3468KB)
>>>
>>> - Arena Chunk (reserved=178KB, committed=178KB)
>>> (malloc=178KB)
>>>
>>> [oracle at oel6u7 bin]$ /home/oracle/jdk1.8.0_72/bin/jcmd 18906
>>> VM.native_memory detail | grep -A 10 'for Class'
>>> [0x0000000100000000 - 0x0000000140000000] reserved 1048576KB for Class
>>> from
>>> [0x00007f71950e2ad2] ReservedSpace::initialize(unsigned long,
>>> unsigned long, bool, char*, unsigned long, bool)+0xc2
>>> [0x00007f71950e2d0b] ReservedSpace::ReservedSpace(unsigned long,
>>> unsigned long, bool, char*, unsigned long)+0x1b
>>> [0x00007f7194ec8300]
>>> Metaspace::allocate_metaspace_compressed_klass_ptrs(char*, unsigned
>>> char*)+0x40
>>> [0x00007f7194eca7df] Metaspace::global_initialize()+0x4cf
>>>
>>> [0x0000000100000000 - 0x0000000100c00000] committed 12288KB from
>>> [0x00007f71950e2549] VirtualSpace::expand_by(unsigned long,
>>> bool)+0x199
>>> [0x00007f7194ec6766]
>>> VirtualSpaceList::expand_node_by(VirtualSpaceNode*, unsigned long, unsigned
>>> long)+0x76
>>> [0x00007f7194ec9580] VirtualSpaceList::expand_by(unsigned
>>> long, unsigned long)+0xf0
>>> [0x00007f7194ec9713]
>>> VirtualSpaceList::get_new_chunk(unsigned long, unsigned long, unsigned
>>> long)+0xb3
>>> ...
>>>
>>> [oracle at oel6u7 bin]$ head ../logs/gc.log
>>> Java HotSpot(TM) 64-Bit Server VM (25.72-b15) for linux-amd64 JRE
>>> (1.8.0_72-b15), built on Dec 22 2015 22:00:07 by "java_re" with gcc 4.3.0
>>> 20080428 (Red Hat 4.3.0-8)
>>> Memory: 4k page, physical 3082784k(506980k free), swap 1257468k(1148276k
>>> free)
>>> CommandLine flags: -XX:InitialCodeCacheSize=50331648
>>> -XX:InitialHeapSize=49324544 -XX:MaxHeapSize=805306368
>>> -XX:MaxMetaspaceSize=268435456 -XX:MetaspaceSize=268435456
>>> -XX:NativeMemoryTracking=detail -XX:+PrintGC -XX:+PrintGCTimeStamps
>>> -XX:ReservedCodeCacheSize=50331648 -XX:+UseCompressedClassPointers
>>> -XX:+UseCompressedOops -XX:+UseLargePages -XX:+UseLargePagesInMetaspace
>>>
>>> --
>>> Regards
>>> Timur Akhmadeev
>>>
>>> _______________________________________________
>>> hotspot-gc-use mailing list
>>> hotspot-gc-use at openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>>
>>>
>>
>>
>> _______________________________________________
>> hotspot-gc-use mailing listhotspot-gc-use at openjdk.java.nethttp://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>
>>
>>
>> _______________________________________________
>> hotspot-gc-use mailing list
>> hotspot-gc-use at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>
>>
>
>
> --
> Regards
> Timur Akhmadeev
>
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20160205/3a73d21f/attachment.html>
More information about the hotspot-gc-use
mailing list