max heap size for compressed oops

[Volker Simonis]

Hi Matthias,

the problem with compressed opps is that you usually want zero-based
compressed oops (there are other compressed oops and they help to save
some heap memory as well, but they usually behave worse from a
performance point of view).

But the problem of getting a 32GB chunk of virtual memory at a very
low address (near 0) is inherently platform and system dependent. So
if you configure your Java heap just a little too big, you may loose
the ability to use compressed oops which may suddenly lead to out of
memory situations compared to a slightly smaller heap with compressed
oops. That's because in the bigger heap you can now (i.e. without
compressed oops) store fewer Java objects (because the single Java
objects are bigger).

Using 30.5GB instead of 32 is probably just a heuristically found
value which should ensure you can run with compressed oops. But again
that's platform, system and even application dependent (e.g. if you
launch Java from a custom launcher which already reserves memory in
the low address space it may be much fewer memory which is available
for compressed oops).

Regards,
Volker


On Fri, Dec 11, 2015 at 4:13 PM, Matthias Wahl <matthiaswahl at m7w3.de> wrote:
> Thanks for your quick reply.
>
> This makes complete sense.
>
> It sheds some light on how compressed oops work.
> Unfortunately it does not answer my initial question.
>
> In the mean time i was able to figure out that we actually have
> compressed oops up to 32 GB:
>
>
> $ java -XX:+UnlockDiagnosticVMOptions -XX:+PrintCompressedOopsMode
> -Xmx32g -Xms32g -XX:+UseCompressedOops -version
> Java HotSpot(TM) 64-Bit Server VM warning: Max heap size too large for
> Compressed Oops
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>
> ----
>
> $ java -XX:+UnlockDiagnosticVMOptions -XX:+PrintCompressedOopsMode
> -Xmx31999m -Xms31999m -XX:+UseCompressedOops -version
>
> Protected page at the reserved heap base: 0x000000011e000000 / 2097152 bytes
>
> heap address: 0x000000011e200000, size: 30502 MB, Compressed Oops with
> base: 0x000000011e1ff000
>
> Narrow klass base: 0x00000008d6263000, Narrow klass shift: 0
> Compressed class space size: 1073741824 Address: 0x00000008d6263000 Req
> Addr: 0x0000000890800000
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>
> ----
>
> So I am still searching for a reason for the 30.5GB suggestion.
> But I think we can narrow it down to some GC (G1) performance hint.
>
> Does anybody have a clue on why heaps > 30.5 GB may induce really worse
> performance characteristics? Be it G1 or CMS Garbage Collector.
>
> Thank you!
>
> Just ignore this mail if you think it's not the right place for such
> questions.
>
>
> Matthias
>
>
> Am 12/9/15 um 6:11 PM schrieb Andrew Haley:
>> On 12/09/2015 04:04 PM, Matthias Wahl wrote:
>>> Do you know any reason for limiting the heap size to 30.5 GB instead of
>>> 32 GB?
>>
>> The interesting thing you can do is use -XX:+PrintAssembly and have a
>> look.  My guess is that it has to do with zero-based compressed OOPs:
>> if you can use zero as the base pointer you don't have to dedicate a
>> register to the job and the compressing/decompressing code is faster.
>>
>> A narrow OOP is computed by
>>
>>   oop_address == 0  ? 0 : (oop_address - heap_base) >> shift
>>
>> The OOP at 0 is a null, of course.
>>
>> If the heap base is zero, you can encode an OOP with a single
>> shift instruction:
>>
>>      lsr x0, x0, #3
>>
>> But if the heap base is not zero, encoding an OOP gets nasty:
>>
>>      subs x0, x0, xheapbase
>>      csel x0, x0, xzr, cs
>>      lsr  x0, x0, #3
>>
>> Here there are three instructions, one of them conditional.  In the
>> commonest case most modern processors can do the shifting in parallel
>> with some other operation, so the cost of compressed OOPs is often
>> zero as long as the heap base is zero.  But there's no way that the
>> three instructions used encoding an OOP with a nonzero heap base are
>> gong to cost nothing.
>>
>> So why is there a difference between 30.5G and 32G?  Because in
>> practice the JVM sits in the bottom 1.5G or so of memory and the heap
>> is immediately above it, so the heap base for compressed OOPs can be
>> zero.
>>
>> Does that make sense?
>>
>> Andrew.
>>