PING: RFR: 8217432: MetaspaceGC::_capacity_until_GC exceeds MaxMetaspaceSize

[Yasumasa Suenaga]

PING: Could you review this?


Yasumasa


On 2019/01/21 17:52, Yasumasa Suenaga wrote:
> Hi all,
> 
> I filed this issue to JBS:
>    https://bugs.openjdk.java.net/browse/JDK-8217432
> 
> I investigated it, and I found that `minimum_desired_capacity` and
> `maximum_desired_capacity` in `MetaspaceGC::compute_new_size()`
> might exceed MaxMetaspaceSize.
> They shouldn't exceed MaxMetaspaceSize.
> 
> I uploaded webrev for this bug. Could you review it?
>    http://cr.openjdk.java.net/~ysuenaga/JDK-8217432/webrev.00/
> 
> This change has passed all tests on submit repo, :vmTestbase_vm_metaspace,
> and :vmTestbase_nsk_sysdict jtreg tests.
> 
> 
> Thanks,
> 
> Yasumasa
> 
> 
> On 2019/01/15 18:30, Yasumasa Suenaga wrote:
>> PING: Did you read my email?
>>
>> I wrote the problem and suggested fix in it.
>> Please tell me if you have question(s).
>>
>> I can file it to JBS if it is a bug.
>>
>>
>> Thanks,
>>
>> Yasumasa
>>
>>
>> On 2019/01/10 18:02, Yasumasa Suenaga wrote:
>>> Hi all,
>>>
>>> I'm Yasumasa Suenaga (OpenJDK reviewer: ysuenaga)
>>>
>>> My customer uses OpenJDK 8u131 with following commandline arguments:
>>>
>>>    -XX:+CMSClassUnloadingEnabled -XX:CMSInitiatingOccupancyFraction=80
>>>    -XX:CompressedClassSpaceSize=128m -Xms4500m -Xmx4500m
>>>    -XX:MaxMetaspaceSize=256m -Xmn=768m -XX:MaxTenuringThreshold=15
>>>    -XX:OnOutOfMemoryError="/bin/kill -ABRT %p"
>>>    -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseConcMarkSweepGC
>>>
>>> Their system has encountered OutOfMemoryError which is caused by Metaspace.
>>>
>>> I checked core image which was collected by OnOutOfMemoryError, and I saw
>>> strange values in it as below:
>>>
>>> -------------------
>>> (gdb) p Metaspace::_space_list->_reserved_words
>>> $67 = 31195136
>>> (gdb) p Metaspace::_class_space_list->_reserved_words
>>> $68 = 16777216
>>> (gdb) p MetaspaceGC::_capacity_until_GC
>>> $75 = 448045056
>>> -------------------
>>>
>>> MaxMetaspaceSize and CompressedClassSpaceSize seem to affect Metaspace.
>>> However, MetaspaceGC::_capacity_until_GC exceeds MaxMetaspaceSize.
>>> Is it correct?
>>>
>>> I checked used and committed memory from the core, they seem to be enough
>>> to allocate new objects as below:
>>>
>>> -------------------
>>> (gdb) p MetaspaceAux::_used_words
>>> $62 = {1742673, 17905981}
>>> (gdb) p Metaspace::_space_list->_committed_words
>>> $64 = 30932992
>>> (gdb) p Metaspace::_class_space_list->_committed_words
>>> $65 = 2621440
>>> -------------------
>>>
>>> I guess the cause of OOME is invalid value in _capacity_until_GC.
>>> _capacity_until_GC is used for expanding / shrinking Metaspace.
>>> However it is not checked when it changes.
>>>
>>> So I think we need to add the code to check it as below:
>>> (The change is for jdk/jdk)
>>> -------------------
>>> diff -r 32c6cc430526 src/hotspot/share/memory/metaspace.cpp
>>> --- a/src/hotspot/share/memory/metaspace.cpp    Wed Jan 09 22:59:49 2019 +0100
>>> +++ b/src/hotspot/share/memory/metaspace.cpp    Thu Jan 10 17:56:55 2019 +0900
>>> @@ -143,6 +143,10 @@
>>>       new_value = align_down(max_uintx, Metaspace::commit_alignment());
>>>     }
>>>
>>> +  if (new_value > MaxMetaspaceSize) {
>>> +    return false;
>>> +  }
>>> +
>>>     size_t prev_value = Atomic::cmpxchg(new_value, &_capacity_until_GC, old_capacity_until_GC);
>>>
>>>     if (old_capacity_until_GC != prev_value) {
>>> -------------------
>>>
>>> _capacity_until_GC is set to MaxMetaspaceSize at MetaspaceGC::initialize(),
>>> and it set to committed size or MetaspaceSize at MetaspaceGC::post_initialize().
>>> So I guess max value of it is MaxMetaspaceSize.
>>>
>>>
>>> What do you think?
>>> If it is a bug, I will file JBS and send review request.
>>>
>>>
>>> Thanks,
>>>
>>> Yasumasa
>>>
>>>
>>>
>>>