PING: MetaspaceGC::_capacity_until_GC exceeds MaxMetaspaceSize

[Yasumasa Suenaga]

PING: Did you read my email?

I wrote the problem and suggested fix in it.
Please tell me if you have question(s).

I can file it to JBS if it is a bug.


Thanks,

Yasumasa


On 2019/01/10 18:02, Yasumasa Suenaga wrote:
> Hi all,
> 
> I'm Yasumasa Suenaga (OpenJDK reviewer: ysuenaga)
> 
> My customer uses OpenJDK 8u131 with following commandline arguments:
> 
>    -XX:+CMSClassUnloadingEnabled -XX:CMSInitiatingOccupancyFraction=80
>    -XX:CompressedClassSpaceSize=128m -Xms4500m -Xmx4500m
>    -XX:MaxMetaspaceSize=256m -Xmn=768m -XX:MaxTenuringThreshold=15
>    -XX:OnOutOfMemoryError="/bin/kill -ABRT %p"
>    -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseConcMarkSweepGC
> 
> Their system has encountered OutOfMemoryError which is caused by Metaspace.
> 
> I checked core image which was collected by OnOutOfMemoryError, and I saw
> strange values in it as below:
> 
> -------------------
> (gdb) p Metaspace::_space_list->_reserved_words
> $67 = 31195136
> (gdb) p Metaspace::_class_space_list->_reserved_words
> $68 = 16777216
> (gdb) p MetaspaceGC::_capacity_until_GC
> $75 = 448045056
> -------------------
> 
> MaxMetaspaceSize and CompressedClassSpaceSize seem to affect Metaspace.
> However, MetaspaceGC::_capacity_until_GC exceeds MaxMetaspaceSize.
> Is it correct?
> 
> I checked used and committed memory from the core, they seem to be enough
> to allocate new objects as below:
> 
> -------------------
> (gdb) p MetaspaceAux::_used_words
> $62 = {1742673, 17905981}
> (gdb) p Metaspace::_space_list->_committed_words
> $64 = 30932992
> (gdb) p Metaspace::_class_space_list->_committed_words
> $65 = 2621440
> -------------------
> 
> I guess the cause of OOME is invalid value in _capacity_until_GC.
> _capacity_until_GC is used for expanding / shrinking Metaspace.
> However it is not checked when it changes.
> 
> So I think we need to add the code to check it as below:
> (The change is for jdk/jdk)
> -------------------
> diff -r 32c6cc430526 src/hotspot/share/memory/metaspace.cpp
> --- a/src/hotspot/share/memory/metaspace.cpp    Wed Jan 09 22:59:49 2019 +0100
> +++ b/src/hotspot/share/memory/metaspace.cpp    Thu Jan 10 17:56:55 2019 +0900
> @@ -143,6 +143,10 @@
>       new_value = align_down(max_uintx, Metaspace::commit_alignment());
>     }
> 
> +  if (new_value > MaxMetaspaceSize) {
> +    return false;
> +  }
> +
>     size_t prev_value = Atomic::cmpxchg(new_value, &_capacity_until_GC, old_capacity_until_GC);
> 
>     if (old_capacity_until_GC != prev_value) {
> -------------------
> 
> _capacity_until_GC is set to MaxMetaspaceSize at MetaspaceGC::initialize(),
> and it set to committed size or MetaspaceSize at MetaspaceGC::post_initialize().
> So I guess max value of it is MaxMetaspaceSize.
> 
> 
> What do you think?
> If it is a bug, I will file JBS and send review request.
> 
> 
> Thanks,
> 
> Yasumasa
> 
> 
> 
>