SEGV occurs when ClassLoader and Metaspace is released in JDK 8

[Osamu Sakamoto]

Hi Thomas,

Thank you for advise.
I'm sorry but I can't send to Gmail Address, So I reply to this ML only.

 > Could be https://bugs.openjdk.java.net/browse/JDK-8016731 but that 
has never been solved.
This situation is similar, but native stack traces are different and I 
can't determine if it is the same.

 > Your best bet would be to obtain a debug (fastdebug) build of that 
JVM version and attempt to reproduce the bug. The hs-err file may be 
more helpful then.
This problem occurred a few time, but I haven't understood how to reproduce.
I have the hs-err file, could you tell me what I should check?

 > Also, does this error happen in jdk11 too?
I haven't try on jdk11.
Our system is running, so it is difficult to try to change JDK version.

This problem seems to occurs on CMS only. (Our System uses CMS GC.)
I checked native stack trace[1], 
ClassLoaderDataGraph::purge_if_needed()[2] was executed on frame #11.
purge_if_needed() is commented that "Only purge the CLDG for CMS if 
concurrent sweep is complete.",
so I think this method passes only when CMS GC is used.

[1]https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-October/027449.html
[2]http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot/file/eed8e846c982/src/share/vm/classfile/classLoaderData.hpp#l100

Focusing on this strange ClassLoader, it is referenced from 
ClassLoaderDataGraph::_unloading[3] before crashed.
I understand as ClassLoaderDataGraph::_unloading is head address of 
purging ClassLoader list.
In this problem, It looks like that _unloading is also broken, not only 
metaspace middle chank.
So, I think this is a bug in combination of Metaspace and CMS.
[3]http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot/file/eed8e846c982/src/share/vm/classfile/classLoaderData.hpp#l66

Could you let me know you thoughts about this?

Thanks,

Osamu
On 11/5/19 19:34, Thomas Stüfe wrote:
> Hi Osamu,
>
> Strange and not easy to analyze. Looks too "deterministic" to be a 
> random memory overwrite. The first chunk of the medium chunk list is 
> 0x10, but the only place where this is set we immediately beforehand 
> dereference this pointer 
> (http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/b44df6c5942c/src/share/vm/memory/metaspace.cpp#l2423) 
> so that should have crashed.
>
> Could be https://bugs.openjdk.java.net/browse/JDK-8016731 but that has 
> never been solved.
>
> Your best bet would be to obtain a debug (fastdebug) build of that JVM 
> version and attempt to reproduce the bug. The hs-err file may be more 
> helpful then.
>
> Also, does this error happen in jdk11 too?
>
> Cheers Thomas
>
>
> On Tue, Nov 5, 2019 at 10:51 AM Osamu Sakamoto 
> <sakamoto.osamu at nttcom.co.jp <mailto:sakamoto.osamu at nttcom.co.jp>> wrote:
>
>     Hi all,
>
>     I have investigated the cause of SEGV occurring in CMS GC of
>     OpenJDK 8,
>     and I've not been able to clarify the cause.
>     Could you help me to solve the problem?
>
>     Our system uses OpenJDK 1.8.0.171 and crashed by SEGV when purging a
>     ClassLoader at safepoint.
>     I found 2 strange points,
>
>     1. SEGV occrred when the metaspace destructor maneged by that
>     Classloader is executed,
>         because the metaspace has illegal chunk address(0x10).
>
>     2. That ClassLoader's oop address indicates a character array, not a
>     ClassLoader.
>
>     I think memory corruption was occurred, but I've not understood the
>     reason yet.
>
>     I also asked this problem to hotspot-gc-dev mailing list, and I
>     received
>     a comment that it might be a JDK bug.
>     Detailed Information is summarized in the following hotspot-gc-dev ML
>     thread.
>     <https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-October/027583.html>
>
>     Does anyone know this problem?
>
>     Thanks,
>
>     Osamu
>
>
>
>