Failed assert during VM_HeapWalkOperation

[Roman Kennke]

Hello Rodrigo,

> Dear Shenandoah dev list,
> 
> my name is Rodrigo, I am an ETH postdoc researcher, and my group currently
> has a project that involves using Shenandoah and tracing objects upon
> request by Java code (yes, many people will think this is a terrible
> idea... :-), I can explain if someone is interested...).
> 
> We are using a slightly modified copy of the VM_HeapWalkOperation VM
> operation to trace objects starting for a particular initial object.
> However we started noticing failed asserts as soon as we introduced more
> load into the system (more allocations, leading to more GC effort).
> 
> After some debugging, it turns out to be a load barrier check that fails
> during the VM_HeapWalkOperation because it i) sets the mark bits and ii)
> tries to resolve oops (triggering the barrier). For some reason, the gc
> state indicates that object headers should have the forwardee pointer in
> place and thus it fails. I saw that there is already an exception for this
> particular VM op but it does not cover this path.
> 
> This failed assert can be reproduced using an unmodified JDK. I just tested
> it today by downloading [1], building, and creating a simple test which
> basically calls a JNI function that calls JVMTI FollowReferences. I can
> provide the sources to reproduce the failed assert. If I
> use -XX:ShenandoahGCMode=passive, no barriers are enabled and everything
> works fine. Dump file is attached.
> 
> I think this might be a bug. Please let me know if you need any further
> information/help.

This does indeed sound like a bug that we fixed recently:

https://mail.openjdk.java.net/pipermail/shenandoah-dev/2020-March/thread.html

https://bugs.openjdk.java.net/browse/JDK-8237632

 and

https://bugs.openjdk.java.net/browse/JDK-8237396

Can you tell us which exact version you are using? The attachment has
been stripped by the mailing list.

Thank you,
Roman