RFR: 8286830: ~HandshakeState should not touch oops

[Daniel D.Daugherty]

On Thu, 19 May 2022 19:17:09 GMT, Patricio Chilano Mateo <pchilanomate at openjdk.org> wrote:

> On 8284632 we added ~HandshakeState to cleanup any potential async operations left in the handshake queue. This could trigger the release of an oophandle from its associated oop-storage when trying to delete an AsyncExceptionHandshake object. Since the exiting JT already executed on_thread_detach() this is forbidden. 
> To fix this I used the original approach that @dcubed-ojdk posted on the 8284632 fix, which is cleaning up those operations before on_thread_detach() is executed. To make sure that no other async exception operation is added to the queue after that I did two things:
> - Added a check in install_async_exception() to avoid installing the operation if the target already moved to the _thread_exiting state.
> - Move the setting of the _thread_exiting state to include the case where JavaThread::exit() is called with destroy_vm=true.
> 
> I also added another run to tests StopAtExit.java and SuspendAtExit.java with extra contention on the Threads_lock. That increases the odds of the target being handshake safe while being in the _thread_exiting state which allows testing for more pathological cases.
> 
> I was able to reproduce the issue by running StopAtExit.java with -XX:+UseShenandoahGC and verified that the test now passes. I also run mach5 tiers1-3. Will run the upper tiers also and will do an additional pass after that.
> 
> Thanks,
> Patricio

StressWrapper_StopAtExit.jtr release bits:

----------System.out:(10/297)----------

Agent_OnLoad started
Agent_OnLoad finished

About to execute for 7200 seconds.
Executed 27471345 loops in 7200 seconds.
About to execute for 7200 seconds.
Executed 12930003 loops in 7200 seconds.
vm_exit: dcubed_async_global_alloc_cnt=232552804
vm_exit: dcubed_async_global_release_cnt=232552804
----------System.err:(1852/93678)----------


StressWrapper_StopAtExit.jtr fastdebug bits:

----------System.out:(10/294)----------

Agent_OnLoad started
Agent_OnLoad finished

About to execute for 7200 seconds.
Executed 18516793 loops in 7200 seconds.
About to execute for 7200 seconds.
Executed 9128937 loops in 7200 seconds.
vm_exit: dcubed_async_global_alloc_cnt=29621450
vm_exit: dcubed_async_global_release_cnt=29621450
----------System.err:(1660/83996)----------


StressWrapper_StopAtExit.jtr slowdebug bits:

----------System.out:(10/293)----------

Agent_OnLoad started
Agent_OnLoad finished

About to execute for 7200 seconds.
Executed 8147300 loops in 7200 seconds.
About to execute for 7200 seconds.
Executed 3913063 loops in 7200 seconds.
vm_exit: dcubed_async_global_alloc_cnt=15924123
vm_exit: dcubed_async_global_release_cnt=15924123
----------System.err:(691/34976)----------

No leaks detected in any config.

-------------

PR: https://git.openjdk.java.net/jdk/pull/8795