RFR: In TsanSymbolize, use ThreadInVMfromUnknown to transition to VM state. [v2]

[Jiangli Zhou]

> Our internal testing on JDK 21 fastdebug binary runs into following assertion failure when tsan tries to report a race during executing a JNI native method.
> 
> 
> #  Internal Error (/tmp/jdkbuild/srcs/src/hotspot/share/oops/accessBackend.cpp:224), pid=2318779, tid=2318806
> #  assert(state == _thread_in_vm || state == _thread_in_Java || state == _thread_new) failed: Wrong thread state for accesses: 4
> 
> 
> 
> V  [libjvm.so+0x84113d]  AccessInternal::check_access_thread_state()+0x8d  (accessBackend.cpp:224)
> V  [libjvm.so+0xdc65d7]  ClassLoaderData::is_alive() const+0x37
> V  [libjvm.so+0x173165a]  Method::checked_resolve_jmethod_id(_jmethodID*)+0xaa
> V  [libjvm.so+0x1bb3293]  TsanSymbolize+0x73
> <snip>
> C  [<snip>]  __tsan::ReportRace(__tsan::ThreadState*, __tsan::RawShadow*, __tsan::Shadow, __tsan::Shadow, unsigned long)+0xb2f
> C  [<snip>]  WriteHeapVar+0x1d
> C  [<snip>]  <JNI_native_method>
> <snip>
> 
> 
> The assert is hit as the thread state is `_thread_in_native`, which is not expected by `AccessInternal::check_access_thread_state`.
> 
> Since `__tsan::ReportRace ... -> TsanSymbolize` could be triggered during executing both Java code or native code, making `TsanSymbolize` a `JVM_ENTRY` for transitioning into VM state does not work in all cases. Instead, `ThreadInVMfromUnknown` is used for transitioning into VM state only if the current JavaThread is in native. As part of the fix, I also remove `DEBUG_ONLY(NoSafepointVerifier nsv;)` from  `TsanSymbolize`. `NoSafepointVerifier nsv;)` does `JavaThread::cast(_thread)->inc_no_safepoint_count()`, which causes `_no_safepoint_count` to be `>0` and asserts during the thread state transition.
> 
> 
> #  Internal Error (/tmp/jdkbuild/srcs/src/hotspot/share/runtime/javaThread.cpp:362), pid=454073, tid=454110
> #  assert(false) failed: Possible safepoint reached by thread that does not allow it
> 
> 
> Tested the change with tsan jtreg tests on fastdebug build. The fix was also tested with the affected internal tests on fastdebug build.
> 
> 
> $ bash configure --with-boot-jdk=/usr/local/google/home/jianglizhou/openjdk/jdk-21.0.7 --with-debug-level=fastdebug --disable-warnings-as-errors --with-jtreg=/usr/local/google/home/jianglizhou/github/jtreg/build/images/jtreg --with-stdc++lib=static --disable-precompiled-headers --enable-unlimited-crypto --with-native-debug-symbols=internal --with-default-make-target=jdk-image --disable-warnings-as-errors --with-toolchain-type=clang --disable-warnings-as-errors
> 
> 
> 
> $ make test TEST=hotspot/jtreg/tsan
> 
> 
> 
> =====================...

Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:

  Address @rasbold review: Remove internal bug ID.

-------------

Changes:
  - all: https://git.openjdk.org/tsan/pull/24/files
  - new: https://git.openjdk.org/tsan/pull/24/files/fc8cd3b9..2d05b63b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=tsan&pr=24&range=01
 - incr: https://webrevs.openjdk.org/?repo=tsan&pr=24&range=00-01

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/tsan/pull/24.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/24/head:pull/24

PR: https://git.openjdk.org/tsan/pull/24