From jiangli at openjdk.org  Fri Feb 16 22:00:55 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Fri, 16 Feb 2024 22:00:55 GMT
Subject: RFR: Merge jdk-17-ga
Message-ID: <uOGrNEf-oqrro4yfPZHioos9iPY_0k3K_wVz-4xxkpQ=.035b0ed7-4f49-4c4c-99e8-16e451889bd0@github.com>

This is a redo of https://github.com/openjdk/tsan/pull/16 to address the commits issue. Updated description:

Command: $ git merge --no-ff jdk-17-ga

Most manual merge for conflicts are mechanical.

Additional manual updates:

Add #include "memory/resourceArea.hpp" in src/hotspot/share/tsan/tsanOopMap.cpp to resolve compiler error:
tsanOopMap.cpp:351:5: error: use of undeclared identifier 'ResourceMark'; did you mean 'ResourceStack'?
ResourceMark rm;
^~~~~~~~~~~~
ResourceStack

Remove AMD64_ONLY(assert((id_u8 & left_n_bits(17)) == 0, "jmethodID is not aligned");) in classFileParser.cpp. left_n_bits has been removed by https://bugs.openjdk.org/browse/JDK-8178348

Change obj_field_addr_raw<oop>() to field_addr() in java_lang_Class::init_lock_addr in javaClasses.cpp.

Remove ShenandoahTSANWeakRoot::ShenandoahTSANWeakRoot as ShenandoahWeakSerialRoot is removed in shenandoahRootProcessor.cpp.

Remove JVMWrapper calls in tsan related functions in jvm.cpp. The macro has been removed.

Change to use current for JRT_ENTRY(void, SharedRuntime::tsan_interp_method_entry(JavaThread* thread)) as JRT_ENTRY macro expects current, sharedRuntime.cpp.

Change THREAD to current in synchronizer.cpp.

Tested by running hotspot/jtreg/tsan locally:


$ bash configure --with-boot-jdk=/usr/local/google/home/jianglizhou/openjdk/jdk-17.0.10 --with-debug-level=release --disable-warnings-as-errors --with-jtreg=/usr/local/google/home/jianglizhou/github/jtreg/build/images/jtreg --with-stdc++lib=static --disable-precompiled-headers --enable-unlimited-crypto --with-native-debug-symbols=internal --with-default-make-target=jdk-image --disable-warnings-as-errors --with-toolchain-type=clang --with-toolchain-path=/usr/local/google/home/jianglizhou/clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04 --disable-warnings-as-errors

$ make test TEST=hotspot/jtreg/tsan

New runs found more random failures in addition to tsan/NonRacySyncBlockExceptionLoopTest.java. The failed tests appear to be inconsistent, but tsan/NonRacyGarbageCollectionLoopTest.java is more consistently failing.

Also built and tested using Clang/LLVM 16.0.6. It appears to be working with jdk-17-ga merge.

-------------

Commit messages:
 - Merge tag 'jdk-17-ga' into 17-merge
 - 8270872: Final nroff manpage update for JDK 17
 - 8271588: JFR Recorder Thread crashed with SIGSEGV in write_klass
 - 8271863: ProblemList serviceability/sa/TestJmapCore.java on linux-x64 with ZGC
 - 8271894: ProblemList javax/swing/JComponent/7154030/bug7154030.java in JDK17
 - 8271877: ProblemList jdk/jfr/event/gc/detailed/TestEvacuationFailedEvent.java in JDK17
 - 8271064: ZGC several jvm08 perf regressions after JDK-8268372
 - 8067223: [TESTBUG] Rename Whitebox API package
 - 8271150: Remove EA from JDK 17 version string starting with Initial RC promotion on Aug 5, 2021(B34)
 - 8271272: C2: assert(!had_error) failed: bad dominance
 - ... and 5180 more: https://git.openjdk.org/tsan/compare/8fe80f4e...b21ad21d

The webrevs contain the adjustments done while merging with regards to each parent branch:
 - tsan: https://webrevs.openjdk.org/?repo=tsan&pr=17&range=00.0
 - jdk-17-ga: https://webrevs.openjdk.org/?repo=tsan&pr=17&range=00.1

Changes: https://git.openjdk.org/tsan/pull/17/files
  Stats: 2477284 lines in 22798 files changed: 1467490 ins; 880336 del; 129458 mod
  Patch: https://git.openjdk.org/tsan/pull/17.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/17/head:pull/17

PR: https://git.openjdk.org/tsan/pull/17

From jiangli at openjdk.org  Wed Feb 28 00:00:16 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 28 Feb 2024 00:00:16 GMT
Subject: RFR: Merge jdk-17-ga
In-Reply-To: <uOGrNEf-oqrro4yfPZHioos9iPY_0k3K_wVz-4xxkpQ=.035b0ed7-4f49-4c4c-99e8-16e451889bd0@github.com>
References: <uOGrNEf-oqrro4yfPZHioos9iPY_0k3K_wVz-4xxkpQ=.035b0ed7-4f49-4c4c-99e8-16e451889bd0@github.com>
Message-ID: <TVAG3S75_JmHK-ZRueaIzlJoOUA39iCSC2SLZuu1bUM=.503ebe5f-4afc-4209-b80d-7a6d869f0219@github.com>

On Fri, 16 Feb 2024 21:43:43 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

> This is a redo of https://github.com/openjdk/tsan/pull/16 to address the commits issue. Updated description:
> 
> Command: $ git merge --no-ff jdk-17-ga
> 
> Most manual merge for conflicts are mechanical.
> 
> Additional manual updates:
> 
> Add #include "memory/resourceArea.hpp" in src/hotspot/share/tsan/tsanOopMap.cpp to resolve compiler error:
> tsanOopMap.cpp:351:5: error: use of undeclared identifier 'ResourceMark'; did you mean 'ResourceStack'?
> ResourceMark rm;
> ^~~~~~~~~~~~
> ResourceStack
> 
> Remove AMD64_ONLY(assert((id_u8 & left_n_bits(17)) == 0, "jmethodID is not aligned");) in classFileParser.cpp. left_n_bits has been removed by https://bugs.openjdk.org/browse/JDK-8178348
> 
> Change obj_field_addr_raw<oop>() to field_addr() in java_lang_Class::init_lock_addr in javaClasses.cpp.
> 
> Remove ShenandoahTSANWeakRoot::ShenandoahTSANWeakRoot as ShenandoahWeakSerialRoot is removed in shenandoahRootProcessor.cpp.
> 
> Remove JVMWrapper calls in tsan related functions in jvm.cpp. The macro has been removed.
> 
> Change to use current for JRT_ENTRY(void, SharedRuntime::tsan_interp_method_entry(JavaThread* thread)) as JRT_ENTRY macro expects current, sharedRuntime.cpp.
> 
> Change THREAD to current in synchronizer.cpp.
> 
> Tested by running hotspot/jtreg/tsan locally:
> 
> 
> $ bash configure --with-boot-jdk=/usr/local/google/home/jianglizhou/openjdk/jdk-17.0.10 --with-debug-level=release --disable-warnings-as-errors --with-jtreg=/usr/local/google/home/jianglizhou/github/jtreg/build/images/jtreg --with-stdc++lib=static --disable-precompiled-headers --enable-unlimited-crypto --with-native-debug-symbols=internal --with-default-make-target=jdk-image --disable-warnings-as-errors --with-toolchain-type=clang --with-toolchain-path=/usr/local/google/home/jianglizhou/clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04 --disable-warnings-as-errors
> 
> $ make test TEST=hotspot/jtreg/tsan
> 
> New runs found more random failures in addition to tsan/NonRacySyncBlockExceptionLoopTest.java. The failed tests appear to be inconsistent, but tsan/NonRacyGarbageCollectionLoopTest.java is more consistently failing.
> 
> Also built and tested using Clang/LLVM 16.0.6. It appears to be working with jdk-17-ga merge.

I checked tsan specific changes in original src/hotspot/share/gc/shared/weakProcessorPhases.cpp and src/hotspot/share/gc/shared/weakProcessorPhases.hpp. The tsan specific changes in those are no longer applicable. https://bugs.openjdk.org/browse/JDK-8212879 made JVMTI tagmap table  walked by GC concurrently, and the serial iterator was removed. I think the tsan test failures that I encountered with 17 merge was related to tagmap change. 

@caoman We can move forward with the review for the jdk 17 merge. I'll do some study on the JDK-8212879 to see how to include the handling for TsanOopMap.

-------------

PR Comment: https://git.openjdk.org/tsan/pull/17#issuecomment-1967933599

From manc at openjdk.org  Wed Feb 28 02:45:16 2024
From: manc at openjdk.org (Man Cao)
Date: Wed, 28 Feb 2024 02:45:16 GMT
Subject: RFR: Merge jdk-17-ga
In-Reply-To: <uOGrNEf-oqrro4yfPZHioos9iPY_0k3K_wVz-4xxkpQ=.035b0ed7-4f49-4c4c-99e8-16e451889bd0@github.com>
References: <uOGrNEf-oqrro4yfPZHioos9iPY_0k3K_wVz-4xxkpQ=.035b0ed7-4f49-4c4c-99e8-16e451889bd0@github.com>
Message-ID: <6L-F8TQu8ujZNKEQtjBp8Gs8o5I4e7u-Kyql7y_1_tY=.e448c4b1-e310-45b5-beef-bba467d141df@github.com>

On Fri, 16 Feb 2024 21:43:43 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

> This is a redo of https://github.com/openjdk/tsan/pull/16 to address the commits issue. Updated description:
> 
> Command: $ git merge --no-ff jdk-17-ga
> 
> Most manual merge for conflicts are mechanical.
> 
> Additional manual updates:
> 
> Add #include "memory/resourceArea.hpp" in src/hotspot/share/tsan/tsanOopMap.cpp to resolve compiler error:
> tsanOopMap.cpp:351:5: error: use of undeclared identifier 'ResourceMark'; did you mean 'ResourceStack'?
> ResourceMark rm;
> ^~~~~~~~~~~~
> ResourceStack
> 
> Remove AMD64_ONLY(assert((id_u8 & left_n_bits(17)) == 0, "jmethodID is not aligned");) in classFileParser.cpp. left_n_bits has been removed by https://bugs.openjdk.org/browse/JDK-8178348
> 
> Change obj_field_addr_raw<oop>() to field_addr() in java_lang_Class::init_lock_addr in javaClasses.cpp.
> 
> Remove ShenandoahTSANWeakRoot::ShenandoahTSANWeakRoot as ShenandoahWeakSerialRoot is removed in shenandoahRootProcessor.cpp.
> 
> Remove JVMWrapper calls in tsan related functions in jvm.cpp. The macro has been removed.
> 
> Change to use current for JRT_ENTRY(void, SharedRuntime::tsan_interp_method_entry(JavaThread* thread)) as JRT_ENTRY macro expects current, sharedRuntime.cpp.
> 
> Change THREAD to current in synchronizer.cpp.
> 
> Tested by running hotspot/jtreg/tsan locally:
> 
> 
> $ bash configure --with-boot-jdk=/usr/local/google/home/jianglizhou/openjdk/jdk-17.0.10 --with-debug-level=release --disable-warnings-as-errors --with-jtreg=/usr/local/google/home/jianglizhou/github/jtreg/build/images/jtreg --with-stdc++lib=static --disable-precompiled-headers --enable-unlimited-crypto --with-native-debug-symbols=internal --with-default-make-target=jdk-image --disable-warnings-as-errors --with-toolchain-type=clang --with-toolchain-path=/usr/local/google/home/jianglizhou/clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04 --disable-warnings-as-errors
> 
> $ make test TEST=hotspot/jtreg/tsan
> 
> New runs found more random failures in addition to tsan/NonRacySyncBlockExceptionLoopTest.java. The failed tests appear to be inconsistent, but tsan/NonRacyGarbageCollectionLoopTest.java is more consistently failing.
> 
> Also built and tested using Clang/LLVM 16.0.6. It appears to be working with jdk-17-ga merge.

Agree that it is better to adapt TsanOopMap in a separate PR. Another reference is https://bugs.openjdk.org/browse/JDK-8235573, which moved `JFR::weak_oops_do` to use OopStorage and support concurrent processing.

Could you update the description to mention that after this merge, TSAN is not yet fully working? This is because `TsanOopMap::weak_oops_do()` is not called anywhere, so TSAN fails to keep track of objects moved by GC. 

A few comments:
1. In .jcheck/conf, could we use JDK 17's lines for `error=` and `files=`?
2. In src/hotspot/share/classfile/systemDictionary.cpp, line `assert(calledholdinglock,"must hold lock for notify");` seems an accidental removal of space.
3. The changes to shenandoahPhaseTimings.hpp, shenandoahRootProcessor.hpp/cpp look unnecessary. It seems to revive removed classes `ShenandoahSerialRoot`, `ShenandoahWeakSerialRoot`, `ShenandoahSerialWeakRoots`, etc, I think we can drop these changes to shenandoah files similarly to dropping the TSAN changes to weakProcessorPhases.cpp and weakProcessorPhases.hpp. They are all related to `TsanOopMap`, and the TSAN-touched Shenandoah classes no longer exist in JDK 17.
4. In src/hotspot/share/runtime/globals.hpp, there is a missing empty line between the `ThreadSanitizerIgnoreFile` flag and the later `VerifyCrossModifyFence` flag.
5. In src/hotspot/share/runtime/synchronizer.cpp for `ObjectSynchronizer::wait()`, the change to `DTRACE_MONITOR_WAIT_PROBE` line seems unnecessary.
6. The change to test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c seems a backport of https://bugs.openjdk.org/browse/JDK-8272472. Perhaps mention this in description?

-------------

PR Comment: https://git.openjdk.org/tsan/pull/17#issuecomment-1968087335