From manc at openjdk.org  Mon Jul 22 18:20:22 2024
From: manc at openjdk.org (Man Cao)
Date: Mon, 22 Jul 2024 18:20:22 GMT
Subject: RFR: 8336924: Disable jcheck for referencing an issue number in commit
 message title.
Message-ID: <uhBuTpffWo4oJctAMNBDQilAfT2rmxPKfiYgHbFZJ04=.0fec6712-1d05-42ce-a5c1-6002f65c8967@github.com>

Hi all,

https://github.com/openjdk/tsan/pull/19 sees a jcheck error "The commit message does not reference any issue. To add an issue reference to this PR, edit the title to be of the format issue number: message."
TSAN project currently does not need an issue for a commit, so I propose we disable this check.

The jcheck was probably enabled in Sep, 2020 by https://github.com/openjdk/tsan/commit/e0c8d4420c8e1a84581927cf77314498b8e5aa52.  However, TSAN project did not have new PRs except merges after Sep, 2020. (I think this check probably skips checking merge commits and merge PRs.)

-Man

-------------

Commit messages:
 - Disable jcheck for referencing an issue number in commit message title.

Changes: https://git.openjdk.org/tsan/pull/20/files
  Webrev: https://webrevs.openjdk.org/?repo=tsan&pr=20&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8336924
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/tsan/pull/20.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/20/head:pull/20

PR: https://git.openjdk.org/tsan/pull/20

From cushon at openjdk.org  Mon Jul 22 18:20:22 2024
From: cushon at openjdk.org (Liam Miller-Cushon)
Date: Mon, 22 Jul 2024 18:20:22 GMT
Subject: RFR: 8336924: Disable jcheck for referencing an issue number in
 commit message title.
In-Reply-To: <uhBuTpffWo4oJctAMNBDQilAfT2rmxPKfiYgHbFZJ04=.0fec6712-1d05-42ce-a5c1-6002f65c8967@github.com>
References: <uhBuTpffWo4oJctAMNBDQilAfT2rmxPKfiYgHbFZJ04=.0fec6712-1d05-42ce-a5c1-6002f65c8967@github.com>
Message-ID: <6sFYTejeQi85n_z_J2PQPdYFb-LK-EpWerGJfAjss1I=.34831322-f3c5-480c-ab03-064b797190a0@github.com>

On Sat, 20 Jul 2024 00:29:29 GMT, Man Cao <manc at openjdk.org> wrote:

> Hi all,
> 
> https://github.com/openjdk/tsan/pull/19 sees a jcheck error "The commit message does not reference any issue. To add an issue reference to this PR, edit the title to be of the format issue number: message."
> TSAN project currently does not need an issue for a commit, so I propose we disable this check.
> 
> The jcheck was probably enabled in Sep, 2020 by https://github.com/openjdk/tsan/commit/e0c8d4420c8e1a84581927cf77314498b8e5aa52.  However, TSAN project did not have new PRs except merges after Sep, 2020. (I think this check probably skips checking merge commits and merge PRs.)
> 
> -Man

LGTM

I wondered if you could unblock it by filing an actual bug for this one PR and using that in the title?

-------------

Marked as reviewed by cushon (Reviewer).

PR Review: https://git.openjdk.org/tsan/pull/20#pullrequestreview-2189578712
PR Comment: https://git.openjdk.org/tsan/pull/20#issuecomment-2240824506

From manc at openjdk.org  Mon Jul 22 18:20:22 2024
From: manc at openjdk.org (Man Cao)
Date: Mon, 22 Jul 2024 18:20:22 GMT
Subject: RFR: 8336924: Disable jcheck for referencing an issue number in
 commit message title.
In-Reply-To: <uhBuTpffWo4oJctAMNBDQilAfT2rmxPKfiYgHbFZJ04=.0fec6712-1d05-42ce-a5c1-6002f65c8967@github.com>
References: <uhBuTpffWo4oJctAMNBDQilAfT2rmxPKfiYgHbFZJ04=.0fec6712-1d05-42ce-a5c1-6002f65c8967@github.com>
Message-ID: <kQDNa7x1jiOBBrXlgnNYeq6wskGmfg2cXwTGPkcAyOc=.eb30e4ab-7840-4641-9a8b-f4c8798738b3@github.com>

On Sat, 20 Jul 2024 00:29:29 GMT, Man Cao <manc at openjdk.org> wrote:

> Hi all,
> 
> https://github.com/openjdk/tsan/pull/19 sees a jcheck error "The commit message does not reference any issue. To add an issue reference to this PR, edit the title to be of the format issue number: message."
> TSAN project currently does not need an issue for a commit, so I propose we disable this check.
> 
> The jcheck was probably enabled in Sep, 2020 by https://github.com/openjdk/tsan/commit/e0c8d4420c8e1a84581927cf77314498b8e5aa52.  However, TSAN project did not have new PRs except merges after Sep, 2020. (I think this check probably skips checking merge commits and merge PRs.)
> 
> -Man

Edited title to a fake issue number to work around the jcheck.

I opened https://bugs.openjdk.org/browse/SKARA-2337 to ask how to proceed.

-------------

PR Comment: https://git.openjdk.org/tsan/pull/20#issuecomment-2240805197
PR Comment: https://git.openjdk.org/tsan/pull/20#issuecomment-2240823984

From manc at openjdk.org  Mon Jul 22 18:20:22 2024
From: manc at openjdk.org (Man Cao)
Date: Mon, 22 Jul 2024 18:20:22 GMT
Subject: RFR: 8336924: Disable jcheck for referencing an issue number in
 commit message title.
In-Reply-To: <6sFYTejeQi85n_z_J2PQPdYFb-LK-EpWerGJfAjss1I=.34831322-f3c5-480c-ab03-064b797190a0@github.com>
References: <uhBuTpffWo4oJctAMNBDQilAfT2rmxPKfiYgHbFZJ04=.0fec6712-1d05-42ce-a5c1-6002f65c8967@github.com>
 <6sFYTejeQi85n_z_J2PQPdYFb-LK-EpWerGJfAjss1I=.34831322-f3c5-480c-ab03-064b797190a0@github.com>
Message-ID: <VPQCEeF1MAbqjk4_7v1xbx9wISpxGooXAJxmlcp1nsk=.68b3489c-3033-41ec-a424-5dba113d7459@github.com>

On Sat, 20 Jul 2024 01:36:47 GMT, Liam Miller-Cushon <cushon at openjdk.org> wrote:

> I wondered if you could unblock it by filing an actual bug for this one PR and using that in the title?

That's possible, but the bug will be under the "JDK" project, which seems inappropriate. There is not a "TSAN" project component.

-------------

PR Comment: https://git.openjdk.org/tsan/pull/20#issuecomment-2240826328

From manc at openjdk.org  Mon Jul 22 18:43:57 2024
From: manc at openjdk.org (Man Cao)
Date: Mon, 22 Jul 2024 18:43:57 GMT
Subject: Integrated: 8336924: Disable jcheck for issue number in TSAN project
In-Reply-To: <uhBuTpffWo4oJctAMNBDQilAfT2rmxPKfiYgHbFZJ04=.0fec6712-1d05-42ce-a5c1-6002f65c8967@github.com>
References: <uhBuTpffWo4oJctAMNBDQilAfT2rmxPKfiYgHbFZJ04=.0fec6712-1d05-42ce-a5c1-6002f65c8967@github.com>
Message-ID: <9AJzOzLtywG8BebVcDnWXlui76rEbA6ARfOjCErwnt4=.71ae30da-eb88-4f2a-a24a-94f66789d173@github.com>

On Sat, 20 Jul 2024 00:29:29 GMT, Man Cao <manc at openjdk.org> wrote:

> Hi all,
> 
> https://github.com/openjdk/tsan/pull/19 sees a jcheck error "The commit message does not reference any issue. To add an issue reference to this PR, edit the title to be of the format issue number: message."
> TSAN project currently does not need an issue for a commit, so I propose we disable this check.
> 
> The jcheck was probably enabled in Sep, 2020 by https://github.com/openjdk/tsan/commit/e0c8d4420c8e1a84581927cf77314498b8e5aa52.  However, TSAN project did not have new PRs except merges after Sep, 2020. (I think this check probably skips checking merge commits and merge PRs.)
> 
> -Man

This pull request has now been integrated.

Changeset: 592de08d
Author:    Man Cao <manc at openjdk.org>
URL:       https://git.openjdk.org/tsan/commit/592de08d8771c8401df10f980a77950023237f98
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

8336924: Disable jcheck for issue number in TSAN project

Reviewed-by: cushon

-------------

PR: https://git.openjdk.org/tsan/pull/20

From jiangli at openjdk.org  Mon Jul 22 19:03:29 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Mon, 22 Jul 2024 19:03:29 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable
Message-ID: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>

The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):

- Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
- Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.

The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:

 - A Weakhandle that holds a pointer to the oop;
 - The original (or updated) address of the object in Java heap;

For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.

To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:

- If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
- If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
- After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.

Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.

As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.

Suppressed following Tsan errors in JDK:
1)  In `java.util.concurrent.locks`, for

WARNING: ThreadSanitizer: data race (pid=787116)
  Write of size 4 at 0x0000fff80818 by thread T9:
    #0 java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(Ljava/util/concurrent/locks/AbstractQueuedSynchronizer$Node;IZZZJ)I AbstractQueuedSynchronizer.java:719 
    #1 java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(I)V AbstractQueuedSynchronizer.java:990 
    #2 java.util.concurrent.locks.ReentrantLock$Sync.lock()V ReentrantLock.java:153 
    #3 java.util.concurrent.locks.ReentrantLock.lock()V ReentrantLock.java:322 
    #4 java.lang.ref.ReferenceQueue.enqueue(Ljava/lang/ref/Reference;)Z ReferenceQueue.java:163 
    #5 java.lang.ref.Reference.enqueueFromPending()V Reference.java:235 
    #6 java.lang.ref.Reference.processPendingReferences()V Reference.java:266 
    #7 java.lang.ref.Reference$ReferenceHandler.run()V Reference.java:208 
    #8 (Generated Stub) <null> 

  Previous read of size 4 at 0x0000fff80818 by thread T14:
    #0 java.util.concurrent.locks.AbstractQueuedSynchronizer.signalNext(Ljava/util/concurrent/locks/AbstractQueuedSynchronizer$Node;)V AbstractQueuedSynchronizer.java:645 
    #1 java.util.concurrent.locks.AbstractQueuedSynchronizer.release(I)Z AbstractQueuedSynchronizer.java:1060 
    #2 java.util.concurrent.locks.ReentrantLock.unlock()V ReentrantLock.java:494 
    #3 java.lang.ref.ReferenceQueue.remove(J)Ljava/lang/ref/Reference; ReferenceQueue.java:220 
    #4 jdk.internal.ref.CleanerImpl.run()V CleanerImpl.java:140 
    #5 java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V Thread.java:1596 
    #6 java.lang.Thread.run()V Thread.java:1583 
    #7 jdk.internal.misc.InnocuousThread.run()V InnocuousThread.java:186 
    #8 (Generated Stub) <null> 


2) In `libjava.so` for:

WARNING: ThreadSanitizer: data race (pid=448501)
  Write of size 8 at 0x7bb000000280 by thread T28:
    #0 close <null> (java+0x55316) (BuildId: 9fa4aa9db3d9f57a2752512a1169f64e1b2aa897)
    #1 fileDescriptorClose make/src/java.base/unix/native/libjava/io_util_md.c:176:18 (libjava.so+0x15a5b) (BuildId: b4e70d2ea8d3d26ecd754accf8062c243bbe8115)

  Previous read of size 8 at 0x7bb000000280 by thread T30:
    #0 read <null> (java+0x5c473) (BuildId: 9fa4aa9db3d9f57a2752512a1169f64e1b2aa897)
    #1 handleRead make/src/java.base/unix/native/libjava/io_util_md.c:188:5 (libjava.so+0x15add) (BuildId: b4e70d2ea8d3d26ecd754accf8062c243bbe8115)

  Location is file descriptor 20 created by thread T27 at:
    #0 pipe <null> (java+0x5573f) (BuildId: 9fa4aa9db3d9f57a2752512a1169f64e1b2aa897)
    #1 Java_java_lang_ProcessImpl_forkAndExec make/src/java.base/unix/native/libjava/ProcessImpl_md.c:675:26 (libjava.so+0x10c9e) (BuildId: b4e70d2ea8d3d26ecd754accf8062c243bbe8115)

  Thread T28 (tid=448534, running) created by thread T27 at:
    #0 pthread_create <null> (java+0x4f36f) (BuildId: 9fa4aa9db3d9f57a2752512a1169f64e1b2aa897)
    #1 os::create_thread(Thread*, os::ThreadType, unsigned long) make/hotspot/src/hotspot/os/linux/os_linux.cpp:963:13 (libjvm.so+0xbe9921) (BuildId: 2e27aced1e4ff9bdae34b1db96b7f94ca7cf7a4d)
...


Testing:

Built with following command for linux-x64
`$ bash configure --with-boot-jdk=/usr/local/google/home/jianglizhou/openjdk/jdk-21.0.1 --with-debug-level=release --disable-warnings-as-errors --with-jtreg=/usr/local/google/home/jianglizhou/github/jtreg/build/images/jtreg --with-stdc++lib=static --disable-precompiled-headers --enable-unlimited-crypto --with-native-debug-symbols=internal --with-default-make-target=jdk-image --disable-warnings-as-errors --with-toolchain-type=clang`

Ran tsan jtreg tests
`$ make test TEST=hotspot/jtreg/tsan`

All tsan jtreg tests now pass on a release build with the change.

-------------

Commit messages:
 - Fix more Whitespace error.
 - Fix Whitespace errors.
 - Remove TsanOopMap::notify_tsan_for_freed_and_moved_objects() call from WeakProcessor::weak_oops_do.
 - Handle table resize in TsanOopMapTable::add_oop_with_size.
 - Replace DEBUG_PRINT in tsanOopMap.cpp with tsan log_trace.
 - Implement TsanOopMapTable::~TsanOopMapTable.
 - - Remove TsanOopMap::gc_notification and register_num_dead_callback() for TsanOopMap weak oop storage.
 - Add Xlog for tsan:
 - Rename TsanOopMap::update() to TsanOopMap::notify_tsan_for_freed_and_moved_objects()
 - Add back TsanOopMapTableKey copy constructor to avoid compiler warning:
 - ... and 4 more: https://git.openjdk.org/tsan/compare/46ab1ec9...4bd91916

Changes: https://git.openjdk.org/tsan/pull/19/files
  Webrev: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=00
  Stats: 723 lines in 11 files changed: 375 ins; 239 del; 109 mod
  Patch: https://git.openjdk.org/tsan/pull/19.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/19/head:pull/19

PR: https://git.openjdk.org/tsan/pull/19

From manc at openjdk.org  Tue Jul 23 01:56:57 2024
From: manc at openjdk.org (Man Cao)
Date: Tue, 23 Jul 2024 01:56:57 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>

On Tue, 16 Jul 2024 00:37:09 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: data race (pid=787116)...

Thanks a lot for this heavy-lifting work!
I have not fully finished the review, mainly not yet for details in TsanOopMapTableKey. But here are some feedback.

In description: 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.

Last sentence seems incorrect. These WeakHandles are currently processed by a subset of parallel GC worker threads. We probably should change it to be processed by the VM thread sequentially. See comment in weakProcessor.inline.hpp.

src/hotspot/share/gc/shared/weakProcessor.inline.hpp line 100:

> 98: 
> 99:   TSAN_RUNTIME_ONLY(
> 100:     TsanOopMap::notify_tsan_for_freed_and_moved_objects();

The `WeakProcessor::Task::work()` function is called by a subset of GC worker thread. This function will be called multiple times in parallel, but each thread runs the same code and processes the same data, so it is redundant work. It should probably be called only once, by the VM thread.

We can invoke `notify_tsan_for_freed_and_moved_objects()` from the VM thread by adding it to the two versions of `WeakProcessor::weak_oops_do` below:
- After line 142 `task.report_num_dead()`, in this file (weakProcessor.inline.hpp).
- In weakProcessor.cpp, between inside `WeakProcessor::weak_oops_do(BoolObjectClosure* is_alive, OopClosure* keep_alive)`.

We have another internal patch that adds a callback during a GC pause, at those two exact places. It is very similar to what TSAN needs to do.

src/hotspot/share/gc/shared/weakProcessor.inline.hpp line 158:

> 156:   WeakProcessorTimes times(nworkers);
> 157:   weak_oops_do(workers, is_alive, keep_alive, &times);
> 158: 

Nit: unnecessary newline.

src/hotspot/share/tsan/tsanOopMap.cpp line 232:

> 230:         if (can_move) {
> 231:           // Notify TSan, update occupied region.
> 232:           log_trace(tsan)("__tsan_java_move for [" PTR_FORMAT ", " PTR_FORMAT  "] -> [" PTR_FORMAT ", " PTR_FORMAT "]\n",

How about using `log_trace` only for this line, the line in `notify_tsan_for_freed_and_moved_objects` and the line above `__tsan_java_alloc`, and using `log_debug` for all other logging lines?
I suppose these 3 lines print a lot more messages than other logging lines.

src/hotspot/share/tsan/tsanOopMap.cpp line 240:

> 238:       assert(to > mem_begin_ && to <= mem_end_, "end address outside range");
> 239:       BitMap::idx_t idx_to = to_idx(to);
> 240:       return bitmap_.find_first_set_bit(to_idx(from), idx_to) == idx_to;

I have not looked into it deeply, but could you summarize the difference between `find_first_set_bit` and JDK 11's `get_next_one_offset`? I thought `find_first_set_bit` does the same thing.

src/hotspot/share/tsan/tsanOopMap.cpp line 263:

> 261: // OopStorage so the number matches with the 'weak_count' in oopStorageSet.hpp.
> 262: void TsanOopMap::initialize_map() {
> 263:   // No need to do register_num_dead_callback for concurrent work as we do

Perhaps we can document this more explicitly, perhaps in tsanOopMap.hpp, that currently TsanOopMap/OopStorage must be processed during a stop-the-world GC pause. This is because we need to update TSAN's metadata for all moved or freed objects during GC, before the mutator reads or writes to these objects. Thus, unlikely other weak OopStorage, TsanOopMap does not support concurrent processing, and does not need to call `OopStorage::register_num_dead_callback()`.

src/hotspot/share/tsan/tsanOopMap.cpp line 268:

> 266:   // WeakProcessor.
> 267:   _weak_oop_storage = OopStorageSet::create_weak("Tsan weak OopStorage", mtInternal);
> 268:   assert(_weak_oop_storage != NULL, "sanity");

Let's use `nullptr` instead of `NULL` in these touched lines. (I'm also OK if we replace all `NULL` with `nullptr` in these touched TSAN files).

src/hotspot/share/tsan/tsanOopMap.cpp line 286:

> 284: // Called during GC by WeakProcessor.
> 285: void TsanOopMap::notify_tsan_for_freed_and_moved_objects() {
> 286:   if (_oop_map == nullptr) {

Could this be an `assert(_oop_map != nullptr)`?
Also, do we want to keep the old `assert(SafepointSynchronize::is_at_safepoint())`?

src/hotspot/share/tsan/tsanOopMap.cpp line 302:

> 300: 
> 301:   {
> 302:     MutexLocker mu(TsanOopMap_lock, Mutex::_no_safepoint_check_flag);

Do we really need this lock if it is called during a GC pause by only the VM thread? Currently it is called by multiple GC threads, so the lock might appear necessary.

Also see the comment in the old `TsanOopMap::weak_oops_do`. We should probably keep that comment.

src/hotspot/share/tsan/tsanOopMap.cpp line 316:

> 314:     max_high = MAX2(source_high, target_high);
> 315: 
> 316:     moves.sort((2 * n_downward_moves > moves.length()) ?

Sorting is only needed before calling `handle_overlapping_moves` right? Then it should be in the else branch.

src/hotspot/share/tsan/tsanOopMap.cpp line 450:

> 448:     }
> 449: 
> 450:     // Source and target ranges overlap, the moves need to be ordered to prevent

Can we retain this comment paragraph?

src/hotspot/share/tsan/tsanOopMap.hpp line 46:

> 44: }  // namespace TsanOopMapImpl
> 45: 
> 46: #include "tsan/tsanOopMapTable.hpp"

How about moving `PendingMove` declaration to tsanOopMapTable.hpp, so this `#include` statement could be immediately after `#define SHARE_TSAN_TSANOOPMAP_HPP`?

-------------

Changes requested by manc (Reviewer).

PR Review: https://git.openjdk.org/tsan/pull/19#pullrequestreview-2192687934
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1687251963
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1687231901
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1687289253
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1687286079
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1687294251
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1687278527
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1687305054
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1687304054
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1687307604
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1687310668
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1687274453

From jiangli at openjdk.org  Tue Jul 23 18:29:31 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Tue, 23 Jul 2024 18:29:31 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v2]
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <ez-TX083sJJNlHmHsNKBS4ZyybR0w1djR3Q7qDCmcR4=.7f47e7bd-1e01-4158-bd1c-77a10c3255fd@github.com>

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: data race (pid=787116)...

Jiangli Zhou has updated the pull request incrementally with three additional commits since the last revision:

 - - Replace the `if` check with `assert(_oop_map != nullptr, "must be") in `TsanOopMap::notify_tsan_for_freed_and_moved_objects().
     Add SafepointSynchronize::is_at_safepoint() assert check. I rechecked these asserts with a slowdebug binary.
   
   - Add back "Source and target ranges overlap..." comments for sorting moves.
 - Move PendingMove declaration to tsanOopMapTable.hpp.
 - Remove newline.

-------------

Changes:
  - all: https://git.openjdk.org/tsan/pull/19/files
  - new: https://git.openjdk.org/tsan/pull/19/files/4bd91916..2073896d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=01
 - incr: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=00-01

  Stats: 40 lines in 4 files changed: 19 ins; 18 del; 3 mod
  Patch: https://git.openjdk.org/tsan/pull/19.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/19/head:pull/19

PR: https://git.openjdk.org/tsan/pull/19

From jiangli at openjdk.org  Tue Jul 23 18:57:41 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Tue, 23 Jul 2024 18:57:41 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v2]
In-Reply-To: <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
Message-ID: <pprdeDAvu-I3WaNxydikOnR0iHmIRcoYuYM45ldDHb4=.d063a7b2-5f33-4a19-8bf5-b379e11837c8@github.com>

On Mon, 22 Jul 2024 23:47:48 GMT, Man Cao <manc at openjdk.org> wrote:

> The WeakProcessor::Task::work() function is called by a subset of GC worker thread. This function will be called multiple times in parallel, but each thread runs the same code and processes the same data, so it is redundant work. It should probably be called only once, by the VM thread.

Yes, that's correct. I've noticed it have been thinking updating this part already.

> We can invoke notify_tsan_for_freed_and_moved_objects() from the > VM thread by adding it to the two versions of  WeakProcessor::weak_oops_do below:
>
> After line 142 task.report_num_dead(), in this file (weakProcessor.inline.hpp).
> In weakProcessor.cpp, between inside  WeakProcessor::weak_oops_do(BoolObjectClosure* is_alive, OopClosure* keep_alive).

I think we only need to add `notify_tsan_for_freed_and_moved_objects()` call in `void WeakProcessor::weak_oops_do(WorkerThreads* workers, IsAlive* is_alive, KeepAlive* keep_alive,WeakProcessorTimes* times)` in weakProcessor.inline.hpp. I think that should be able to cover all cases.

`WeakProcessor::weak_oops_do(BoolObjectClosure* is_alive, OopClosure* keep_alive)` in weakProcessor.cpp is only used by serial GC. We can add the call in there, but it would not be fully tested.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1688552281

From jiangli at openjdk.org  Tue Jul 23 20:40:56 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Tue, 23 Jul 2024 20:40:56 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v2]
In-Reply-To: <pprdeDAvu-I3WaNxydikOnR0iHmIRcoYuYM45ldDHb4=.d063a7b2-5f33-4a19-8bf5-b379e11837c8@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
 <pprdeDAvu-I3WaNxydikOnR0iHmIRcoYuYM45ldDHb4=.d063a7b2-5f33-4a19-8bf5-b379e11837c8@github.com>
Message-ID: <vlm5pTmc8U78XhQOL8Ii4GEM0dV4M5r_JrE0dTon7iM=.01f5b2f5-d254-4c1d-8b23-63dea00739f8@github.com>

On Tue, 23 Jul 2024 18:55:32 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

>> src/hotspot/share/gc/shared/weakProcessor.inline.hpp line 100:
>> 
>>> 98: 
>>> 99:   TSAN_RUNTIME_ONLY(
>>> 100:     TsanOopMap::notify_tsan_for_freed_and_moved_objects();
>> 
>> The `WeakProcessor::Task::work()` function is called by a subset of GC worker thread. This function will be called multiple times in parallel, but each thread runs the same code and processes the same data, so it is redundant work. It should probably be called only once, by the VM thread.
>> 
>> We can invoke `notify_tsan_for_freed_and_moved_objects()` from the VM thread by adding it to the two versions of `WeakProcessor::weak_oops_do` below:
>> - After line 142 `task.report_num_dead()`, in this file (weakProcessor.inline.hpp).
>> - In weakProcessor.cpp, between inside `WeakProcessor::weak_oops_do(BoolObjectClosure* is_alive, OopClosure* keep_alive)`.
>> 
>> We have another internal patch that adds a callback during a GC pause, at those two exact places. It is very similar to what TSAN needs to do.
>
>> The WeakProcessor::Task::work() function is called by a subset of GC worker thread. This function will be called multiple times in parallel, but each thread runs the same code and processes the same data, so it is redundant work. It should probably be called only once, by the VM thread.
> 
> Yes, that's correct. I've noticed it have been thinking updating this part already.
> 
>> We can invoke notify_tsan_for_freed_and_moved_objects() from the > VM thread by adding it to the two versions of  WeakProcessor::weak_oops_do below:
>>
>> After line 142 task.report_num_dead(), in this file (weakProcessor.inline.hpp).
>> In weakProcessor.cpp, between inside  WeakProcessor::weak_oops_do(BoolObjectClosure* is_alive, OopClosure* keep_alive).
> 
> I think we only need to add `notify_tsan_for_freed_and_moved_objects()` call in `void WeakProcessor::weak_oops_do(WorkerThreads* workers, IsAlive* is_alive, KeepAlive* keep_alive,WeakProcessorTimes* times)` in weakProcessor.inline.hpp. I think that should be able to cover all cases.
> 
> `WeakProcessor::weak_oops_do(BoolObjectClosure* is_alive, OopClosure* keep_alive)` in weakProcessor.cpp is only used by serial GC. We can add the call in there, but it would not be fully tested.

My further testing indicates moving `notify_tsan_for_freed_and_moved_objects()` from `WeakProcessor::Task::work()` to `WeakProcessor::weak_oops_do` calls cannot cover all cases.


 stdout: [];
 stderr: [ThreadSanitizer: CHECK failed: tsan_sync.cpp:261 "((*dst_meta)) == ((0))" (0x80000238, 0x0) (tid=709887)
    #0 __tsan::CheckUnwind() <null> (java+0xbc4ff) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
    #1 __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) <null> (java+0x34b26) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
    #2 __tsan::MetaMap::MoveMemory(unsigned long, unsigned long, unsigned long) <null> (java+0xd40c7) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
    #3 __tsan_java_move <null> (java+0xb2919) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
    #4 TsanOopMap::notify_tsan_for_freed_and_moved_objects() make/hotspot/src/hotspot/share/tsan/tsanOopMap.cpp:326:9 (libjvm.so+0xe1e5a2) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
    #5 void WeakProcessor::weak_oops_do<G1IsAliveClosure, DoNothingClosure>(WorkerThreads*, G1IsAliveClosure*, DoNothingClosure*, WeakProcessorTimes*) make/hotspot/src/hotspot/share/gc/shared/weakProcessor.inline.hpp:145:3 (libjvm.so+0x73d991) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
    #6 void WeakProcessor::weak_oops_do<G1IsAliveClosure, DoNothingClosure>(WorkerThreads*, G1IsAliveClosure*, DoNothingClosure*, unsigned int) make/hotspot/src/hotspot/share/gc/shared/weakProcessor.inline.hpp:159:3 (libjvm.so+0x73645c) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
    #7 G1FullCollector::phase1_mark_live_objects() make/hotspot/src/hotspot/share/gc/g1/g1FullCollector.cpp:316:5 (libjvm.so+0x73645c)
    #8 G1FullCollector::collect() make/hotspot/src/hotspot/share/gc/g1/g1FullCollector.cpp:207:3 (libjvm.so+0x735e07) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
    #9 G1CollectedHeap::do_full_collection(bool, bool) make/hotspot/src/hotspot/share/gc/g1/g1CollectedHeap.cpp:932:13 (libjvm.so+0x7146d5) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
    #10 VM_G1CollectFull::doit() make/hotspot/src/hotspot/share/gc/g1/g1VMOperations.cpp:54:24 (libjvm.so+0x7816e1) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
    #11 VM_Operation::evaluate() make/hotspot/src/hotspot/share/runtime/vmOperations.cpp:71:3 (libjvm.so+0xe8bd32) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
    #12 VMThread::evaluate_operation(VM_Operation*) make/hotspot/src/hotspot/share/runtime/vmThread.cpp:281:9 (libjvm.so+0xe8db1e) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
    #13 VMThread::inner_execute(VM_Operation*) make/hotspot/src/hotspot/share/runtime/vmThread.cpp:435:3 (libjvm.so+0xe8e13e) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
    #14 VMThread::loop() make/hotspot/src/hotspot/share/runtime/vmThread.cpp:502:5 (libjvm.so+0xe8d829) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
    #15 VMThread::run() make/hotspot/src/hotspot/share/runtime/vmThread.cpp:175:9 (libjvm.so+0xe8d829)
    #16 Thread::call_run() make/hotspot/src/hotspot/share/runtime/thread.cpp:217:9 (libjvm.so+0xe0efe4) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
    #17 thread_native_entry(Thread*) make/hotspot/src/hotspot/os/linux/os_linux.cpp:778:11 (libjvm.so+0xbe9da8) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
    #18 __tsan_thread_start_func <null> (java+0x4f2d2) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
    #19 start_thread nptl/pthread_create.c:444:8 (libc.so.6+0x8845b) (BuildId: b830dcb9c6f0ee1f8679364e0f99ce35db953847)
    #20 clone3 misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 (libc.so.6+0x108bbb) (BuildId: b830dcb9c6f0ee1f8679364e0f99ce35db953847)


Looking more closely now, I see `WeakProcessor::Task::work()` is also used in `G1FullGCAdjustTask::work` and `PSAdjustTask`.  I'm reluctant to add the calls in various different places  in GC code, as that's less maintainable. 

I'm looking into a bit more to see to reduce the overhead by doing it in the concurrent worker thread.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1688687454

From jiangli at openjdk.org  Tue Jul 23 20:53:47 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Tue, 23 Jul 2024 20:53:47 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v2]
In-Reply-To: <vlm5pTmc8U78XhQOL8Ii4GEM0dV4M5r_JrE0dTon7iM=.01f5b2f5-d254-4c1d-8b23-63dea00739f8@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
 <pprdeDAvu-I3WaNxydikOnR0iHmIRcoYuYM45ldDHb4=.d063a7b2-5f33-4a19-8bf5-b379e11837c8@github.com>
 <vlm5pTmc8U78XhQOL8Ii4GEM0dV4M5r_JrE0dTon7iM=.01f5b2f5-d254-4c1d-8b23-63dea00739f8@github.com>
Message-ID: <gXINeRlK2Z01pad-cBFwRmcp9ktnLvywZiKuN6DDZJQ=.e89247ec-273c-45e1-93c1-3ae8b778ba99@github.com>

On Tue, 23 Jul 2024 20:38:20 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

>>> The WeakProcessor::Task::work() function is called by a subset of GC worker thread. This function will be called multiple times in parallel, but each thread runs the same code and processes the same data, so it is redundant work. It should probably be called only once, by the VM thread.
>> 
>> Yes, that's correct. I've noticed it have been thinking updating this part already.
>> 
>>> We can invoke notify_tsan_for_freed_and_moved_objects() from the > VM thread by adding it to the two versions of  WeakProcessor::weak_oops_do below:
>>>
>>> After line 142 task.report_num_dead(), in this file (weakProcessor.inline.hpp).
>>> In weakProcessor.cpp, between inside  WeakProcessor::weak_oops_do(BoolObjectClosure* is_alive, OopClosure* keep_alive).
>> 
>> I think we only need to add `notify_tsan_for_freed_and_moved_objects()` call in `void WeakProcessor::weak_oops_do(WorkerThreads* workers, IsAlive* is_alive, KeepAlive* keep_alive,WeakProcessorTimes* times)` in weakProcessor.inline.hpp. I think that should be able to cover all cases.
>> 
>> `WeakProcessor::weak_oops_do(BoolObjectClosure* is_alive, OopClosure* keep_alive)` in weakProcessor.cpp is only used by serial GC. We can add the call in there, but it would not be fully tested.
>
> My further testing indicates moving `notify_tsan_for_freed_and_moved_objects()` from `WeakProcessor::Task::work()` to `WeakProcessor::weak_oops_do` calls cannot cover all cases.
> 
> 
>  stdout: [];
>  stderr: [ThreadSanitizer: CHECK failed: tsan_sync.cpp:261 "((*dst_meta)) == ((0))" (0x80000238, 0x0) (tid=709887)
>     #0 __tsan::CheckUnwind() <null> (java+0xbc4ff) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
>     #1 __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) <null> (java+0x34b26) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
>     #2 __tsan::MetaMap::MoveMemory(unsigned long, unsigned long, unsigned long) <null> (java+0xd40c7) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
>     #3 __tsan_java_move <null> (java+0xb2919) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
>     #4 TsanOopMap::notify_tsan_for_freed_and_moved_objects() make/hotspot/src/hotspot/share/tsan/tsanOopMap.cpp:326:9 (libjvm.so+0xe1e5a2) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>     #5 void WeakProcessor::weak_oops_do<G1IsAliveClosure, DoNothingClosure>(WorkerThreads*, G1IsAliveClosure*, DoNothingClosure*, WeakProcessorTimes*) make/hotspot/src/hotspot/share/gc/shared/weakProcessor.inline.hpp:145:3 (libjvm.so+0x73d991) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>     #6 void WeakProcessor::weak_oops_do<G1IsAliveClosure, DoNothingClosure>(WorkerThreads*, G1IsAliveClosure*, DoNothingClosure*, unsigned int) make/hotspot/src/hotspot/share/gc/shared/weakProcessor.inline.hpp:159:3 (libjvm.so+0x73645c) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>     #7 G1FullCollector::phase1_mark_live_objects() make/hotspot/src/hotspot/share/gc/g1/g1FullCollector.cpp:316:5 (libjvm.so+0x73645c)
>     #8 G1FullCollector::collect() make/hotspot/src/hotspot/share/gc/g1/g1FullCollector.cpp:207:3 (libjvm.so+0x735e07) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>     #9 G1CollectedHeap::do_full_collection(bool, bool) make/hotspot/src/hotspot/share/gc/g1/g1CollectedHeap.cpp:932:13 (libjvm.so+0x7146d5) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>     #10 VM_G1CollectFull::doit() make/hotspot/src/hotspot/share/gc/g1/g1VMOperations.cpp:54:24 (libjvm.so+0x7816e1) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>     #11 VM_Operation::evaluate() make/hotspot/src/hotspot/share/runtime/vmOperations.cpp:71:3 (libjvm.so+0xe8bd32) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>     #12 VMThread::evaluate_operation(VM_Operation*) ...

Thinking more of  `TsanOopMap::notify_tsan_for_freed_and_moved_objects()` and `TsanOopMapTable::collect_moved_objects_and_notify_freed()`, I think the overhead is minimum.

Each concurrent worker thread iterates the TsanOopMapTable. However, the table entries are immediately updated by the worker thread:

- entry is removed when a freed object is encountered
- entry is updated to point to the new address when a moved object is encountered

So each entry in the table is actually processed only once. Tsan is also only notified once for any freed or moved object.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1688700198

From jiangli at openjdk.org  Tue Jul 23 22:56:21 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Tue, 23 Jul 2024 22:56:21 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v3]
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <MiY279XUBSMwfaiIKDZintmqkM8gmr6z8pK0mM_GADA=.47c69687-b93c-4a5e-85f5-0c716d5139b0@github.com>

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: data race (pid=787116)...

Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:

  Remove TsanOopBitMap. We can use CHeapBitMap instead. Thanks manc@ for catching it in https://github.com/openjdk/tsan/pull/19/files#r1687286079.

-------------

Changes:
  - all: https://git.openjdk.org/tsan/pull/19/files
  - new: https://git.openjdk.org/tsan/pull/19/files/2073896d..a41bc987

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=02
 - incr: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=01-02

  Stats: 83 lines in 2 files changed: 0 ins; 80 del; 3 mod
  Patch: https://git.openjdk.org/tsan/pull/19.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/19/head:pull/19

PR: https://git.openjdk.org/tsan/pull/19

From jiangli at openjdk.org  Tue Jul 23 22:58:40 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Tue, 23 Jul 2024 22:58:40 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v3]
In-Reply-To: <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
Message-ID: <lqH2fh-KmriSa2GwxJPCzveCJXKXb0ib6MQPB_4J2BI=.3a8cc3aa-228f-4e47-abd4-17f52655f3cb@github.com>

On Tue, 23 Jul 2024 00:48:43 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Remove TsanOopBitMap. We can use CHeapBitMap instead. Thanks manc@ for catching it in https://github.com/openjdk/tsan/pull/19/files#r1687286079.
>
> src/hotspot/share/tsan/tsanOopMap.cpp line 240:
> 
>> 238:           // Indicate that this move has been done and remember that we
>> 239:           // made some progress.
>> 240:           m.n_bytes = 0;
> 
> I have not looked into it deeply, but could you summarize the difference between `find_first_set_bit` and JDK 11's `get_next_one_offset`? I thought `find_first_set_bit` does the same thing.

Thanks for catching this! I had forgotten that `find_first_set_bit|get_next_one_offset` changes has already resolved and merged.

I've removed `TsanOopBitMap` completely. This is cleaner now.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1688808837

From manc at openjdk.org  Tue Jul 23 23:15:40 2024
From: manc at openjdk.org (Man Cao)
Date: Tue, 23 Jul 2024 23:15:40 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v3]
In-Reply-To: <gXINeRlK2Z01pad-cBFwRmcp9ktnLvywZiKuN6DDZJQ=.e89247ec-273c-45e1-93c1-3ae8b778ba99@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
 <pprdeDAvu-I3WaNxydikOnR0iHmIRcoYuYM45ldDHb4=.d063a7b2-5f33-4a19-8bf5-b379e11837c8@github.com>
 <vlm5pTmc8U78XhQOL8Ii4GEM0dV4M5r_JrE0dTon7iM=.01f5b2f5-d254-4c1d-8b23-63dea00739f8@github.com>
 <gXINeRlK2Z01pad-cBFwRmcp9ktnLvywZiKuN6DDZJQ=.e89247ec-273c-45e1-93c1-3ae8b778ba99@github.com>
Message-ID: <BkTetEqM5tQMrECJdL-M2X3bG0q00t9us_uMyjB1PQY=.8eb9547a-a146-40e5-ab54-9896e5303308@github.com>

On Tue, 23 Jul 2024 20:51:06 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

>> My further testing indicates moving `notify_tsan_for_freed_and_moved_objects()` from `WeakProcessor::Task::work()` to `WeakProcessor::weak_oops_do` calls cannot cover all cases.
>> 
>> 
>>  stdout: [];
>>  stderr: [ThreadSanitizer: CHECK failed: tsan_sync.cpp:261 "((*dst_meta)) == ((0))" (0x80000238, 0x0) (tid=709887)
>>     #0 __tsan::CheckUnwind() <null> (java+0xbc4ff) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
>>     #1 __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) <null> (java+0x34b26) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
>>     #2 __tsan::MetaMap::MoveMemory(unsigned long, unsigned long, unsigned long) <null> (java+0xd40c7) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
>>     #3 __tsan_java_move <null> (java+0xb2919) (BuildId: 64a25ac2795c9dd62d0a35fdf577c14b358e86be)
>>     #4 TsanOopMap::notify_tsan_for_freed_and_moved_objects() make/hotspot/src/hotspot/share/tsan/tsanOopMap.cpp:326:9 (libjvm.so+0xe1e5a2) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>>     #5 void WeakProcessor::weak_oops_do<G1IsAliveClosure, DoNothingClosure>(WorkerThreads*, G1IsAliveClosure*, DoNothingClosure*, WeakProcessorTimes*) make/hotspot/src/hotspot/share/gc/shared/weakProcessor.inline.hpp:145:3 (libjvm.so+0x73d991) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>>     #6 void WeakProcessor::weak_oops_do<G1IsAliveClosure, DoNothingClosure>(WorkerThreads*, G1IsAliveClosure*, DoNothingClosure*, unsigned int) make/hotspot/src/hotspot/share/gc/shared/weakProcessor.inline.hpp:159:3 (libjvm.so+0x73645c) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>>     #7 G1FullCollector::phase1_mark_live_objects() make/hotspot/src/hotspot/share/gc/g1/g1FullCollector.cpp:316:5 (libjvm.so+0x73645c)
>>     #8 G1FullCollector::collect() make/hotspot/src/hotspot/share/gc/g1/g1FullCollector.cpp:207:3 (libjvm.so+0x735e07) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>>     #9 G1CollectedHeap::do_full_collection(bool, bool) make/hotspot/src/hotspot/share/gc/g1/g1CollectedHeap.cpp:932:13 (libjvm.so+0x7146d5) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>>     #10 VM_G1CollectFull::doit() make/hotspot/src/hotspot/share/gc/g1/g1VMOperations.cpp:54:24 (libjvm.so+0x7816e1) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>>     #11 VM_Operation::evaluate() make/hotspot/src/hotspot/share/runtime/vmOperations.cpp:71:3 (libjvm.so+0xe8bd32) (BuildId: 8cda182fb0a5fe22eb8242eab7d73b7123b7659b)
>>     #12 VMThread...
>
> Thinking more of  `TsanOopMap::notify_tsan_for_freed_and_moved_objects()` and `TsanOopMapTable::collect_moved_objects_and_notify_freed()`, I think the overhead is minimum.
> 
> Each concurrent worker thread iterates the TsanOopMapTable. However, the table entries are immediately updated by the worker thread:
> 
> - entry is removed when a freed object is encountered
> - entry is updated to point to the new address when a moved object is encountered
> 
> So each entry in the table is actually processed only once. Tsan is also only notified once for any freed or moved object.

We looked into this offline. Yes, the failure is indeed due to the missing call to `notify_tsan_for_freed_and_moved_objects()` in `G1FullGCAdjustTask` and `PSAdjustTask`. In particular, in G1 and Parallel GC's full collector, we have to process TSAN oop map at least twice:
1. In Phase 1 after marking. This iteration gets rid of dead objects from TSAN oop map, but it does not process to-be-moved objects, because the destination addresses (forwarding pointers) are not yet available.
2. In Phase 3 after or during adjusting pointers. This iteration processes to-be-moved objects in TSAN oop map, as forwarding pointers are available during this phase.

In JDK 21, we cannot find a shared function in weak processor that is executed by VM thread from G1 and Parallel collector's adjusting pointers phase. We have two options:

a. Keep Jiangli's current implementation, that the TSAN oop map is iterated by multiple GC worker threads. The worker threads need to hold `TsanOopMap_lock`. Also note that `notify_tsan_for_freed_and_moved_objects()` needs to run after the existing code in `WeakProcessor::Task::work` iterates through weak OopStorage in `OopStorageSet` by all worker threads. It appears difficult to correctly make `notify_tsan_for_freed_and_moved_objects()` called by only one GC worker thread.

b. Insert `notify_tsan_for_freed_and_moved_objects()` to G1 and Parallel GC's full collector (`G1FullCollector::phase3_adjust_pointers()` and `PSParallelCompact::adjust_roots()`), in addition to the two versions of `WeakProcessor::weak_oops_do()`. The downside is that we will be modifying code specific to a garbage collector.

It is not obvious which approach will be easier to maintain in the long term. I slightly prefer `b.`, but `a.` is also OK. The performance downside of `a.` is not a big concern as TSAN is not optimized for performance.

Also note that TSAN currently is only tested under G1, and it is unknown if it works correctly under ParallelGC. I'll update TSAN's wiki page to document this. With this change, for approach `a.` TSAN won't work under serial GC; for `b.` TSAN probably does not work for Shenandoah. Neither approach is likely to work under ZGC or XGC.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1688819270

From jiangli at openjdk.org  Tue Jul 23 23:39:23 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Tue, 23 Jul 2024 23:39:23 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v4]
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <ETwbTLu05T4XNJUmOx5g1RvyR4jHagFToUm6hze0U9I=.cc789275-6b65-43b1-a8b9-fa89809b376d@github.com>

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: data race (pid=787116)...

Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:

  Update TsanOopMap.hpp comment with more details.

-------------

Changes:
  - all: https://git.openjdk.org/tsan/pull/19/files
  - new: https://git.openjdk.org/tsan/pull/19/files/a41bc987..dd6a4c45

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=03
 - incr: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=02-03

  Stats: 8 lines in 1 file changed: 8 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/tsan/pull/19.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/19/head:pull/19

PR: https://git.openjdk.org/tsan/pull/19

From jiangli at openjdk.org  Tue Jul 23 23:39:24 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Tue, 23 Jul 2024 23:39:24 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v4]
In-Reply-To: <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
Message-ID: <B5QrSiZlcfQsII_vvp01rf4VqyHneNrBusMSyIi8Z5E=.5417a98e-5eb0-4c58-9e32-2073f49f6947@github.com>

On Tue, 23 Jul 2024 01:04:48 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Update TsanOopMap.hpp comment with more details.
>
> src/hotspot/share/tsan/tsanOopMap.cpp line 263:
> 
>> 261: // OopStorage so the number matches with the 'weak_count' in oopStorageSet.hpp.
>> 262: void TsanOopMap::initialize_map() {
>> 263:   // No need to do register_num_dead_callback for concurrent work as we do
> 
> Perhaps we can document this more explicitly, perhaps in tsanOopMap.hpp, that currently TsanOopMap/OopStorage must be processed during a stop-the-world GC pause. This is because we need to update TSAN's metadata for all moved or freed objects during GC, before the mutator reads or writes to these objects. Thus, unlikely other weak OopStorage, TsanOopMap does not support concurrent processing, and does not need to call `OopStorage::register_num_dead_callback()`.

SG. I added more details in tsanOopMap.hpp.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1688873401

From jiangli at openjdk.org  Tue Jul 23 23:45:40 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Tue, 23 Jul 2024 23:45:40 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v4]
In-Reply-To: <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
Message-ID: <3aI-FGJ-1_i_EztJVAxSBVVGMNsLBC3ggfuMZzoEOaE=.833dfc5a-96f8-43c7-9f91-da275a429e68@github.com>

On Tue, 23 Jul 2024 01:23:40 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Update TsanOopMap.hpp comment with more details.
>
> src/hotspot/share/tsan/tsanOopMap.cpp line 302:
> 
>> 300: 
>> 301:   {
>> 302:     MutexLocker mu(TsanOopMap_lock, Mutex::_no_safepoint_check_flag);
> 
> Do we really need this lock if it is called during a GC pause by only the VM thread? Currently it is called by multiple GC threads, so the lock might appear necessary.
> 
> Also see the comment in the old `TsanOopMap::weak_oops_do`. We should probably keep that comment.

We need the lock here as it's done in concurrent worker threads that handle the WeakProcessor::WeakOopsDoTasks.

As discussed offline today, doing the tsanOopMap processing and tsan notification in WeakProcessor::Task::work is cleaner and more maintainable than other possible alternatives.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1688889861

From jiangli at openjdk.org  Wed Jul 24 00:00:31 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 00:00:31 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v5]
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <-XQ0P-4WvQUxTY5ObG1Tg_SdTbO5Ig17gS_c3mU6Zjk=.4c157b00-3f70-4844-b470-e9e05f25b689@github.com>

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: data race (pid=787116)...

Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:

  Change to use log_debug for the pre-existing DEBUG_PRINT.

-------------

Changes:
  - all: https://git.openjdk.org/tsan/pull/19/files
  - new: https://git.openjdk.org/tsan/pull/19/files/dd6a4c45..80c1f558

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=04
 - incr: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=03-04

  Stats: 6 lines in 1 file changed: 0 ins; 0 del; 6 mod
  Patch: https://git.openjdk.org/tsan/pull/19.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/19/head:pull/19

PR: https://git.openjdk.org/tsan/pull/19

From jiangli at openjdk.org  Wed Jul 24 00:05:40 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 00:05:40 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v5]
In-Reply-To: <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
Message-ID: <7LQmrh60_2-IIuBFOqymPOlQ6jA-Skgd7ylR80kkAh0=.fa7151c6-b5e3-40f1-a9ef-8faaea71644f@github.com>

On Tue, 23 Jul 2024 00:55:05 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Change to use log_debug for the pre-existing DEBUG_PRINT.
>
> src/hotspot/share/tsan/tsanOopMap.cpp line 232:
> 
>> 230:         if (can_move) {
>> 231:           // Notify TSan, update occupied region.
>> 232:           log_trace(tsan)("__tsan_java_move for [" PTR_FORMAT ", " PTR_FORMAT  "] -> [" PTR_FORMAT ", " PTR_FORMAT "]\n",
> 
> How about using `log_trace` only for this line, the line in `notify_tsan_for_freed_and_moved_objects` and the line above `__tsan_java_alloc`, and using `log_debug` for all other logging lines?
> I suppose these 3 lines print a lot more messages than other logging lines.

Changed to use log_debug for the pre-existing DEBUG_PRINT calls. The new ones are kept as log_trace. Note, log_trace is higher level (more verbose) than log_debug.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1688932703

From jiangli at openjdk.org  Wed Jul 24 00:24:18 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 00:24:18 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: data race (pid=787116)...

Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:

  Sort moves for overlapping case only.

-------------

Changes:
  - all: https://git.openjdk.org/tsan/pull/19/files
  - new: https://git.openjdk.org/tsan/pull/19/files/80c1f558..a4496362

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=05
 - incr: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=04-05

  Stats: 10 lines in 1 file changed: 5 ins; 5 del; 0 mod
  Patch: https://git.openjdk.org/tsan/pull/19.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/19/head:pull/19

PR: https://git.openjdk.org/tsan/pull/19

From jiangli at openjdk.org  Wed Jul 24 00:32:48 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 00:32:48 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
Message-ID: <m2yJUErGXtigo8LkrdDvSoW4GIMXpnH4sScAR5X5-ZI=.91cb824b-a0b3-4c50-85f0-de1f489ee42d@github.com>

On Tue, 23 Jul 2024 01:30:21 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Sort moves for overlapping case only.
>
> src/hotspot/share/tsan/tsanOopMap.cpp line 316:
> 
>> 314:     max_high = MAX2(source_high, target_high);
>> 315: 
>> 316:     moves.sort((2 * n_downward_moves > moves.length()) ?
> 
> Sorting is only needed before calling `handle_overlapping_moves` right? Then it should be in the else branch.

I moved around `moves.sort` a few times for the disjoint and overlapping cases during implementing/testing, due to the `((*dst_meta)) == ((0))` check failures. 

I agree that it only needs sorting for the overlapping case. I just retested with sorting for overlapping case only. All tsan jtreg tests still pass with a release JDK binary.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1688960726

From jiangli at openjdk.org  Wed Jul 24 00:54:40 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 00:54:40 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <BkTetEqM5tQMrECJdL-M2X3bG0q00t9us_uMyjB1PQY=.8eb9547a-a146-40e5-ab54-9896e5303308@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
 <pprdeDAvu-I3WaNxydikOnR0iHmIRcoYuYM45ldDHb4=.d063a7b2-5f33-4a19-8bf5-b379e11837c8@github.com>
 <vlm5pTmc8U78XhQOL8Ii4GEM0dV4M5r_JrE0dTon7iM=.01f5b2f5-d254-4c1d-8b23-63dea00739f8@github.com>
 <gXINeRlK2Z01pad-cBFwRmcp9ktnLvywZiKuN6DDZJQ=.e89247ec-273c-45e1-93c1-3ae8b778ba99@github.com>
 <BkTetEqM5tQMrECJdL-M2X3bG0q00t9us_uMyjB1PQY=.8eb9547a-a146-40e5-ab54-9896e5303308@github.com>
Message-ID: <ErvjvXoDWMlMFK9q8nm-zqyd0kxtY3PRNtYA56056R8=.12e7b809-f577-46fe-bb04-43106d077c3a@github.com>

On Tue, 23 Jul 2024 23:13:32 GMT, Man Cao <manc at openjdk.org> wrote:

>> Thinking more of  `TsanOopMap::notify_tsan_for_freed_and_moved_objects()` and `TsanOopMapTable::collect_moved_objects_and_notify_freed()`, I think the overhead is minimum.
>> 
>> Each concurrent worker thread iterates the TsanOopMapTable. However, the table entries are immediately updated by the worker thread:
>> 
>> - entry is removed when a freed object is encountered
>> - entry is updated to point to the new address when a moved object is encountered
>> 
>> So each entry in the table is actually processed only once. Tsan is also only notified once for any freed or moved object.
>
> We looked into this offline. Yes, the failure is indeed due to the missing call to `notify_tsan_for_freed_and_moved_objects()` in `G1FullGCAdjustTask` and `PSAdjustTask`. In particular, in G1 and Parallel GC's full collector, we have to process TSAN oop map at least twice:
> 1. In Phase 1 after marking. This iteration gets rid of dead objects from TSAN oop map, but it does not process to-be-moved objects, because the destination addresses (forwarding pointers) are not yet available.
> 2. In Phase 3 after or during adjusting pointers. This iteration processes to-be-moved objects in TSAN oop map, as forwarding pointers are available during this phase.
> 
> In JDK 21, we cannot find a shared function in weak processor that is executed by VM thread from G1 and Parallel collector's adjusting pointers phase. We have two options:
> 
> a. Keep Jiangli's current implementation, that the TSAN oop map is iterated by multiple GC worker threads. The worker threads need to hold `TsanOopMap_lock`. Also note that `notify_tsan_for_freed_and_moved_objects()` needs to run after the existing code in `WeakProcessor::Task::work` iterates through weak OopStorage in `OopStorageSet` by all worker threads. It appears difficult to correctly make `notify_tsan_for_freed_and_moved_objects()` called by only one GC worker thread.
> 
> b. Insert `notify_tsan_for_freed_and_moved_objects()` to G1 and Parallel GC's full collector (`G1FullCollector::phase3_adjust_pointers()` and `PSParallelCompact::adjust_roots()`), in addition to the two versions of `WeakProcessor::weak_oops_do()`. The downside is that we will be modifying code specific to a garbage collector.
> 
> It is not obvious which approach will be easier to maintain in the long term. I slightly prefer `b.`, but `a.` is also OK. The performance downside of `a.` is not a big concern as TSAN is not optimized for performance.
> 
> Also note that TSAN currently is only tested under G1, and it is unknown if it works correctly under ParallelGC. I'll update TSAN's wiki page to document this. With this change, for approach `a.` TSAN won't work under serial GC; for `b.` TSAN probably does not work for Shenandoah. Neither approach is likely to work under ZGC or XGC.

@caoman thanks for documenting our offline discussion and considerations in details in your above comment.

As I briefly mentioned during the offline discussion, it's also possible to handle the TsanOopMap update and tsan notification in `WeakProcessor::CountingClosure` as another alternative. It could avoid iterating the TsanOopMap table, however I think that's more intrusive change.

I agree optimizing for tsan is a lower priority for now.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1688973154

From manc at openjdk.org  Wed Jul 24 03:21:43 2024
From: manc at openjdk.org (Man Cao)
Date: Wed, 24 Jul 2024 03:21:43 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
Message-ID: <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>

On Wed, 24 Jul 2024 00:24:18 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

>> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
>> 
>> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
>> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
>> 
>> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
>> 
>>  - A Weakhandle that holds a pointer to the oop;
>>  - The original (or updated) address of the object in Java heap;
>> 
>> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.
>> 
>> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
>> 
>> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
>> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
>> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
>> 
>> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
>> 
>> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
>> 
>> Suppressed following Tsan errors in JDK:
>> 1)  In `java.util.concurrent.locks`, for
>>...
>
> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Sort moves for overlapping case only.

Changes requested by manc (Reviewer).

src/hotspot/share/tsan/tsanOopMap.cpp line 279:

> 277:   // We need object size when notify tsan about a freed object.
> 278:   // We cannot call size() for an object after it's freed, so we
> 279:   // need to save the size information in the table.

Could we move this comment about the size field to tsanOopMapTable.hpp? It was in tsanOopMap.hpp before.

src/hotspot/share/tsan/tsanOopMap.cpp line 284:

> 282: 
> 283: #ifdef ASSERT
> 284: bool TsanOopMap::exists(oopDesc *addr) {

Is `TsanOopMap::exists` dead code? If so, we can delete it and the `find`, `is_empty` methods in `tsanOppMapTable`.

src/hotspot/share/tsan/tsanOopMapTable.cpp line 49:

> 47: 
> 48: void TsanOopMapTableKey::update_obj() {
> 49:   oop obj = _wh.peek();

Since `_obj` is `oopDesc*` and `oop` could be an object, is it better to do `oopDesc* obj = _wh.peek()`?

src/hotspot/share/tsan/tsanOopMapTable.cpp line 134:

> 132:                                      _dest_low(dest_low), _dest_high(dest_high),
> 133:                                      _n_downward_moves(n_downward_moves) {}
> 134:     bool do_entry(TsanOopMapTableKey& entry, uintx size) {

`uintx` should also be `jlong`, right?

src/hotspot/share/tsan/tsanOopMapTable.hpp line 25:

> 23:  */
> 24: 
> 25: #ifndef SHARE_VM_PRIMS_TSAN_OOPMAPTABLE_HPP

The `VM_PRIMS_` part can be removed and it misses a `TSAN` prefix. It should be `SHARE_TSAN_TSANOOPMAPTABLE_HPP`.

src/hotspot/share/tsan/tsanOopMapTable.hpp line 67:

> 65: 
> 66:   void release_weak_handle() const;
> 67:   oop object_no_keepalive() const;

I'm a bit unsure about the use of `oop` type vs `oopDesc*` type. Could we keep using `oopDesc*` type in `object_no_keepalive()`, `obj()`, `equals()` and `update_obj()`? E.g., whenever calling `_wh.peek()`, immediately cast the return value to a `oopDesc*`.

They are the same type in a release build, but different in a debug build. I vaguely recall that TsanOopSizeMap used `oop` in the JDK7/8 era, but switched to `oopDesc*` due to errors in debug build.

src/hotspot/share/tsan/tsanOopMapTable.hpp line 74:

> 72:   static unsigned get_hash(const TsanOopMapTableKey& entry) {
> 73:     assert(entry._obj != nullptr, "sanity");
> 74:     return (unsigned int)entry._obj->identity_hash();

In theory `entry._obj` could be an invalid object (old object that is moved or freed), so `identity_hash()` may misbehave. In practice, this probably never happens due to the way we process the hash table in a GC pause. This may deserve a comment, or may be an assert such as `Metaspace::contains(entry._obj->klass())`. But feel free to do nothing.

src/hotspot/share/tsan/tsanOopMapTable.hpp line 78:

> 76: 
> 77:   static bool equals(const TsanOopMapTableKey& lhs, const TsanOopMapTableKey& rhs) {
> 78:     oop lhs_obj = lhs._obj != nullptr ? (oop)lhs._obj : lhs.object_no_keepalive();

I think `lhs._obj` and `rhs._obj` will never be `nullptr` in practice. This is because:
- We never add a null oop to the table.
- Once `_wh.peek()` returns null, the associated entry is immediately removed. We never update `_obj` to null.

Would it better to add an assert that `lhs._obj` and `rhs._obj` are both non-null and remove the call to `object_no_keepalive()`?

(I find it a bit hard to reason why we would resort to `_wh` for equality. I know `JvmtiTagMapKey::_obj` would be null most of the time in contrast.)

src/hotspot/share/tsan/tsanOopMapTable.hpp line 104:

> 102:   unsigned size() const { return _table.table_size(); };
> 103: 
> 104:   bool add_oop_with_size(oop obj, int size);

Type for `size` should be `jlong`, right? As `typedef ResizeableResourceHashtable` indicates.
Let's fix this problem also in `TsanOopMap::add_oop_with_size`.  According to https://github.com/microsoft/compiler-rt/blob/master/test/tsan/java.h, the tsan callbacks take `unsigned long` for size, so `jlong` should be fine.

(We have run into issues with overflowing size field for large objects (>4G), related to JVMTI SampledObjectAlloc. So this is a real bug that could manifest with large objects.)

-------------

PR Review: https://git.openjdk.org/tsan/pull/19#pullrequestreview-2195424296
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1689001611
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1689052679
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1689013309
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1689007134
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1688985804
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1689018481
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1689044556
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1689033960
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1689004755

From jiangli at openjdk.org  Wed Jul 24 17:24:30 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 17:24:30 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v7]
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <F13iBArCaB53QZgXAOo6LSglDT_-bV-JOXqE7KuGiDU=.be16601e-8705-4924-8ed3-7b28e2be76ab@github.com>

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: data race (pid=787116)...

Jiangli Zhou has updated the pull request incrementally with two additional commits since the last revision:

 - - Add comment about saved oop sizes in TsanOopMapTable.
 - - Rename SHARE_VM_PRIMS_TSAN_OOPMAPTABLE_HPP to SHARE_TSAN_TSANOOPMAPTABLE_HPP.

-------------

Changes:
  - all: https://git.openjdk.org/tsan/pull/19/files
  - new: https://git.openjdk.org/tsan/pull/19/files/a4496362..00bcec7f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=06
 - incr: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=05-06

  Stats: 5 lines in 1 file changed: 1 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/tsan/pull/19.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/19/head:pull/19

PR: https://git.openjdk.org/tsan/pull/19

From jiangli at openjdk.org  Wed Jul 24 17:24:31 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 17:24:31 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
Message-ID: <ZPan38m8bRt1_alGueDvxkNfOsyMHTTMd6Mx5C3jdgo=.44b0c553-e053-4514-904b-342c59c03990@github.com>

On Wed, 24 Jul 2024 03:18:42 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Sort moves for overlapping case only.
>
> src/hotspot/share/tsan/tsanOopMap.cpp line 284:
> 
>> 282: 
>> 283: #ifdef ASSERT
>> 284: bool TsanOopMap::exists(oopDesc *addr) {
> 
> Is `TsanOopMap::exists` dead code? If so, we can delete it and the `find`, `is_empty` methods in `tsanOppMapTable`.

`TsanOopMap::exists` is used in `SharedRuntime::tsan_oop_unlock`.

> src/hotspot/share/tsan/tsanOopMapTable.hpp line 25:
> 
>> 23:  */
>> 24: 
>> 25: #ifndef SHARE_VM_PRIMS_TSAN_OOPMAPTABLE_HPP
> 
> The `VM_PRIMS_` part can be removed and it misses a `TSAN` prefix. It should be `SHARE_TSAN_TSANOOPMAPTABLE_HPP`.

Renamed.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690174223
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690175930

From jiangli at openjdk.org  Wed Jul 24 17:27:45 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 17:27:45 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
Message-ID: <GDm7KzZYup9-rZeEop3uU2AQ_LT3TpX1-SWI-BbxQJ4=.4f1c8aac-f411-428a-84ba-2f52558b2dac@github.com>

On Wed, 24 Jul 2024 01:40:30 GMT, Man Cao <manc at openjdk.org> wrote:

> Could we move this comment about the size field to tsanOopMapTable.hpp? It was in tsanOopMap.hpp before.

This is new.

I think you are referring to the following comment in tsanOopMap.hpp:

"// Oop size must be cached, as it is unsafe to call size() after reference is
// collected."

I added following in tsanOopMapTable.hpp:

"// The oop sizes are saved in the table because we need to
// use the size information when notify TSAN about an freed object."

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690180452

From jiangli at openjdk.org  Wed Jul 24 18:26:22 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 18:26:22 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v8]
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <KuWAM_r99bzQNb4NlLdR7MHdGlTIM2sp6hZQJVSa1LI=.9b526f75-1ed8-46e7-b581-ea083cb2a58e@github.com>

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: data race (pid=787116)...

Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:

  - Change the 'size' arg to 'size_t' type in add_oop_with_size() functions.
  - Fix the entry value type to be 'size_t' in RRHT (a ResizeableResourceHashtable). The entry values are oop sizes.

-------------

Changes:
  - all: https://git.openjdk.org/tsan/pull/19/files
  - new: https://git.openjdk.org/tsan/pull/19/files/00bcec7f..8c498475

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=07
 - incr: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=06-07

  Stats: 11 lines in 4 files changed: 0 ins; 0 del; 11 mod
  Patch: https://git.openjdk.org/tsan/pull/19.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/19/head:pull/19

PR: https://git.openjdk.org/tsan/pull/19

From jiangli at openjdk.org  Wed Jul 24 18:29:44 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 18:29:44 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
Message-ID: <fC0KPpBp3v78Q1b6WplRG_ehcaYIsM4KOseuiRAiYUo=.9f223976-3321-44e5-8673-6dd3965f12cb@github.com>

On Wed, 24 Jul 2024 01:46:32 GMT, Man Cao <manc at openjdk.org> wrote:

> Type for size should be jlong, right? As typedef ResizeableResourceHashtable indicates.

The table size is `unsigned`, see https://github.com/openjdk/jdk21u-dev/blob/570a6bd7df8062ac168f39f368303feaab056978/src/hotspot/share/utilities/resourceHash.hpp#L124 and https://github.com/openjdk/jdk21u-dev/blob/570a6bd7df8062ac168f39f368303feaab056978/src/hotspot/share/utilities/resizeableResourceHash.hpp#L38. 

The specifically referred type (as `jlong` in your comment) in `typedef ResizeableResourceHashtable` is the type for entry `value`. That's not the table size type. Thank for flagging it though. It's a bug that I missed during code cleanup iterations. I originally tried to use oop as the entry value, hence using `jlong` was appropriate. I ended up using oop size as the entry value instead, but missed to change the value type accordingly. I've fixed to use `size_t` instead.

> Let's fix this problem also in TsanOopMap::add_oop_with_size. According to https://github.com/microsoft/compiler-rt/blob/master/test/tsan/java.h, the tsan callbacks take unsigned long for size, so jlong should be fine.

Are you referring to the `size` arg passed in `TsanOopMap::add_oop_with_size(oopDesc *addr, int size)`? That's the oop size, and not the table size. 

I've also changed the arg type to `size_t` in add_oop_with_size() functions. Note, the caller `SharedRuntime::tsan_track_obj_with_size` still passes the `size` arg as `int`. We can leave that for now and cleanup later.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690257806

From jiangli at openjdk.org  Wed Jul 24 18:35:27 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 18:35:27 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v9]
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <CTdcP-xjoohKBqCB372u4wJKbIKz-cBSlSGZs1lP00c=.d92bf3b0-b4ac-40db-9334-710930c9ae77@github.com>

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: data race (pid=787116)...

Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:

  - Fix do_entry() to take 'size_t' for the value arg.

-------------

Changes:
  - all: https://git.openjdk.org/tsan/pull/19/files
  - new: https://git.openjdk.org/tsan/pull/19/files/8c498475..74a3d365

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=08
 - incr: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=07-08

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/tsan/pull/19.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/19/head:pull/19

PR: https://git.openjdk.org/tsan/pull/19

From jiangli at openjdk.org  Wed Jul 24 18:39:45 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 18:39:45 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
Message-ID: <uKJgiq71J6oQnC7DYktvpagU8-t4KdcOwf4YXeyMfF0=.f8030cea-2869-472a-a651-9c08bbe01384@github.com>

On Wed, 24 Jul 2024 01:51:14 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Sort moves for overlapping case only.
>
> src/hotspot/share/tsan/tsanOopMapTable.cpp line 134:
> 
>> 132:                                      _dest_low(dest_low), _dest_high(dest_high),
>> 133:                                      _n_downward_moves(n_downward_moves) {}
>> 134:     bool do_entry(TsanOopMapTableKey& entry, uintx size) {
> 
> `uintx` should also be `jlong`, right?

Fixed to use `size_t`.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690268641

From jiangli at openjdk.org  Wed Jul 24 18:53:51 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 18:53:51 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
Message-ID: <H-3Puq11bPmhW9o9AHUSc4FxlK7pqdIeS6Z8A1TL1Jg=.dad5dbba-7740-41bd-9b5d-be157769527c@github.com>

On Wed, 24 Jul 2024 02:02:05 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Sort moves for overlapping case only.
>
> src/hotspot/share/tsan/tsanOopMapTable.cpp line 49:
> 
>> 47: 
>> 48: void TsanOopMapTableKey::update_obj() {
>> 49:   oop obj = _wh.peek();
> 
> Since `_obj` is `oopDesc*` and `oop` could be an object, is it better to do `oopDesc* obj = _wh.peek()`?

`oopDesc*` is `oop`.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690283269

From jiangli at openjdk.org  Wed Jul 24 19:01:54 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 19:01:54 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
Message-ID: <4-tNn-nHiY5MeRP-AY8Irf3UDfRvPOPnBzcHN5O75DA=.720b0469-7d85-4bd2-80ba-82fb611fbdf8@github.com>

On Wed, 24 Jul 2024 02:40:38 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Sort moves for overlapping case only.
>
> src/hotspot/share/tsan/tsanOopMapTable.hpp line 78:
> 
>> 76: 
>> 77:   static bool equals(const TsanOopMapTableKey& lhs, const TsanOopMapTableKey& rhs) {
>> 78:     oop lhs_obj = lhs._obj != nullptr ? (oop)lhs._obj : lhs.object_no_keepalive();
> 
> I think `lhs._obj` and `rhs._obj` will never be `nullptr` in practice. This is because:
> - We never add a null oop to the table.
> - Once `_wh.peek()` returns null, the associated entry is immediately removed. We never update `_obj` to null.
> 
> Would it better to add an assert that `lhs._obj` and `rhs._obj` are both non-null and remove the call to `object_no_keepalive()`?
> 
> (I find it a bit hard to reason why we would resort to `_wh` for equality. I know `JvmtiTagMapKey::_obj` would be null most of the time in contrast.)

I had also considered to not check if `_obj` is nullptr and decided to keep the check. The main concern is that I don't know if there could be a case where it wants to compare an entry with freed object. I feel it's safer to keep the check.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690293796

From jiangli at openjdk.org  Wed Jul 24 19:32:02 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 19:32:02 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v10]
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <5s91W13bHvlbbjevIvVUsJT4-BepkHK1MObL5lexzWY=.d0d5f7ea-ab40-42a7-ab53-cd7d4acfcb60@github.com>

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: data race (pid=787116)...

Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:

  - Added `assert(entry._obj == entry.object_no_keepalive(), "sanity")` in `get_hash`.

-------------

Changes:
  - all: https://git.openjdk.org/tsan/pull/19/files
  - new: https://git.openjdk.org/tsan/pull/19/files/74a3d365..447e44a6

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=09
 - incr: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=08-09

  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/tsan/pull/19.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/19/head:pull/19

PR: https://git.openjdk.org/tsan/pull/19

From jiangli at openjdk.org  Wed Jul 24 19:32:02 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 24 Jul 2024 19:32:02 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
Message-ID: <04Lkyn44_bYOCJ4hegXeVruQEaN-UiiKw6QVdeefIVk=.970983d7-71a0-4dae-8174-1a1d64e9ee46@github.com>

On Wed, 24 Jul 2024 03:01:31 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Sort moves for overlapping case only.
>
> src/hotspot/share/tsan/tsanOopMapTable.hpp line 74:
> 
>> 72:   static unsigned get_hash(const TsanOopMapTableKey& entry) {
>> 73:     assert(entry._obj != nullptr, "sanity");
>> 74:     return (unsigned int)entry._obj->identity_hash();
> 
> In theory `entry._obj` could be an invalid object (old object that is moved or freed), so `identity_hash()` may misbehave. In practice, this probably never happens due to the way we process the hash table in a GC pause. This may deserve a comment, or may be an assert such as `Metaspace::contains(entry._obj->klass())`. But feel free to do nothing.

I added an `assert(entry._obj == entry.object_no_keepalive(), "sanity")` instead.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690324058

From manc at openjdk.org  Wed Jul 24 20:28:46 2024
From: manc at openjdk.org (Man Cao)
Date: Wed, 24 Jul 2024 20:28:46 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <H-3Puq11bPmhW9o9AHUSc4FxlK7pqdIeS6Z8A1TL1Jg=.dad5dbba-7740-41bd-9b5d-be157769527c@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
 <H-3Puq11bPmhW9o9AHUSc4FxlK7pqdIeS6Z8A1TL1Jg=.dad5dbba-7740-41bd-9b5d-be157769527c@github.com>
Message-ID: <0l22KHu_tUYxXXRf7fyF7DbohYN5BqV1rGHFO62E-d0=.b28857d5-89f6-4591-b64e-6eedd62f4a09@github.com>

On Wed, 24 Jul 2024 18:50:34 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

>> src/hotspot/share/tsan/tsanOopMapTable.cpp line 49:
>> 
>>> 47: 
>>> 48: void TsanOopMapTableKey::update_obj() {
>>> 49:   oop obj = _wh.peek();
>> 
>> Since `_obj` is `oopDesc*` and `oop` could be an object, is it better to do `oopDesc* obj = _wh.peek()`?
>
> `oopDesc*` is `oop`.

`oopDesc*` is different from `oop` in a fastdebug build, when `CHECK_UNHANDLED_OOPS` is defined.

>> src/hotspot/share/tsan/tsanOopMapTable.hpp line 78:
>> 
>>> 76: 
>>> 77:   static bool equals(const TsanOopMapTableKey& lhs, const TsanOopMapTableKey& rhs) {
>>> 78:     oop lhs_obj = lhs._obj != nullptr ? (oop)lhs._obj : lhs.object_no_keepalive();
>> 
>> I think `lhs._obj` and `rhs._obj` will never be `nullptr` in practice. This is because:
>> - We never add a null oop to the table.
>> - Once `_wh.peek()` returns null, the associated entry is immediately removed. We never update `_obj` to null.
>> 
>> Would it better to add an assert that `lhs._obj` and `rhs._obj` are both non-null and remove the call to `object_no_keepalive()`?
>> 
>> (I find it a bit hard to reason why we would resort to `_wh` for equality. I know `JvmtiTagMapKey::_obj` would be null most of the time in contrast.)
>
> I had also considered to not check if `_obj` is nullptr and decided to keep the check. The main concern is that I don't know if there could be a case where it wants to compare an entry with freed object. I feel it's safer to keep the check.

Could we try adding an assertion? `assert(lhs._obj != nullptr && rhs._obj != nullptr)`.
Even if we run into an entry whose `_obj` is freed, `_wh.peek()` should also return null in that case. So there's no need to look at `_wh`.

>> src/hotspot/share/tsan/tsanOopMapTable.hpp line 104:
>> 
>>> 102:   unsigned size() const { return _table.table_size(); };
>>> 103: 
>>> 104:   bool add_oop_with_size(oop obj, int size);
>> 
>> Type for `size` should be `jlong`, right? As `typedef ResizeableResourceHashtable` indicates.
>> Let's fix this problem also in `TsanOopMap::add_oop_with_size`.  According to https://github.com/microsoft/compiler-rt/blob/master/test/tsan/java.h, the tsan callbacks take `unsigned long` for size, so `jlong` should be fine.
>> 
>> (We have run into issues with overflowing size field for large objects (>4G), related to JVMTI SampledObjectAlloc. So this is a real bug that could manifest with large objects.)
>
>> Type for size should be jlong, right? As typedef ResizeableResourceHashtable indicates.
> 
> The table size is `unsigned`, see https://github.com/openjdk/jdk21u-dev/blob/570a6bd7df8062ac168f39f368303feaab056978/src/hotspot/share/utilities/resourceHash.hpp#L124 and https://github.com/openjdk/jdk21u-dev/blob/570a6bd7df8062ac168f39f368303feaab056978/src/hotspot/share/utilities/resizeableResourceHash.hpp#L38. 
> 
> The specifically referred type (as `jlong` in your comment) in `typedef ResizeableResourceHashtable` is the type for entry `value`. That's not the table size type. Thank for flagging it though. It's a bug that I missed during code cleanup iterations. I originally tried to use oop as the entry value, hence using `jlong` was appropriate. I ended up using oop size as the entry value instead, but missed to change the value type accordingly. I've fixed to use `size_t` instead.
> 
>> Let's fix this problem also in TsanOopMap::add_oop_with_size. According to https://github.com/microsoft/compiler-rt/blob/master/test/tsan/java.h, the tsan callbacks take unsigned long for size, so jlong should be fine.
> 
> Are you referring to the `size` arg passed in `TsanOopMap::add_oop_with_size(oopDesc *addr, int size)`? That's the oop size, and not the table size. 
> 
> I've also changed the arg type to `size_t` in add_oop_with_size() functions. Note, the caller `SharedRuntime::tsan_track_obj_with_size` still passes the `size` arg as `int`. We can leave that for now and cleanup later.

I'm referring to the argument `int size` passed to `TsanOopMapTable::add_oop_with_size` and `TsanOopMap::add_oop_with_size`. Yes, `size_t` also works.

Thanks for noticing `SharedRuntime::tsan_track_obj_with_size`, we can fix it separately.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690368455
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690377305
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690363507

From manc at openjdk.org  Wed Jul 24 20:28:47 2024
From: manc at openjdk.org (Man Cao)
Date: Wed, 24 Jul 2024 20:28:47 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
Message-ID: <Xit5k89Tm8WmDAB-sYahmy8x_Lz0sW-GmV76JkuS95Y=.ebc4edec-e55b-47a9-9116-16e9151c7f3a@github.com>

On Wed, 24 Jul 2024 02:11:22 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Sort moves for overlapping case only.
>
> src/hotspot/share/tsan/tsanOopMapTable.hpp line 67:
> 
>> 65: 
>> 66:   void release_weak_handle() const;
>> 67:   oop object_no_keepalive() const;
> 
> I'm a bit unsure about the use of `oop` type vs `oopDesc*` type. Could we keep using `oopDesc*` type in `object_no_keepalive()`, `obj()`, `equals()` and `update_obj()`? E.g., whenever calling `_wh.peek()`, immediately cast the return value to a `oopDesc*`.
> 
> They are the same type in a release build, but different in a debug build. I vaguely recall that TsanOopSizeMap used `oop` in the JDK7/8 era, but switched to `oopDesc*` due to errors in debug build.

Alternatively, it might be OK to declare TsanOopMapTableKey::_obj as an `oop` instead of `oopDesc*`. The main point is to avoid type conversions in various places.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690382737

From manc at openjdk.org  Thu Jul 25 01:28:47 2024
From: manc at openjdk.org (Man Cao)
Date: Thu, 25 Jul 2024 01:28:47 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <0l22KHu_tUYxXXRf7fyF7DbohYN5BqV1rGHFO62E-d0=.b28857d5-89f6-4591-b64e-6eedd62f4a09@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
 <H-3Puq11bPmhW9o9AHUSc4FxlK7pqdIeS6Z8A1TL1Jg=.dad5dbba-7740-41bd-9b5d-be157769527c@github.com>
 <0l22KHu_tUYxXXRf7fyF7DbohYN5BqV1rGHFO62E-d0=.b28857d5-89f6-4591-b64e-6eedd62f4a09@github.com>
Message-ID: <7bABz4VuYxjvo9MoEJ8AnficejwFUJ6zpJf1WCamjwY=.5fe1ee62-737b-4144-ad42-4e3451453458@github.com>

On Wed, 24 Jul 2024 20:10:39 GMT, Man Cao <manc at openjdk.org> wrote:

>> `oopDesc*` is `oop`.
>
> `oopDesc*` is different from `oop` in a fastdebug build, when `CHECK_UNHANDLED_OOPS` is defined.

See the `oop` class definition [here](https://github.com/openjdk/jdk/blob/0898ab7f7496689e5de52a5dc4530ca21def1fca/src/hotspot/share/oops/oopsHierarchy.hpp#L53-L115). Note that `CHECK_UNHANDLED_OOPS` is only enabled in [fastdebug](https://github.com/openjdk/jdk/blob/0898ab7f7496689e5de52a5dc4530ca21def1fca/make/hotspot/lib/JvmFlags.gmk#L77-L81), not slowdebug.
As said in the other comment,  we could declare `TsanOopMapTableKey::_obj` as an `oop`, but make sure to test with fastdebug build.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1690669797

From jiangli at openjdk.org  Fri Jul 26 23:11:54 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Fri, 26 Jul 2024 23:11:54 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <7bABz4VuYxjvo9MoEJ8AnficejwFUJ6zpJf1WCamjwY=.5fe1ee62-737b-4144-ad42-4e3451453458@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
 <H-3Puq11bPmhW9o9AHUSc4FxlK7pqdIeS6Z8A1TL1Jg=.dad5dbba-7740-41bd-9b5d-be157769527c@github.com>
 <0l22KHu_tUYxXXRf7fyF7DbohYN5BqV1rGHFO62E-d0=.b28857d5-89f6-4591-b64e-6eedd62f4a09@github.com>
 <7bABz4VuYxjvo9MoEJ8AnficejwFUJ6zpJf1WCamjwY=.5fe1ee62-737b-4144-ad42-4e3451453458@github.com>
Message-ID: <56rh9BWfpK6pEjj1cyHTSR9S_iUkdw9wEjmXuUD8OcM=.d8e23357-ff7d-48f0-a49e-e5a93ba208a3@github.com>

On Thu, 25 Jul 2024 01:26:14 GMT, Man Cao <manc at openjdk.org> wrote:

>> `oopDesc*` is different from `oop` in a fastdebug build, when `CHECK_UNHANDLED_OOPS` is defined.
>
> See the `oop` class definition [here](https://github.com/openjdk/jdk/blob/0898ab7f7496689e5de52a5dc4530ca21def1fca/src/hotspot/share/oops/oopsHierarchy.hpp#L53-L115). Note that `CHECK_UNHANDLED_OOPS` is only enabled in [fastdebug](https://github.com/openjdk/jdk/blob/0898ab7f7496689e5de52a5dc4530ca21def1fca/make/hotspot/lib/JvmFlags.gmk#L77-L81), not slowdebug.
> As said in the other comment,  we could declare `TsanOopMapTableKey::_obj` as an `oop`, but make sure to test with fastdebug build.

I just reread your earlier suggestion more closely. 

> is it better to do oopDesc* obj = _wh.peek()?

WeakHandle only defines `peek()` with `oop` as return type (see `inline oop peek() const;`). With `CHECK_UNHANDLED_OOPS`, the implicit cast from `oop` to `oopDesc*` is handled by the `oopDesc* ()` operator. I think we should keep `oop obj = _wh.peek();` for consistency with the return type declared in `peek`. 

For the issue with `obj != _obj`, changing `obj != cast_to_oop(_obj)` resolves it.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1693681404

From manc at openjdk.org  Fri Jul 26 23:34:47 2024
From: manc at openjdk.org (Man Cao)
Date: Fri, 26 Jul 2024 23:34:47 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <56rh9BWfpK6pEjj1cyHTSR9S_iUkdw9wEjmXuUD8OcM=.d8e23357-ff7d-48f0-a49e-e5a93ba208a3@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
 <H-3Puq11bPmhW9o9AHUSc4FxlK7pqdIeS6Z8A1TL1Jg=.dad5dbba-7740-41bd-9b5d-be157769527c@github.com>
 <0l22KHu_tUYxXXRf7fyF7DbohYN5BqV1rGHFO62E-d0=.b28857d5-89f6-4591-b64e-6eedd62f4a09@github.com>
 <7bABz4VuYxjvo9MoEJ8AnficejwFUJ6zpJf1WCamjwY=.5fe1ee62-737b-4144-ad42-4e3451453458@github.com>
 <56rh9BWfpK6pEjj1cyHTSR9S_iUkdw9wEjmXuUD8OcM=.d8e23357-ff7d-48f0-a49e-e5a93ba208a3@github.com>
Message-ID: <GWK5gwzaxnr6SscAL3K5IgWAqoki8TgfR1Ph3oKVJ78=.52399d40-8b1b-4259-a43b-e0df014c8fdf@github.com>

On Fri, 26 Jul 2024 23:08:52 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

>> See the `oop` class definition [here](https://github.com/openjdk/jdk/blob/0898ab7f7496689e5de52a5dc4530ca21def1fca/src/hotspot/share/oops/oopsHierarchy.hpp#L53-L115). Note that `CHECK_UNHANDLED_OOPS` is only enabled in [fastdebug](https://github.com/openjdk/jdk/blob/0898ab7f7496689e5de52a5dc4530ca21def1fca/make/hotspot/lib/JvmFlags.gmk#L77-L81), not slowdebug.
>> As said in the other comment,  we could declare `TsanOopMapTableKey::_obj` as an `oop`, but make sure to test with fastdebug build.
>
> I just reread your earlier suggestion more closely. 
> 
>> is it better to do oopDesc* obj = _wh.peek()?
> 
> WeakHandle only defines `peek()` with `oop` as return type (see `inline oop peek() const;`). With `CHECK_UNHANDLED_OOPS`, the implicit cast from `oop` to `oopDesc*` is handled by the `oopDesc* ()` operator. I think we should keep `oop obj = _wh.peek();` for consistency with the return type declared in `peek`. 
> 
> For the issue with `obj != _obj`, changing `obj != cast_to_oop(_obj)` resolves it.

Yes, I'm aware `oopDesc* obj = _wh.peek()` will do the cast with the `oopDesc* ()` operator under `CHECK_UNHANDLED_OOPS`.

The point is to avoid unnecessary type casting and conversion if possible.  `oopDesc* obj = _wh.peek()` will just do the cast once, and avoid later casting with `obj != _obj` and `_obj = obj`.

Alternative is to declare `TsanOopMapTableKey::_obj` as an `oop`, which is probably cleaner.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1693689617

From jiangli at openjdk.org  Fri Jul 26 23:52:44 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Fri, 26 Jul 2024 23:52:44 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <Xit5k89Tm8WmDAB-sYahmy8x_Lz0sW-GmV76JkuS95Y=.ebc4edec-e55b-47a9-9116-16e9151c7f3a@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
 <Xit5k89Tm8WmDAB-sYahmy8x_Lz0sW-GmV76JkuS95Y=.ebc4edec-e55b-47a9-9116-16e9151c7f3a@github.com>
Message-ID: <i3JKkey9T794VCJLbuDtiDybvQcVkYFRNkOtHUOvdxs=.4c7d7c31-7fda-4280-89c5-1bea302e5a16@github.com>

On Wed, 24 Jul 2024 20:25:23 GMT, Man Cao <manc at openjdk.org> wrote:

>> src/hotspot/share/tsan/tsanOopMapTable.hpp line 67:
>> 
>>> 65: 
>>> 66:   void release_weak_handle() const;
>>> 67:   oop object_no_keepalive() const;
>> 
>> I'm a bit unsure about the use of `oop` type vs `oopDesc*` type. Could we keep using `oopDesc*` type in `object_no_keepalive()`, `obj()`, `equals()` and `update_obj()`? E.g., whenever calling `_wh.peek()`, immediately cast the return value to a `oopDesc*`.
>> 
>> They are the same type in a release build, but different in a debug build. I vaguely recall that TsanOopSizeMap used `oop` in the JDK7/8 era, but switched to `oopDesc*` due to errors in debug build.
>
> Alternatively, it might be OK to declare TsanOopMapTableKey::_obj as an `oop` instead of `oopDesc*`. The main point is to avoid type conversions in various places.

Using `oop` for `TsanOopMapTableKey::_obj` is ok.

I also just built and tested with `fastdebug` JDK binary. I found a few places need `cast_from_oop<type>` conversions, e.g. in `InstanceKlass::initialize` when calling `SharedRuntime::tsan_acquire`. I've fixed those. I'll push my changes shortly. 

The tsan jtreg tests pass with `fastdebug` binary, except `RawRacyNativeLoopTest.java` and `NonRacyCasLoopTest.java`. That's the same with `slowdebug` binary. Those two tests fail with following assert. The failures should be unrelated to the tsan oop map. There is a separate internal bug that I created earlier to this specific issue.


#  Internal Error (/usr/local/google/home/jianglizhou/github/tsan-oopMap-weakhandle/src/hotspot/share/oops/accessBackend.cpp:224), pid=1986120, tid=1986364
#  assert(state == _thread_in_vm || state == _thread_in_Java || state == _thread_new) failed: Wrong thread state for accesses: 4

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1693696088

From jiangli at openjdk.org  Sat Jul 27 00:28:10 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Sat, 27 Jul 2024 00:28:10 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v11]
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <gveEN7jFcyGfVjBHkKTXCCFpdKbc2hZT05P6prjN7iM=.5a87f759-9b65-43c0-9ed1-f60b94223e78@github.com>

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles in concurrent WeakProcessor worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: data race (pid=787116)...

Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:

  - Use cast_from_oop<T> in various places when coverting oop to different types.
  - Change TsanOopMapTableKey::_obj to 'oop'.
  - Change TsanOopMapTableKey::equals to just do `lhs._obj == rhs._obj`.

-------------

Changes:
  - all: https://git.openjdk.org/tsan/pull/19/files
  - new: https://git.openjdk.org/tsan/pull/19/files/447e44a6..b8bcafe3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=10
 - incr: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=09-10

  Stats: 12 lines in 3 files changed: 0 ins; 2 del; 10 mod
  Patch: https://git.openjdk.org/tsan/pull/19.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/19/head:pull/19

PR: https://git.openjdk.org/tsan/pull/19

From jiangli at openjdk.org  Sat Jul 27 00:39:42 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Sat, 27 Jul 2024 00:39:42 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <0l22KHu_tUYxXXRf7fyF7DbohYN5BqV1rGHFO62E-d0=.b28857d5-89f6-4591-b64e-6eedd62f4a09@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
 <H-3Puq11bPmhW9o9AHUSc4FxlK7pqdIeS6Z8A1TL1Jg=.dad5dbba-7740-41bd-9b5d-be157769527c@github.com>
 <0l22KHu_tUYxXXRf7fyF7DbohYN5BqV1rGHFO62E-d0=.b28857d5-89f6-4591-b64e-6eedd62f4a09@github.com>
Message-ID: <t-UPNu9IClUf3O9oN9x__I5uon035P2Fe8sG76zcuwk=.0ee042eb-e99d-4cff-9ef2-453118824f4f@github.com>

On Wed, 24 Jul 2024 20:19:55 GMT, Man Cao <manc at openjdk.org> wrote:

> Even if we run into an entry whose _obj is freed, _wh.peek() should also return null in that case. So there's no need to look at _wh

That's true. I changed to just do `lhs._obj == rhs._obj`. I think it's safer to not add `lhs._obj != nullptr` assert.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1693708848

From jiangli at openjdk.org  Sat Jul 27 00:49:39 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Sat, 27 Jul 2024 00:49:39 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v6]
In-Reply-To: <GWK5gwzaxnr6SscAL3K5IgWAqoki8TgfR1Ph3oKVJ78=.52399d40-8b1b-4259-a43b-e0df014c8fdf@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <AdXG7ZFNDrhGuINgOB5xHyyUkS6I3ORqiDtHS2Ay5Xc=.b73bc00f-03eb-4c00-b646-36f2082b7911@github.com>
 <mOStMb9zpfqBe_TOJSKVgWlKOKdXaX2CJP1gBFg847A=.206f765e-2331-46fb-b0be-fb89c795dd94@github.com>
 <H-3Puq11bPmhW9o9AHUSc4FxlK7pqdIeS6Z8A1TL1Jg=.dad5dbba-7740-41bd-9b5d-be157769527c@github.com>
 <0l22KHu_tUYxXXRf7fyF7DbohYN5BqV1rGHFO62E-d0=.b28857d5-89f6-4591-b64e-6eedd62f4a09@github.com>
 <7bABz4VuYxjvo9MoEJ8AnficejwFUJ6zpJf1WCamjwY=.5fe1ee62-737b-4144-ad42-4e3451453458@github.com>
 <56rh9BWfpK6pEjj1cyHTSR9S_iUkdw9wEjmXuUD8OcM=.d8e23357-ff7d-48f0-a49e-e5a93ba208a3@github.com>
 <GWK5gwzaxnr6SscAL3K5IgWAqoki8TgfR1Ph3oKVJ78=.52399d40-8b1b-4259-a43b-e0df014c8fdf@github.com>
Message-ID: <XBdVPil1Z6Sbw15GGuy6_4z8KTHuhqTOCOvIAvT0Yyg=.ee42885b-60cc-40aa-90c2-50608829f7b4@github.com>

On Fri, 26 Jul 2024 23:31:45 GMT, Man Cao <manc at openjdk.org> wrote:

>> I just reread your earlier suggestion more closely. 
>> 
>>> is it better to do oopDesc* obj = _wh.peek()?
>> 
>> WeakHandle only defines `peek()` with `oop` as return type (see `inline oop peek() const;`). With `CHECK_UNHANDLED_OOPS`, the implicit cast from `oop` to `oopDesc*` is handled by the `oopDesc* ()` operator. I think we should keep `oop obj = _wh.peek();` for consistency with the return type declared in `peek`. 
>> 
>> For the issue with `obj != _obj`, changing `obj != cast_to_oop(_obj)` resolves it.
>
> Yes, I'm aware `oopDesc* obj = _wh.peek()` will do the cast with the `oopDesc* ()` operator under `CHECK_UNHANDLED_OOPS`.
> 
> The point is to avoid unnecessary type casting and conversion if possible.  `oopDesc* obj = _wh.peek()` will just do the cast once, and avoid later casting with `obj != _obj` and `_obj = obj`.
> 
> Alternative is to declare `TsanOopMapTableKey::_obj` as an `oop`, which is probably cleaner.

> For the issue with obj != _obj, changing obj != cast_to_oop(_obj) resolves it.

`cast_to_oop(_obj)` is not needed. This is addressed by changing `_obj` to `oop`.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1693711455

From manc at openjdk.org  Sat Jul 27 03:34:43 2024
From: manc at openjdk.org (Man Cao)
Date: Sat, 27 Jul 2024 03:34:43 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v11]
In-Reply-To: <gveEN7jFcyGfVjBHkKTXCCFpdKbc2hZT05P6prjN7iM=.5a87f759-9b65-43c0-9ed1-f60b94223e78@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <gveEN7jFcyGfVjBHkKTXCCFpdKbc2hZT05P6prjN7iM=.5a87f759-9b65-43c0-9ed1-f60b94223e78@github.com>
Message-ID: <pa1RLoLXWBdvTIKiM16u7A9DufP8GauQgnl7OSAYr0I=.139efe20-90b6-4e94-b643-fe93c5f0f762@github.com>

On Sat, 27 Jul 2024 00:28:10 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

>> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
>> 
>> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
>> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
>> 
>> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
>> 
>>  - A Weakhandle that holds a pointer to the oop;
>>  - The original (or updated) address of the object in Java heap;
>> 
>> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles with `WeakProcessor::Task::work` in concurrent worker threads.
>> 
>> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
>> 
>> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
>> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
>> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
>> 
>> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
>> 
>> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
>> 
>> Suppressed following Tsan errors in JDK:
>> 1)  In `java.util.concu...
>
> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
> 
>   - Use cast_from_oop<T> in various places when coverting oop to different types.
>   - Change TsanOopMapTableKey::_obj to 'oop'.
>   - Change TsanOopMapTableKey::equals to just do `lhs._obj == rhs._obj`.

Thanks for bearing with me, and addressing issues and testing diligently!

Here are my last set of comments. The one about `_n_downward_moves` is important, and the others are pretty trivial.

src/hotspot/share/tsan/tsanOopMap.cpp line 33:

> 31: #include "memory/resourceArea.hpp"
> 32: #include "oops/oop.inline.hpp"
> 33: #include "runtime/atomic.hpp"

We can probably remove `#include` for `atomic.hpp` and `resizeableResourceHash.hpp`.

src/hotspot/share/tsan/tsanOopMapTable.cpp line 80:

> 78:   } else {
> 79:     size_t* v = _table.put_if_absent(new_entry, size, &added);
> 80:     *v = size;

This can be an assert: `assert(*v == size, "same oop should have same size")`.
This corresponds to `assert(s == bucket->get_oop_size())` in `TsanOopSizeMap::put()`.

src/hotspot/share/tsan/tsanOopMapTable.cpp line 85:

> 83:   if (added) {
> 84:     if (_table.maybe_grow(true /* use_large_table_sizes */)) {
> 85:       log_info(tsan)("TsanOopMapTable resize to %d, %d entries",

Could it be `log_debug(tsan)` as well?

src/hotspot/share/tsan/tsanOopMapTable.cpp line 150:

> 148:         *_dest_high = MAX2(*_dest_high, move.target_end());
> 149:         if (*_dest_low < *_src_low) {
> 150:           ++(*_n_downward_moves);

This seems incorrect. Increment to `_n_downward_moves` should be based on the current move, but `*_dest_low` and `*_src_low` are the minimum of all moves so far. It should be:
`if (move.target_begin() < move.source_begin())`

An inaccurate value of  `_n_downward_moves` only affects the efficiency of `handle_overlapping_moves()`, but not correctness.

src/hotspot/share/tsan/tsanOopMapTable.cpp line 153:

> 151:         }
> 152: 
> 153:         entry.update_obj();

(Just a comment, no action required.)
It is a bit unusual to mutate the key object of a hash table during iteration, as it could mess up key's hash value. Then subsequent `add`, `contains` operations may not find this existing key. Old `TsanOopSizeMap` implementation avoids this problem by creating a new hash table and add all remaining entries there.

However, it is safe here because `TsanOopMapTable` uses `identity_hash()`, so the hash value won't change. This also means enabling TSAN will invoke `identity_hash()` on all Java objects, which may have other side effects, but is probably OK.

-------------

Changes requested by manc (Reviewer).

PR Review: https://git.openjdk.org/tsan/pull/19#pullrequestreview-2202918789
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1693728785
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1693747331
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1693749070
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1693733272
PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1693734105

From manc at openjdk.org  Sat Jul 27 03:34:44 2024
From: manc at openjdk.org (Man Cao)
Date: Sat, 27 Jul 2024 03:34:44 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v11]
In-Reply-To: <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <1hhmw3rft1tr6m7TFwinvmaPKqhUAIfulGfDdg0OVXM=.f24d859f-9adf-4477-8d02-13cf77d6893d@github.com>
Message-ID: <6j6vy-HoP_7SUTUWnQ3fdoWrKmUczwApbaZ53Qngqnw=.c3144263-ffba-408d-8031-97004176ef85@github.com>

On Tue, 23 Jul 2024 01:35:47 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   - Use cast_from_oop<T> in various places when coverting oop to different types.
>>   - Change TsanOopMapTableKey::_obj to 'oop'.
>>   - Change TsanOopMapTableKey::equals to just do `lhs._obj == rhs._obj`.
>
> src/hotspot/share/tsan/tsanOopMap.cpp line 450:
> 
>> 448:     }
>> 449: 
>> 450:     // Source and target ranges overlap, the moves need to be ordered to prevent
> 
> Can we retain this comment paragraph?

Could we also keep the remaining part of this comment?
"overwhelming majority of the objects move in one direction ... chances are we will be able to order the moves in a few traversals of the moves array"

This is still relevant, i.e. in practice the code does not take O(N^2).

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1693724924

From jiangli at openjdk.org  Mon Jul 29 18:54:46 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Mon, 29 Jul 2024 18:54:46 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v11]
In-Reply-To: <pa1RLoLXWBdvTIKiM16u7A9DufP8GauQgnl7OSAYr0I=.139efe20-90b6-4e94-b643-fe93c5f0f762@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <gveEN7jFcyGfVjBHkKTXCCFpdKbc2hZT05P6prjN7iM=.5a87f759-9b65-43c0-9ed1-f60b94223e78@github.com>
 <pa1RLoLXWBdvTIKiM16u7A9DufP8GauQgnl7OSAYr0I=.139efe20-90b6-4e94-b643-fe93c5f0f762@github.com>
Message-ID: <BP8S7vxSqe_fjsamyvZ-y5xLaQ4Mtc29HY2reIO_w1k=.13b36012-81fa-4a5f-a653-f1ed7e7a7e27@github.com>

On Sat, 27 Jul 2024 02:52:36 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   - Use cast_from_oop<T> in various places when coverting oop to different types.
>>   - Change TsanOopMapTableKey::_obj to 'oop'.
>>   - Change TsanOopMapTableKey::equals to just do `lhs._obj == rhs._obj`.
>
> src/hotspot/share/tsan/tsanOopMapTable.cpp line 153:
> 
>> 151:         }
>> 152: 
>> 153:         entry.update_obj();
> 
> (Just a comment, no action required.)
> It is a bit unusual to mutate the key object of a hash table during iteration, as it could mess up key's hash value. Then subsequent `add`, `contains` operations may not find this existing key. Old `TsanOopSizeMap` implementation avoids this problem by creating a new hash table and add all remaining entries there.
> 
> However, it is safe here because `TsanOopMapTable` uses `identity_hash()`, so the hash value won't change. This also means enabling TSAN will invoke `identity_hash()` on all Java objects, which may have other side effects, but is probably OK.

It's safe because the entry still points to the same object, so it would not affect the key's hash value. This occurs during a STW GC pause, and access to the table from other concurrent worker threads is protected from the TsanOopMap_lock.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1695720619

From jiangli at openjdk.org  Mon Jul 29 19:18:45 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Mon, 29 Jul 2024 19:18:45 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v11]
In-Reply-To: <pa1RLoLXWBdvTIKiM16u7A9DufP8GauQgnl7OSAYr0I=.139efe20-90b6-4e94-b643-fe93c5f0f762@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <gveEN7jFcyGfVjBHkKTXCCFpdKbc2hZT05P6prjN7iM=.5a87f759-9b65-43c0-9ed1-f60b94223e78@github.com>
 <pa1RLoLXWBdvTIKiM16u7A9DufP8GauQgnl7OSAYr0I=.139efe20-90b6-4e94-b643-fe93c5f0f762@github.com>
Message-ID: <YLt_XvmFP60KnNpuGpiFzMfmdqc3JJG_xCusdxu5N_M=.65b55e7d-6b4d-445d-bc97-8790b55dafb3@github.com>

On Sat, 27 Jul 2024 03:24:05 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   - Use cast_from_oop<T> in various places when coverting oop to different types.
>>   - Change TsanOopMapTableKey::_obj to 'oop'.
>>   - Change TsanOopMapTableKey::equals to just do `lhs._obj == rhs._obj`.
>
> src/hotspot/share/tsan/tsanOopMapTable.cpp line 80:
> 
>> 78:   } else {
>> 79:     size_t* v = _table.put_if_absent(new_entry, size, &added);
>> 80:     *v = size;
> 
> This can be an assert: `assert(*v == size, "same oop should have same size")`.
> This corresponds to `assert(s == bucket->get_oop_size())` in `TsanOopSizeMap::put()`.

I followed the example of `JvmtiTagMapTable::add` when initially implemented this. In the initial version, I made oop as the entry value and later changed to oop use size when finalizing the implementation. It's true that the oop size should never change until an entry is removed. So `*v = size` is not needed. Changing to an assert sounds good to me.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1695745413

From jiangli at openjdk.org  Mon Jul 29 19:31:46 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Mon, 29 Jul 2024 19:31:46 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v11]
In-Reply-To: <pa1RLoLXWBdvTIKiM16u7A9DufP8GauQgnl7OSAYr0I=.139efe20-90b6-4e94-b643-fe93c5f0f762@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <gveEN7jFcyGfVjBHkKTXCCFpdKbc2hZT05P6prjN7iM=.5a87f759-9b65-43c0-9ed1-f60b94223e78@github.com>
 <pa1RLoLXWBdvTIKiM16u7A9DufP8GauQgnl7OSAYr0I=.139efe20-90b6-4e94-b643-fe93c5f0f762@github.com>
Message-ID: <bXO41mpu8FtPb_kyFGIIKocMYvWF7Nv-uI70PCLjKWI=.cca7b79b-c5c3-4002-a72d-962c3f01c10e@github.com>

On Sat, 27 Jul 2024 03:27:15 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   - Use cast_from_oop<T> in various places when coverting oop to different types.
>>   - Change TsanOopMapTableKey::_obj to 'oop'.
>>   - Change TsanOopMapTableKey::equals to just do `lhs._obj == rhs._obj`.
>
> src/hotspot/share/tsan/tsanOopMapTable.cpp line 85:
> 
>> 83:   if (added) {
>> 84:     if (_table.maybe_grow(true /* use_large_table_sizes */)) {
>> 85:       log_info(tsan)("TsanOopMapTable resize to %d, %d entries",
> 
> Could it be `log_debug(tsan)` as well?

Let's keep `log_info`. Resizing occurs not frequently, there's no concern with too much resizing log output. The resizing info maybe useful for finding more adequate table size.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1695758029

From jiangli at openjdk.org  Mon Jul 29 19:50:46 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Mon, 29 Jul 2024 19:50:46 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v11]
In-Reply-To: <pa1RLoLXWBdvTIKiM16u7A9DufP8GauQgnl7OSAYr0I=.139efe20-90b6-4e94-b643-fe93c5f0f762@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <gveEN7jFcyGfVjBHkKTXCCFpdKbc2hZT05P6prjN7iM=.5a87f759-9b65-43c0-9ed1-f60b94223e78@github.com>
 <pa1RLoLXWBdvTIKiM16u7A9DufP8GauQgnl7OSAYr0I=.139efe20-90b6-4e94-b643-fe93c5f0f762@github.com>
Message-ID: <ko-V_KinEzNoGPInwAN9n4SzIg53YqmQw827beuptaQ=.c29e427f-d4fe-4f06-a82d-cc8e9b415ae4@github.com>

On Sat, 27 Jul 2024 02:48:00 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   - Use cast_from_oop<T> in various places when coverting oop to different types.
>>   - Change TsanOopMapTableKey::_obj to 'oop'.
>>   - Change TsanOopMapTableKey::equals to just do `lhs._obj == rhs._obj`.
>
> src/hotspot/share/tsan/tsanOopMapTable.cpp line 150:
> 
>> 148:         *_dest_high = MAX2(*_dest_high, move.target_end());
>> 149:         if (*_dest_low < *_src_low) {
>> 150:           ++(*_n_downward_moves);
> 
> This seems incorrect. Increment to `_n_downward_moves` should be based on the current move, but `*_dest_low` and `*_src_low` are the minimum of all moves so far. It should be:
> `if (move.target_begin() < move.source_begin())`
> 
> An inaccurate value of  `_n_downward_moves` only affects the efficiency of `handle_overlapping_moves()`, but not correctness.

Good catch for the `_n_downward_moves`. It only affects which compare function (`TsanOopMapImpl::lessThan` or `TsanOopMapImpl::moreThan`) being used with `moves.sort`. It's still good to fix it.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1695777018

From jiangli at openjdk.org  Mon Jul 29 20:06:22 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Mon, 29 Jul 2024 20:06:22 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v11]
In-Reply-To: <pa1RLoLXWBdvTIKiM16u7A9DufP8GauQgnl7OSAYr0I=.139efe20-90b6-4e94-b643-fe93c5f0f762@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <gveEN7jFcyGfVjBHkKTXCCFpdKbc2hZT05P6prjN7iM=.5a87f759-9b65-43c0-9ed1-f60b94223e78@github.com>
 <pa1RLoLXWBdvTIKiM16u7A9DufP8GauQgnl7OSAYr0I=.139efe20-90b6-4e94-b643-fe93c5f0f762@github.com>
Message-ID: <slZgYtHU0expHs_F172nQdS4Zyp2uHuN7RZCY3zf3b4=.f8db3f2a-b583-43b1-a9b7-55ccaa48044c@github.com>

On Sat, 27 Jul 2024 02:21:50 GMT, Man Cao <manc at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   - Use cast_from_oop<T> in various places when coverting oop to different types.
>>   - Change TsanOopMapTableKey::_obj to 'oop'.
>>   - Change TsanOopMapTableKey::equals to just do `lhs._obj == rhs._obj`.
>
> src/hotspot/share/tsan/tsanOopMap.cpp line 33:
> 
>> 31: #include "memory/resourceArea.hpp"
>> 32: #include "oops/oop.inline.hpp"
>> 33: #include "runtime/atomic.hpp"
> 
> We can probably remove `#include` for `atomic.hpp` and `resizeableResourceHash.hpp`.

Removed.

-------------

PR Review Comment: https://git.openjdk.org/tsan/pull/19#discussion_r1695790020

From jiangli at openjdk.org  Mon Jul 29 20:06:22 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Mon, 29 Jul 2024 20:06:22 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v11]
In-Reply-To: <gveEN7jFcyGfVjBHkKTXCCFpdKbc2hZT05P6prjN7iM=.5a87f759-9b65-43c0-9ed1-f60b94223e78@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <gveEN7jFcyGfVjBHkKTXCCFpdKbc2hZT05P6prjN7iM=.5a87f759-9b65-43c0-9ed1-f60b94223e78@github.com>
Message-ID: <9zzfyL0jrnDA3m2f9HTvydD2WLPUG6kWl9H536TY3Pc=.54d950af-e2d5-4fdf-8ce5-2030978a8975@github.com>

On Sat, 27 Jul 2024 00:28:10 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

>> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
>> 
>> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
>> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
>> 
>> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
>> 
>>  - A Weakhandle that holds a pointer to the oop;
>>  - The original (or updated) address of the object in Java heap;
>> 
>> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles with `WeakProcessor::Task::work` in concurrent worker threads.
>> 
>> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
>> 
>> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
>> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
>> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
>> 
>> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
>> 
>> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
>> 
>> Suppressed following Tsan errors in JDK:
>> 1)  In `java.util.concu...
>
> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
> 
>   - Use cast_from_oop<T> in various places when coverting oop to different types.
>   - Change TsanOopMapTableKey::_obj to 'oop'.
>   - Change TsanOopMapTableKey::equals to just do `lhs._obj == rhs._obj`.

> 

Thanks for the careful and thorough review! It helps make the code cleaner.

-------------

PR Comment: https://git.openjdk.org/tsan/pull/19#issuecomment-2256795187

From jiangli at openjdk.org  Mon Jul 29 20:06:22 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Mon, 29 Jul 2024 20:06:22 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v12]
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <SlbGcddRsTP0WbjbCr3O_jiEg0IKwkD_JOSZbRqu1sI=.687b692f-0612-41ff-ba4d-e9d120377e4b@github.com>

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles with `WeakProcessor::Task::work` in concurrent worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: dat...

Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:

  Address additional review comments from @manc:
  - Remove unneeded #include headers from tsanOopMap.cpp.
  - Change `*v = size` to `*v == size` assert in TsanOopMapTable::add_oop_with_size.
  - Fix incorrect check when updating `_n_downward_moves`.

-------------

Changes:
  - all: https://git.openjdk.org/tsan/pull/19/files
  - new: https://git.openjdk.org/tsan/pull/19/files/b8bcafe3..1261c90b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=11
 - incr: https://webrevs.openjdk.org/?repo=tsan&pr=19&range=10-11

  Stats: 4 lines in 2 files changed: 0 ins; 2 del; 2 mod
  Patch: https://git.openjdk.org/tsan/pull/19.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/19/head:pull/19

PR: https://git.openjdk.org/tsan/pull/19

From manc at openjdk.org  Mon Jul 29 20:31:45 2024
From: manc at openjdk.org (Man Cao)
Date: Mon, 29 Jul 2024 20:31:45 GMT
Subject: RFR: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable [v12]
In-Reply-To: <SlbGcddRsTP0WbjbCr3O_jiEg0IKwkD_JOSZbRqu1sI=.687b692f-0612-41ff-ba4d-e9d120377e4b@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
 <SlbGcddRsTP0WbjbCr3O_jiEg0IKwkD_JOSZbRqu1sI=.687b692f-0612-41ff-ba4d-e9d120377e4b@github.com>
Message-ID: <3wttHzBffQ8xV6Z69K9hJ-uzSi8Nzj0xDwfVY0fk5UI=.f07cb86d-e260-4235-a76b-5bff6b5a14d8@github.com>

On Mon, 29 Jul 2024 20:06:22 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

>> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
>> 
>> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
>> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
>> 
>> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
>> 
>>  - A Weakhandle that holds a pointer to the oop;
>>  - The original (or updated) address of the object in Java heap;
>> 
>> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles with `WeakProcessor::Task::work` in concurrent worker threads.
>> 
>> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
>> 
>> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
>> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
>> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
>> 
>> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
>> 
>> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
>> 
>> Suppressed following Tsan errors in JDK:
>> 1)  In `java.util.concu...
>
> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Address additional review comments from @manc:
>   - Remove unneeded #include headers from tsanOopMap.cpp.
>   - Change `*v = size` to `*v == size` assert in TsanOopMapTable::add_oop_with_size.
>   - Fix incorrect check when updating `_n_downward_moves`.

Great, LGTM!

-------------

Marked as reviewed by manc (Reviewer).

PR Review: https://git.openjdk.org/tsan/pull/19#pullrequestreview-2205933088

From jiangli at openjdk.org  Mon Jul 29 23:36:45 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Mon, 29 Jul 2024 23:36:45 GMT
Subject: Integrated: Reimplement TsanOopMap support using WeakHandle and
 ResizeableResourceHashtable
In-Reply-To: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
References: <EFqy0CjPA0fPIcS7wRE8AfeoGD_7Zz0Vfk-lnfwjpyU=.bc90ccce-0a9d-4cee-b1d1-370482943985@github.com>
Message-ID: <mXSAQvnDwV3NfOiPxVaPK609Oqnacius4-5IgXUY0jY=.11206db5-1c6c-4d10-b12a-61b887889f43@github.com>

On Tue, 16 Jul 2024 00:37:09 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

> The new implementation for TsanOopMap support in JDK 21 is partially modeled after [JvmtiTagMap](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMap.hpp#L37) and [JvmtiTagMapTable](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/prims/jvmtiTagMapTable.hpp#L43):
> 
> - Use [WeakHandle](https://github.com/openjdk/jdk21u-dev/blob/cdbd94f8fa97847c02e1f0f4c6cd75f457a62e25/src/hotspot/share/oops/weakHandle.hpp#L42) for handling oops in the table entries;
> - Use ResizeableResourceHashtable to for the underlying hash table. Table growing/resizing and removal of freed object entries are mostly handled within ResizeableResourceHashtable, except some small parts are done within TsanOopMapTable. That makes the new implementation simpler.
> 
> The TsanOopMapTable contains entries consisting of TsanOopMapTableKey:oop_size (key:value) pairs. The TsanOopMapTableKey contains:
> 
>  - A Weakhandle that holds a pointer to the oop;
>  - The original (or updated) address of the object in Java heap;
> 
> For TsanOopMapTable support, I added a new weak OopStorage, which is created during tsan_init(). All WeakHandles used for tracking Tsan interested Java objects are created from that OopStorage. GC processes these WeakHandles with `WeakProcessor::Task::work` in concurrent worker threads.
> 
> To notify Tsan about object free/move in time, I added a call in WeakProcessor::Task::work to process the TsanOopMap:
> 
> - If an object is freed by GC, call __tsan_java_free to notify Tsan. Remove the entry from the table.
> - If an object is moved by GC, update the raw address in the entry using the new oop address obtained from the WeakHandle and add the ?move? to a GrowableArray.
> - After all entries in TsanOopMap are processed, sort the ?moves? in the GrowableArray and notify Tsan by calling __tsan_java_move for each object in the sorted array. This part is the existing implementation from JDK 11 for Tsan support.
> 
> Note all above operations occur during GC, before any of the mutators sees a moved or freed Java object.
> 
> As an alternative, I initially experimented with doing the above operations concurrently (to mutators) in ServiceThread after GC finished processing the WeakHandles. That could occur after mutators sees moved objects and resulted incorrect data being recorded in Tsan.
> 
> Suppressed following Tsan errors in JDK:
> 1)  In `java.util.concurrent.locks`, for
> 
> WARNING: ThreadSanitizer: dat...

This pull request has now been integrated.

Changeset: 87a8248d
Author:    Jiangli Zhou <jiangli at openjdk.org>
URL:       https://git.openjdk.org/tsan/commit/87a8248d31ad9f20a22a3d2e0626af739eacd6f7
Stats:     719 lines in 11 files changed: 368 ins; 305 del; 46 mod

Reimplement TsanOopMap support using WeakHandle and ResizeableResourceHashtable

Reviewed-by: manc

-------------

PR: https://git.openjdk.org/tsan/pull/19

From jiangli at openjdk.org  Tue Jul 30 22:53:13 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Tue, 30 Jul 2024 22:53:13 GMT
Subject: RFR: Merge jdk21u
In-Reply-To: <x8AxYIldQ6ilT2OW4dGsPofuW6X0TIoqYGHM9-dthSw=.2b49806b-d1ac-4bee-94f7-93cf8c480423@github.com>
References: <qXzzRCrFt_IyKl8CeqFqcRtz_xZb0pzqXEANyv-7EaQ=.7187be2c-1ead-4915-8a5d-27e23d86de26@github.com>
 <x8AxYIldQ6ilT2OW4dGsPofuW6X0TIoqYGHM9-dthSw=.2b49806b-d1ac-4bee-94f7-93cf8c480423@github.com>
Message-ID: <7nsFb2fBntwF7Mi7l9KWHlvySDIG-AF4e9rrLTXPD5w=.394e5da4-a88e-425a-bab6-a68d2d840979@github.com>

On Tue, 30 Jul 2024 20:19:53 GMT, Man Cao <manc at openjdk.org> wrote:

> Could we create a new branch (e.g. "tsan-21u") in the TSAN project to track TSAN based on JDK 21u? The current "tsan" branch in intended to track OpenJDK's tip.

SG for creating tsan-21u branch.

-------------

PR Comment: https://git.openjdk.org/tsan/pull/21#issuecomment-2259331234

From manc at openjdk.org  Tue Jul 30 22:53:13 2024
From: manc at openjdk.org (Man Cao)
Date: Tue, 30 Jul 2024 22:53:13 GMT
Subject: RFR: Merge jdk21u
In-Reply-To: <qXzzRCrFt_IyKl8CeqFqcRtz_xZb0pzqXEANyv-7EaQ=.7187be2c-1ead-4915-8a5d-27e23d86de26@github.com>
References: <qXzzRCrFt_IyKl8CeqFqcRtz_xZb0pzqXEANyv-7EaQ=.7187be2c-1ead-4915-8a5d-27e23d86de26@github.com>
Message-ID: <x8AxYIldQ6ilT2OW4dGsPofuW6X0TIoqYGHM9-dthSw=.2b49806b-d1ac-4bee-94f7-93cf8c480423@github.com>

On Tue, 30 Jul 2024 19:04:57 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

> This PR merges with JDK 21.0.3 (https://github.com/openjdk/jdk/commit/05f18b10620834883180013bb8873a6b4bdc6df2).
> 
> Merge was done most automatically. See manual edit and change details below.
> 
> **Manual changes:**
> - Edited `.jcheck/conf` and `src/hotspot/share/logging/logTag.hpp` to resolve trivial merge conflicts.
> - Changed `TsanRunner `to use `ProcessTools.createLimitedTestJavaProcessBuilder()` to resolve tsan jtreg test failures. `ProcessTools.createJavaProcessBuilder()` was renamed to `ProcessTools.createLimitedTestJavaProcessBuilder` by https://github.com/openjdk/jdk/pull/15452.
> 
> **Merge commands:**
> 
> $ git remote add 21u https://github.com/openjdk/jdk21u.git
> $ git remote update
> $ git merge 05f18b1
> #resolve merge conflicts and commit
> $ git remote remove 21u
> 
> 
> **Testing:**
> 
> $ bash configure --with-boot-jdk=/usr/local/google/home/jianglizhou/openjdk/jdk-21.0.1 --with-debug-level=release --disable-warnings-as-errors --with-jtreg=/usr/local/google/home/jianglizhou/github/jtreg/build/images/jtreg --with-stdc++lib=static --disable-precompiled-headers --enable-unlimited-crypto --with-native-debug-symbols=internal --with-default-make-target=jdk-image --disable-warnings-as-errors --with-toolchain-type=clang --disable-warnings-as-errors
> 
> 
> 
> $ make test TEST=hotspot/jtreg/tsan
> 
> All tests pass with `release` binary
> 
> ==============================
> Test summary
> ==============================
>    TEST                                              TOTAL  PASS  FAIL ERROR   
>    jtreg:test/hotspot/jtreg/tsan                        79    79     0     0   
> ==============================
> TEST SUCCESS

Thank you for the merge.
Could we create a new branch (e.g. "tsan-21u") in the TSAN project to track TSAN based on JDK 21u? The current "tsan" branch in intended to track OpenJDK's tip.

-------------

PR Comment: https://git.openjdk.org/tsan/pull/21#issuecomment-2259140988

From jiangli at openjdk.org  Tue Jul 30 22:53:13 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Tue, 30 Jul 2024 22:53:13 GMT
Subject: RFR: Merge jdk21u
Message-ID: <qXzzRCrFt_IyKl8CeqFqcRtz_xZb0pzqXEANyv-7EaQ=.7187be2c-1ead-4915-8a5d-27e23d86de26@github.com>

This PR merges with JDK 21.0.3 (https://github.com/openjdk/jdk/commit/05f18b10620834883180013bb8873a6b4bdc6df2).

Merge was done most automatically. See manual edit and change details below.

**Manual changes:**
- Edited `.jcheck/conf` and `src/hotspot/share/logging/logTag.hpp` to resolve trivial merge conflicts.
- Changed `TsanRunner `to use `ProcessTools.createLimitedTestJavaProcessBuilder()` to resolve tsan jtreg test failures. `ProcessTools.createJavaProcessBuilder()` was renamed to `ProcessTools.createLimitedTestJavaProcessBuilder` by https://github.com/openjdk/jdk/pull/15452.

**Merge commands:**

$ git remote add 21u https://github.com/openjdk/jdk21u.git
$ git remote update
$ git merge 05f18b1
#resolve merge conflicts and commit
$ git remote remove 21u


**Testing:**

$ bash configure --with-boot-jdk=/usr/local/google/home/jianglizhou/openjdk/jdk-21.0.1 --with-debug-level=release --disable-warnings-as-errors --with-jtreg=/usr/local/google/home/jianglizhou/github/jtreg/build/images/jtreg --with-stdc++lib=static --disable-precompiled-headers --enable-unlimited-crypto --with-native-debug-symbols=internal --with-default-make-target=jdk-image --disable-warnings-as-errors --with-toolchain-type=clang --disable-warnings-as-errors



$ make test TEST=hotspot/jtreg/tsan

All tests pass with `release` binary

==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR   
   jtreg:test/hotspot/jtreg/tsan                        79    79     0     0   
==============================
TEST SUCCESS

-------------

Commit messages:
 - Change TsanRunner to use ProcessTools.createLimitedTestJavaProcessBuilder(). ProcessTools.createJavaProcessBuilder() was renamed to ProcessTools.createLimitedTestJavaProcessBuilder by https://github.com/openjdk/jdk/pull/15452.
 - Merge commit '05f18b1' into tsan-21-0-3-merge
 - 8329838: [21u] Remove designator DEFAULT_PROMOTED_VERSION_PRE=ea for release 21.0.3
 - 8319851: Improve exception logging
 - 8322122: Enhance generation of addresses
 - 8318340: Improve RSA key implementations
 - 8315708: Enhance HTTP/2 client usage
 - 8327391: Add SipHash attribution file
 - 8322750: Test "api/java_awt/interactive/SystemTrayTests.html" failed because A blue ball icon is added outside of the system tray
 - 8323664: java/awt/font/JNICheck/FreeTypeScalerJNICheck.java still fails with JNI warning on some Windows configurations
 - ... and 687 more: https://git.openjdk.org/tsan/compare/87a8248d...31baf134

The webrevs contain the adjustments done while merging with regards to each parent branch:
 - tsan: https://webrevs.openjdk.org/?repo=tsan&pr=21&range=00.0
 - jdk21u: https://webrevs.openjdk.org/?repo=tsan&pr=21&range=00.1

Changes: https://git.openjdk.org/tsan/pull/21/files
  Stats: 88923 lines in 2633 files changed: 50713 ins; 21083 del; 17127 mod
  Patch: https://git.openjdk.org/tsan/pull/21.diff
  Fetch: git fetch https://git.openjdk.org/tsan.git pull/21/head:pull/21

PR: https://git.openjdk.org/tsan/pull/21

From manc at openjdk.org  Tue Jul 30 23:44:41 2024
From: manc at openjdk.org (Man Cao)
Date: Tue, 30 Jul 2024 23:44:41 GMT
Subject: RFR: Merge jdk21u
In-Reply-To: <7nsFb2fBntwF7Mi7l9KWHlvySDIG-AF4e9rrLTXPD5w=.394e5da4-a88e-425a-bab6-a68d2d840979@github.com>
References: <qXzzRCrFt_IyKl8CeqFqcRtz_xZb0pzqXEANyv-7EaQ=.7187be2c-1ead-4915-8a5d-27e23d86de26@github.com>
 <x8AxYIldQ6ilT2OW4dGsPofuW6X0TIoqYGHM9-dthSw=.2b49806b-d1ac-4bee-94f7-93cf8c480423@github.com>
 <7nsFb2fBntwF7Mi7l9KWHlvySDIG-AF4e9rrLTXPD5w=.394e5da4-a88e-425a-bab6-a68d2d840979@github.com>
Message-ID: <zPEhd5lT5bAPZ19j7apAlHyfMLayKnCOr2l-xipw_aE=.4c1396ae-20ab-45a9-9d71-b8a91a17d827@github.com>

On Tue, 30 Jul 2024 22:50:05 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

> > Could we create a new branch (e.g. "tsan-21u") in the TSAN project to track TSAN based on JDK 21u? The current "tsan" branch in intended to track OpenJDK's tip.
> 
> SG for creating tsan-21u branch.

Asked in https://bugs.openjdk.org/browse/SKARA-2345 to allow creating new branches.

-------------

PR Comment: https://git.openjdk.org/tsan/pull/21#issuecomment-2259375365

From jiangli at openjdk.org  Wed Jul 31 00:02:50 2024
From: jiangli at openjdk.org (Jiangli Zhou)
Date: Wed, 31 Jul 2024 00:02:50 GMT
Subject: RFR: Merge jdk21u
In-Reply-To: <zPEhd5lT5bAPZ19j7apAlHyfMLayKnCOr2l-xipw_aE=.4c1396ae-20ab-45a9-9d71-b8a91a17d827@github.com>
References: <qXzzRCrFt_IyKl8CeqFqcRtz_xZb0pzqXEANyv-7EaQ=.7187be2c-1ead-4915-8a5d-27e23d86de26@github.com>
 <x8AxYIldQ6ilT2OW4dGsPofuW6X0TIoqYGHM9-dthSw=.2b49806b-d1ac-4bee-94f7-93cf8c480423@github.com>
 <7nsFb2fBntwF7Mi7l9KWHlvySDIG-AF4e9rrLTXPD5w=.394e5da4-a88e-425a-bab6-a68d2d840979@github.com>
 <zPEhd5lT5bAPZ19j7apAlHyfMLayKnCOr2l-xipw_aE=.4c1396ae-20ab-45a9-9d71-b8a91a17d827@github.com>
Message-ID: <a6sYpIaZHEQP6neMJfY8qZMxgbrSlei_fwiTZVe89uQ=.3f76f7d1-e98a-4de7-89cd-33948a087e36@github.com>

On Tue, 30 Jul 2024 23:41:37 GMT, Man Cao <manc at openjdk.org> wrote:

> > Could we create a new branch (e.g. "tsan-21u") in the TSAN project to track TSAN based on JDK 21u? The current "tsan" branch in intended to track OpenJDK's tip.
> 
> SG for creating tsan-21u branch.

An alternative is to only track the JDK mainline and not track any of the JDK update repo. That can avoid complications with handling branches. Let me try integrate the current tsan repo internally.

For JDK 21, there is a bit delay. For future JDK LTS/STS integrations, we can integrate them to the tsan repo sooner after a `ga` release.

-------------

PR Comment: https://git.openjdk.org/tsan/pull/21#issuecomment-2259389045