RFR: 8267188: gc/stringdedup/TestStringDeduplicationInterned.java fails with Shenandoah [v2]

[Zhengyu Gu]

> Shenandoah currently enqueues deduplication candidates when marks object gray.
> 
> However, it can not do so when it scans roots, as it can potentially result lock rank inversion between stack watermark lock and dedup request buffer lock.
> 
> As the result, it can not enqueue as many as candidates as it should be able to, I believe JDK-8271834 is due to the same problem. I purpose we switch to enqueue candidates when we mark object black.
> 
> We are still not able to enqueue all candidates, only when they have displaced headers or have monitor inflating in progress. Upstream is working on removing displaced headers, we should revisit the logic afterward, or we can choose to deduplicate all string regardless their ages.
> 
> 
> 
> Test:
> - [x] hotspot_gc_shenandoah
> - [x] tier1 with -XX:+UseShenandoahGC and -XX:+UseStringDeduplication

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  Aleksey's comments

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/5151/files
  - new: https://git.openjdk.java.net/jdk/pull/5151/files/98eedbd5..18e59bd5

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5151&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5151&range=00-01

  Stats: 3 lines in 2 files changed: 1 ins; 1 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/5151.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/5151/head:pull/5151

PR: https://git.openjdk.java.net/jdk/pull/5151