RFR: 8358343: [leyden] Drop notify_all in CompilationPolicyUtils::Queue::pop [v3]

[Aleksey Shipilev]

> Found this when reading premain-vs-mainline webrev. Mainline does not have `notify_all` in this method:
> https://github.com/openjdk/jdk/blob/c382da579884c28f2765b2c6ba68c0ad4fdcb2ce/src/hotspot/share/compiler/compilationPolicy.hpp#L85-L92
> 
> But if you remove `notify_all()` in `premain`, then tests start to deadlock, see bug for a sample. The culprit is `CompilationPolicy::flush_replay_training_at_init`, which is only present in premain. I fixed it by using timed waits, which obviates the need for extra notifications. We only enter this method with `-XX:+AOTVerifyTrainingData`, so we don't care much about its performance. This is IMO better than doing a questionable `notify_all` followed by `wait` in load-bearing code.
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, `runtime/cds` (5x, no timeouts yet; still running more iterations)

Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:

 - Coordinate with replay thread shutdown
 - Merge branch 'premain' into JDK-8358343-leyden-training-notify-all
 - Merge branch 'premain' into JDK-8358343-leyden-training-notify-all
 - Fix

-------------

Changes:
  - all: https://git.openjdk.org/leyden/pull/74/files
  - new: https://git.openjdk.org/leyden/pull/74/files/8af7ae69..f8c4debb

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=leyden&pr=74&range=02
 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=74&range=01-02

  Stats: 182204 lines in 3878 files changed: 112334 ins; 45443 del; 24427 mod
  Patch: https://git.openjdk.org/leyden/pull/74.diff
  Fetch: git fetch https://git.openjdk.org/leyden.git pull/74/head:pull/74

PR: https://git.openjdk.org/leyden/pull/74