[jdk11u-dev] RFR: 8225790: Two NestedDialogs tests fail on Ubuntu

[Aleksey Shipilev]

Clean backport to stabilize the tests and match `11.0.13-oracle`.

Additional testing:
 - [x] Ubuntu 20.04 passes

-------------

Commit messages:
 - Backport 21e67e54604e60f57106be32d48ba75ebf8c1130

Changes: https://git.openjdk.java.net/jdk11u-dev/pull/254/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk11u-dev&pr=254&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8225790
  Stats: 788 lines in 4 files changed: 81 ins; 610 del; 97 mod
  Patch: https://git.openjdk.java.net/jdk11u-dev/pull/254.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk11u-dev pull/254/head:pull/254

PR: https://git.openjdk.java.net/jdk11u-dev/pull/254