[jdk17u-dev] RFR: 8376270: [21u, 17u] Redo JDK-8361748: Enforce limits on the size of an XBM image

[Goetz Lindenmaier]

I backport this for parity with 17.0.18-oracle.

We had backed out the original backport in 17.0.18.

-------------

Commit messages:
 - Backport 7aaa31159d6152dce6bcd3a6d32bc4f25b25b3a3

Changes: https://git.openjdk.org/jdk17u-dev/pull/4244/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk17u-dev&pr=4244&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8376270
  Stats: 245 lines in 7 files changed: 157 ins; 33 del; 55 mod
  Patch: https://git.openjdk.org/jdk17u-dev/pull/4244.diff
  Fetch: git fetch https://git.openjdk.org/jdk17u-dev.git pull/4244/head:pull/4244

PR: https://git.openjdk.org/jdk17u-dev/pull/4244