[jdk21u-dev] RFR: 8320836: jtreg gtest runs should limit heap size

[Rui Li]

A clean backport for https://bugs.openjdk.org/browse/JDK-8320836.

This patch limits the gtest max heap size. In the original [JBS](https://bugs.openjdk.org/browse/JDK-8320836), there was already a customer report saying CI pipeline gtest failed due to large heap. Backporting this patch can benefit non tip customers. Running tests. In tip for ~2 months.

-------------

Commit messages:
 - Backport ac9e51023fc34a82b795950a109af2397826adaa

Changes: https://git.openjdk.org/jdk21u-dev/pull/2161/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk21u-dev&pr=2161&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8320836
  Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk21u-dev/pull/2161.diff
  Fetch: git fetch https://git.openjdk.org/jdk21u-dev.git pull/2161/head:pull/2161

PR: https://git.openjdk.org/jdk21u-dev/pull/2161