[jdk17u-dev] RFR: 8339834: Replace usages of -mx and -ms in some tests

[Goetz Lindenmaier]

I backport this to clean up the 17 testing and make later backports clean.

-------------

Commit messages:
 - Backport 1d392492311daceeae12769cb9494eae63289853

Changes: https://git.openjdk.org/jdk17u-dev/pull/3710/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk17u-dev&pr=3710&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8339834
  Stats: 22 lines in 9 files changed: 0 ins; 5 del; 17 mod
  Patch: https://git.openjdk.org/jdk17u-dev/pull/3710.diff
  Fetch: git fetch https://git.openjdk.org/jdk17u-dev.git pull/3710/head:pull/3710

PR: https://git.openjdk.org/jdk17u-dev/pull/3710