RFR: 8170297: runtime/SharedArchiveFile/LargeSharedSpace.java didn't run out of memory

[Jiangli Zhou]

Please review the following fix for test bug JDK-8170297 <https://bugs.openjdk.java.net/browse/JDK-8170297>. 

  http://cr.openjdk.java.net/~jiangli/8170297/webrev.00/ <http://cr.openjdk.java.net/~jiangli/8170297/webrev.00/>

On some test platforms, the archive might be dumped successfully with the specified large size. Added an extra level of checking in the test. Tested with linux-64, linux-32 and aarch64.

Thanks,
Jiangli