Request for review: 6761744 Hotspot crashes if process size limit is exceeded

[Tao Mao]

6761744 Hotspot crashes if process size limit is exceeded
https://jbs.oracle.com/bugs/browse/JDK-6761744

webrev:
http://cr.openjdk.java.net/~tamao/6761744/webrev.00/

changeset:
The fix only needs to go to hsx24 since there's no perm gen in 
hotspot-25. Thus, the webrev is based on hsx24 repo.

It provides for 32-bit builds a preventive check of the size of "the 
object heap + perm gen" before reserving VM memory. The total size 
should not exceed 4096MB (i.e. 4GB) for 32-bit builds; otherwise, the 
total doesn't make sense and, what's worse, overflow occurs. It will 
consequentially trigger anther error of memory access violation, which 
was not protected.

jtreg testing java code is also written, checking both 32-bit and 
(trivially) 64-bit builds.

testing:
check jtreg tests with flags -XX:+UseParallelGC, -XX:+UseG1GC, 
-XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, -XX:+UseSerialGC and builds 
of 32-bit and 64-bit. All passed.

Needs JPRT test when pushing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20130407/ecf627e7/attachment.htm>