RFR(XS) 8239005: [TESTBUG] test/hotspot/jtreg/runtime/StackGuardPages/TestStackGuardPages.java: exeinvoke.c: must initialize static state before calling do_overflow()

[Reingruber, Richard]

Hi,

please review this small test fix

Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239005/webrev.0/
Bug:    https://bugs.openjdk.java.net/browse/JDK-8239005

On one of our linuxppc64le boxes the test was failing due to the missing reset of the static
variable _rec_count before the first call to do_overflow() [1] in run_native_overflow() [2].
Consequently the recursion limit set at [3] is too high. On our linuxppc64le box we then get a
SIGSEGV with SEGV_ACCERR which is reported as test failure.

On my linuxx86_64 box I get an SIGSEGV with SEGV_MAPERR which is not reported as error.

With the fix we don't get a second SIGSEGV. Maybe a second SIGSEGV should be reported as failure...?

I tested on linuxx86_64 and linuxppc64le

Many more details to be found in the bug.

Thanks, Richard.

[1] definition of do_overflow()
    http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c#l128

[2] _rec_count is not reset to 0 before call to do_overflow()
     http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c#l168

[3] recursion limit is set too high
    http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c#l191