RFR(S): 8247941: Use Vectored Exception Handling on Windows

[Ludovic Henry]

(Renaming subject to include bug id, original discussion at [1]). 

Webrev: http://cr.openjdk.java.net/~luhenry/8247941/webrev.00/
JBS: https://bugs.openjdk.java.net/browse/JDK-8247941

I've uploaded the webrev at the link above.

[1] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040228.html

-----Original Message-----
From: hotspot-runtime-dev <hotspot-runtime-dev-retn at openjdk.java.net> On Behalf Of Ludovic Henry
Sent: Tuesday, August 4, 2020 8:34 AM
To: Thomas Stüfe <thomas.stuefe at gmail.com>; hotspot-runtime-dev at openjdk.java.net
Subject: Re: RFR(S): Use Vectored Exception Handling on Windows

Hi,

An update on my current exploration is long overdue.

# On the use of Unhandled Exception Handling.
Unfortunately, this is a dead-end because UEH itself relies on Structured Exception Handling to catch any uncaught exception at the top-most frame, and then needs a properly unwindable stack. This is not the case for Java frames as Hotspot follows its own internal ABI, making it impossible for the native unwinder (which doesn't know about the Java ABI) to walk the stack frames. The easy fallback is simply to keep the existing `__try {} __except (topLevelExceptionFilter(...)) {}` and call to `RtlAddFunctionTable` to add such try/except block to the code cache.

# On the use of Vectored Exception Handling
Despite UEH not being usable in our case, VEH is still helpful in getting Hotspot to treat Hotspot related exceptions first and to separate the normal exception handling from the crash handling.

I'll post an updated webrev with these changes later today.

Thank you,
Ludovic