RFR: 8364664: gtest death tests failing on Windows

[Kim Barrett]

On Wed, 6 Aug 2025 18:05:24 GMT, Saint Wesonga <duke at openjdk.org> wrote:

> https://github.com/openjdk/jdk/commit/0054bbed7fce5b8566655d6910b09b10c952e609 (from https://bugs.openjdk.org/browse/JDK-8343756) caused the gtest death tests to fail on Windows with the error message "Caught std::exception-derived exception escaping the death test statement. Exception message: unknown file: error: SEH exception with code 0xc0000005 thrown in the test body." The error message is from the catch block in https://github.com/google/googletest/blob/v1.14.0/googletest/include/gtest/internal/gtest-death-test-internal.h#L198-L212
> 
> In the failing death tests, the gtests start another process and expect the child process to be terminated by JVM error handling code. However, the structured exception handling code in the googletest code ends up getting executed instead. The death tests expect execution to continue after the instruction that triggered the exception by writing to the poissoned page. This change proposes build Windows gtests without structured exception handling to avoid changing the flow of exceptions in OpenJDK test code. The effect of this change is to stop using the  [SEH path in the HandleSehExceptionsInMethodIfSupported method](https://github.com/google/googletest/blob/v1.14.0/googletest/src/gtest.cc#L2603) and [directly start the test](https://github.com/google/googletest/blob/v1.14.0/googletest/src/gtest.cc#L2612).
> 
> All the Windows gtests now pass with this change.

How is it that we (Oracle) don't see these gtest death test failures in our
CI? I'm guessing others (like SAP - @tstuefe  ?) aren't either, since this
issue is newly reported while the causing change was made more than 8 months
ago.

The code in gtest-death-test-internal.h#L198-L212 referenced in the PR
description is conditionalized on GTEST_HAS_EXCEPTIONS, which leads me to
wonder why exceptions are enabled. If not for that, we wouldn't be in that
code.

That led me to wonder why, on Windows, we build libgtest and rebuild libjvm
with exceptions enabled, by using -EHsc instead of no -EH option as done for
the non-gtest libjvm?

https://github.com/openjdk/jdk/blame/f95af744b07a9ec87e2507b3d584cbcddc827bbd/make/hotspot/lib/CompileGtest.gmk#L71
https://github.com/openjdk/jdk/blame/f95af744b07a9ec87e2507b3d584cbcddc827bbd/make/hotspot/lib/CompileGtest.gmk#L101

That decision seems pretty old, like maybe from the initial introduction of
gtest. I haven't tracked down why, or whether the reasons are still valid. I
think it would be better to change that, assuming that's possible.

I was concerned that this might effectively undo JDK-8185734, since the
suggestion in JBS was to conditionalize some code on GTEST_HAS_SEH being true.
But it looks like the actual change didn't do that.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26661#issuecomment-3162014034