Integrated: 8308094: Add a compilation timeout flag to catch long running compilations

[Manuel Hässig]

On Fri, 27 Jun 2025 17:36:54 GMT, Manuel Hässig <mhaessig at openjdk.org> wrote:

> This PR adds `-XX:CompileTaskTimeout` on Linux to limit the amount of time a compilation task can run. The goal of this is initially to be able to find and investigate long-running compilations.
> 
> The timeout is implemented using a POSIX timer that sends a `SIGALRM` to the compiler thread the compile task is running on. Each compiler thread registers a signal handler that triggers an assert upon receiving `SIGALRM`. This is currently only implemented for Linux, because it relies on `SIGEV_THREAD_ID` to get the signal delivered to the same thread that timed out.
> 
> Since `SIGALRM` is now used, the test `runtime/signal/TestSigalrm.java` now requires `vm.flagless` so it will not interfere with the compiler thread signal handlers.
> 
> Testing:
>  - [x] Github Actions
>  - [x] tier1, tier2 on all platforms
>  - [x] tier3, tier4 and Oracle internal testing on Linux fastdebug
>  - [x] tier1 through tier4 with `-XX:CompileTaskTimeout=60000` (one minute timeout) to see what fails (`compiler/codegen/TestAntiDependenciesHighMemUsage2.java`, `compiler/loopopts/TestMaxLoopOptsCountReached.java`, and `compiler/c2/TestScalarReplacementMaxLiveNodes.java` fail)

This pull request has now been integrated.

Changeset: c74c60fb
Author:    Manuel Hässig <mhaessig at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/c74c60fb8b8aa5c917fc4e1c157cc8083f5797a0
Stats:     282 lines in 8 files changed: 279 ins; 0 del; 3 mod

8308094: Add a compilation timeout flag to catch long running compilations

Co-authored-by: Dean Long <dlong at openjdk.org>
Reviewed-by: dlong, chagedorn

-------------

PR: https://git.openjdk.org/jdk/pull/26023