RFR: 8366118: DontCompileHugeMethods is not respected with -XX:-TieredCompilation [v3]

[Tobias Hartmann]

On Wed, 27 Aug 2025 09:01:28 GMT, Man Cao <manc at openjdk.org> wrote:

>> Hi,
>> 
>> Could anyone review this change that fixes https://bugs.openjdk.org/browse/JDK-8366118? When this bug happens, it is difficult or almost impossible to debug due to the lack of stack trace, hs-err log or core dump. Fortunately we are also experimenting with sigaltstack for https://bugs.openjdk.org/browse/JDK-8364654, and it helped immensely to identify the root cause.
>> 
>> I will also try adding a test case for DontCompileHugeMethod under -XX:-TieredCompilation.
>> 
>> -Man
>
> Man Cao has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Use List.of in test

I think the test needs `-Xbatch`

The new test fails in our testing:


----------System.err:(20/1081)----------
 stdout: [590    1     n       jdk.internal.misc.Unsafe::getReferenceVolatile (native)   
595    2     n       jdk.internal.vm.Continuation::enterSpecial (native)   (static)
595    3     n       jdk.internal.vm.Continuation::doYield (native)   (static)
1999665
];
 stderr: []
 exitValue = 0

java.lang.RuntimeException: '   HugeSwitch::shortMethod (' missing from stdout/stderr
	at jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:253)
	at compiler.runtime.TestDontCompileHugeMethods.runTest(TestDontCompileHugeMethods.java:110)
	at compiler.runtime.TestDontCompileHugeMethods.main(TestDontCompileHugeMethods.java:119)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:565)
	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:335)
	at java.base/java.lang.Thread.run(Thread.java:1474)

-------------

Changes requested by thartmann (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/26932#pullrequestreview-3168288213
PR Comment: https://git.openjdk.org/jdk/pull/26932#issuecomment-3236599206