RFR: 8352016: Improve java/lang/RuntimeTests/RuntimeExitLogTest.java

[Roger Riggs]

On Fri, 14 Mar 2025 09:17:06 GMT, KIRIYAMA Takuya <duke at openjdk.org> wrote:

> The current test program for the logging feature added in JDK-8301627 does not fully check some important cases.
> 
> Issue Details:
> The test does not properly check cases where logging might not happen due to different logging levels. (e.g. ALL, TRACE, WARNING, etc.)
> The check for the logged stack trace is not enough, as it does not confirm enough details in the output.
> 
> Fix Details:
> Added more test cases to check behavior under different logging levels.
> Improved the stack trace check by verifying more details in the logged output.
> These changes make the test more complete and ensure that the logging feature works as expected.
> Also, any existing test cases prior to this pull request are retained.
> 
> The test was verified in the following OS environments, and it passed successfully in both environments.
> - Windows Server 2022 Standard 21H2
> - Red Hat Enterprise Linux release 9.2 (Plow)
> 
> Could you please review this fix?

Out of curiosity, what raised this as an issue that needed more tests? 
Was there a failure of some implementation that caused a bug?

test/jdk/java/lang/RuntimeTests/ExitLogging-FINE.properties line 2:

> 1: ############################################################
> 2: #       java.lang. Runtime logging level to console to FINE

Extra space before Runtime.
Please fix in all .properties files.

test/jdk/java/lang/RuntimeTests/ExitLogging-FINER.properties line 8:

> 6: 
> 7: java.util.logging.ConsoleHandler.level = ALL
> 8: java.lang.Runtime.level = FINER

It is a waste of resources to test every combination of logger levels.
This is not a test of the filtering mechanism built into the logger; it is a test that the message is logged when the logger level is set to DEBUG or finer.
Only two test cases are needed for that.

test/jdk/java/lang/RuntimeTests/RuntimeExitLogTest.java line 78:

> 76:                 "java\\.lang\\.Throwable: Runtime\\.exit\\(" + status + "\\)\\n" +
> 77:                 "\\s+at java\\.base/java\\.lang\\.Shutdown\\.logRuntimeExit\\(Shutdown\\.java:\\d+\\)\\n" +
> 78:                 "\\s+at(?: .+)";

Testing the exact logging message is too brittle/too exact.
It should only be testing for the specific variable information that is unique to the case under test.
It becomes burdensome to update tests when some small change occurs in a message.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/24050#issuecomment-2737976829
PR Review Comment: https://git.openjdk.org/jdk/pull/24050#discussion_r2004182051
PR Review Comment: https://git.openjdk.org/jdk/pull/24050#discussion_r2004204027
PR Review Comment: https://git.openjdk.org/jdk/pull/24050#discussion_r2004191748