RFR: 7903621: jtreg ignores VM exit code when test process reports status with "STATUS: " line [v2]

[Christian Stein]

On Mon, 11 Mar 2024 05:14:13 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review of this change which proposes to address the issue reported in https://bugs.openjdk.org/browse/CODETOOLS-7903621?
>> 
>> When jtreg launches a new java process for the test (like for running a `othervm` test), it uses an internal main class which among other things reports back a status when the test completes. This it does by writing out a status line to `System.err` before calling `System.exit()` with the appropriate code that represents the execution status of the test.
>> 
>> In the case where the test passes, this main class writes out a status line which represents a passed test and then calls `System.exit()` with an exit code that too represents a passed test. However, there are cases where the JVM process might exit with a different error code due to errors/exceptions/VM crash, during the shutdown sequence. This effectively implies that the test did not really complete successfully. In its current form, jtreg notices the status line which was written out as "PASSED" and then completely ignores the exit code of the process, which may not be representing a passed test. Such tests are incorrectly marked as a PASSED and any issue that happened during the JVM exit aren't reported or drawn attention to.
>> 
>> The commit in this PR proposes to address that by verifying the exit code of the test process matches the status line it reported. In case of a mismatch, the test is marked as "error" and the mismatch is now reported back as an error, which should now draw attention to the cause of such errors.
>> 
>> A new self test has been introduced which reproduces the issue with the source change and verifies the fix. All existing self tests and this new test pass with this change.
>
> Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix error message check

Looks good to me.

-------------

Marked as reviewed by cstein (Committer).

PR Review: https://git.openjdk.org/jtreg/pull/191#pullrequestreview-1927514552