RFR: 7903324: Improve per-class reporting of JUnit tests, in .jtr file

[Alan Bateman]

On Tue, 27 Sep 2022 13:30:28 GMT, Christian Stein <cstein at openjdk.org> wrote:

>> `System.err` seems to be more correct, looking at other printing code in jtreg. I used `System.out` first, which was remarked by @jonathan-gibbons in a previous note.
>> 
>>> If my test prints messages to System.out them how will I know which test they came from?
>> 
>> The canonical way is Jupiter's `TestReporter` as described in [Writing Tests - Dependency Injection for Constructors and Methods](https://junit.org/junit5/docs/current/user-guide/#writing-tests-dependency-injection)
>> 
>>> The `TestReporter` can be used to publish additional data about the current test run. The data can be consumed via the `reportingEntryPublished()` method in a [TestExecutionListener](https://junit.org/junit5/docs/current/api/org.junit.platform.launcher/org/junit/platform/launcher/TestExecutionListener.html), allowing it to be viewed in IDEs or included in reports.
>>> ...
>>> In JUnit Jupiter you should use TestReporter where you used to print information to stdout or stderr in JUnit 4.
>> 
>> 
>> class TestReporterDemo {
>> 
>>     @Test
>>     void reportSingleValue(TestReporter testReporter) {
>>         testReporter.publishEntry("a status message");
>>     }
>> 
>>     @Test
>>     void reportKeyValuePair(TestReporter testReporter) {
>>         testReporter.publishEntry("a key", "a value");
>>     }
>> 
>>     @Test
>>     void reportMultipleKeyValuePairs(TestReporter testReporter) {
>>         var values = Map.of("user name", "dk38", "award year", "1974");
>>         testReporter.publishEntry(values);
>>     }
>> 
>> }
>
> Having said that, there's also: [Capturing Standard Output/Error](https://junit.org/junit5/docs/current/user-guide/#running-tests-capturing-output), an opt-in support for capturing output printed to `System.out` and `System.err`.
>> If enabled, the JUnit Platform captures the corresponding output and publishes it as a report entry using the stdout or stderr keys to all registered [TestExecutionListener](https://junit.org/junit5/docs/current/api/org.junit.platform.launcher/org/junit/platform/launcher/TestExecutionListener.html) instances immediately before reporting the test or container as finished.
> 
> Shall we enable capturing standard streams by default?

With TestNG, the "test $testname: $result" messages show up in the jtr System.out section. If a test prints anything then it will inlined and appear before the message emitted by TestNG. So this will take a bit of getting used to.

-------------

PR: https://git.openjdk.org/jtreg/pull/127