RFR: 7903324: Improve per-class reporting of JUnit tests, in .jtr file

[Christian Stein]

On Tue, 27 Sep 2022 13:22:26 GMT, Christian Stein <cstein at openjdk.org> wrote:

>> src/share/classes/com/sun/javatest/regtest/agent/JUnitRunner.java line 136:
>> 
>>> 134:                 Launcher launcher = session.getLauncher();
>>> 135:                 launcher.registerTestExecutionListeners(summaryGeneratingListener);
>>> 136:                 launcher.registerTestExecutionListeners(new PrintingListener(System.err));
>> 
>> Is System.err correct? If my test prints messages to System.out them how will I know which test they came from?
>
> `System.err` seems to be more correct, looking at other printing code in jtreg. I used `System.out` first, which was remarked by @jonathan-gibbons in a previous note.
> 
>> If my test prints messages to System.out them how will I know which test they came from?
> 
> The canonical way is Jupiter's `TestReporter` as described in [Writing Tests - Dependency Injection for Constructors and Methods](https://junit.org/junit5/docs/current/user-guide/#writing-tests-dependency-injection)
> 
>> The `TestReporter` can be used to publish additional data about the current test run. The data can be consumed via the `reportingEntryPublished()` method in a [TestExecutionListener](https://junit.org/junit5/docs/current/api/org.junit.platform.launcher/org/junit/platform/launcher/TestExecutionListener.html), allowing it to be viewed in IDEs or included in reports.
>> ...
>> In JUnit Jupiter you should use TestReporter where you used to print information to stdout or stderr in JUnit 4.
> 
> 
> class TestReporterDemo {
> 
>     @Test
>     void reportSingleValue(TestReporter testReporter) {
>         testReporter.publishEntry("a status message");
>     }
> 
>     @Test
>     void reportKeyValuePair(TestReporter testReporter) {
>         testReporter.publishEntry("a key", "a value");
>     }
> 
>     @Test
>     void reportMultipleKeyValuePairs(TestReporter testReporter) {
>         var values = Map.of("user name", "dk38", "award year", "1974");
>         testReporter.publishEntry(values);
>     }
> 
> }

Having said that, there's also: [Capturing Standard Output/Error](https://junit.org/junit5/docs/current/user-guide/#running-tests-capturing-output), an opt-in support for capturing output printed to `System.out` and `System.err`.
> If enabled, the JUnit Platform captures the corresponding output and publishes it as a report entry using the stdout or stderr keys to all registered [TestExecutionListener](https://junit.org/junit5/docs/current/api/org.junit.platform.launcher/org/junit/platform/launcher/TestExecutionListener.html) instances immediately before reporting the test or container as finished.

Shall we enable capturing standard streams by default?

-------------

PR: https://git.openjdk.org/jtreg/pull/127