RFR: 7904153: JTReg should include crash log data in the failure node for xml reports

William Kemper wkemper at openjdk.org
Tue Feb 17 18:40:55 UTC 2026 

On Tue, 17 Feb 2026 16:06:41 GMT, Christian Stein <cstein at openjdk.org> wrote:

>> The system we use (AWS CodeBuild) for running JTReg understands Junit/testNG xml files. For failed tests, it makes the text from the `failure` node easy to access from the UI. I appreciate that these xml files already have `system.err` and `system.out` nodes, but the tooling we use doesn't expose the data here (and I cannot change that tooling). Also, the current implementation takes the first section of `stderr` or `stdout`, which is often empty and comes from the `javac` portion of the test run.
>
> src/share/classes/com/sun/javatest/regtest/report/XMLWriter.java line 185:
> 
>> 183:                 Section s = tr.getSection(i);
>> 184:                 for (String x : s.getOutputNames()) {
>> 185:                     return s.getOutput(name);
> 
> Although I do agree that this code doesn't make too much sense, especially as variable `x` is unused, but it does some work since 2012. Maybe, others do rely on this feature? ➡️ https://www.hyrumslaw.com
> 
> Did you run this against more/all JDK tests and verify that no other party, like some test output parser tool, will break?

Hi, thank you for looking at this PR. I've run `test/hotspot/jtreg:all` through AWS `CodeBuild` reports. Of course, I cannot test all the possible Junit XML parsers in the world, but I did test these popular search results on my change (with a little help from a friend 😉). If there is some specific parser you would like me to test, please let me know.


<?xml version="1.0" encoding="UTF-8" ?>
<testsuite errors="0" failures="1" tests="1" hostname="776afbe4ce00" name="gc/shenandoah/TestRegionSampling.java#generational" time="2.887" timestamp="2026-02-17T02:11:56+0000">
    <properties>
        <property name="id" value="generational" />
        <property name="keywords" value="othervm" />
        <property name="requires" value="vm.gc.Shenandoah" />
        <property name="run" value="USER_SPECIFIED main/othervm -Xmx1g -Xms1g -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions -XX:+ShenandoahRegionSampling -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational TestRegionSampling
" />
        <property name="source" value="TestRegionSampling.java" />
        <property name="title" value=" " />
        <property name="description" value="file:/codebuild/output/src1098/src/s3/00/test/hotspot/jtreg/gc/shenandoah/TestRegionSampling.java#generational" />
        <property name="elapsed" value="2887 0:00:02.887" />
        <property name="end" value="Tue Feb 17 02:11:59 UTC 2026" />
        <property name="environment" value="regtest" />
        <property name="execStatus" value="Failed. Unexpected exit from test [exit code: 134]" />
        <property name="harnessLoaderMode" value="Classpath Loader" />
        <property name="harnessVariety" value="Full Bundle" />
        <property name="hostname" value="776afbe4ce00" />
        <property name="javatestOS" value="Linux 4.14.355-280.710.amzn2.x86_64 (amd64)" />
        <property name="javatestVersion" value="6.0-ea+b24-2026-02-12-${BUILT_FROM_COMMIT}" />
        <property name="jtregVersion" value="jtreg 8.3 dev 0" />
        <property name="script" value="com.sun.javatest.regtest.exec.RegressionScript" />
        <property name="sections" value="script_messages build compile main" />
        <property name="start" value="Tue Feb 17 02:11:56 UTC 2026" />
        <property name="test" value="gc/shenandoah/TestRegionSampling.java#generational" />
        <property name="testJDK" value="/codebuild/output/src1098/src/s3/00/images/jdk" />
        <property name="testJDK_OS" value="[name:Linux,arch:amd64,version:4.14.355-280.710.amzn2.x86_64,family:linux,simple_arch:x64,simple_version:4.14,processors:8,maxMemory:15032385536,maxSwap:0]" />
        <property name="testJDK_os.arch" value="amd64" />
        <property name="testJDK_os.name" value="Linux" />
        <property name="testJDK_os.version" value="4.14.355-280.710.amzn2.x86_64" />
        <property name="totalTime" value="2888" />
        <property name="user.name" value="root" />
        <property name="work" value="/codebuild/output/src1098/src/s3/00/jtreg/work/gc/shenandoah" />
    </properties>
    <testcase classname="gc/shenandoah/TestRegionSampling.java#generational" name="TestRegionSampling_generational" time="2.887" >
    <failure type="Failed.">
Unexpected exit from test [exit code: 134]

--- JVM Crash Details ---
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/codebuild/output/src2857/src/git-codecommit.us-west-2.amazonaws.com/v1/repos/openjdk-replica/src/hotspot/share/gc/shenandoah/heuristics/shenandoahGenerationalHeuristics.cpp:228), pid=4317, tid=4335
#  assert(collection_set->is_empty()) failed: Must be empty
#
# JRE version: OpenJDK Runtime Environment (27.0) (fastdebug build 27-root-simplify-aged-region-selection.1f9aefa5b03)
# Java VM: OpenJDK 64-Bit Server VM (fastdebug 27-root-simplify-aged-region-selection.1f9aefa5b03, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-amd64)
# Problematic frame:
# V  [libjvm.so+0x1a0c0d8]  ShenandoahGenerationalHeuristics::filter_regions(ShenandoahInPlacePromotionPlanner&, ShenandoahCollectionSet*)+0x5c8
#
# Core dump will be written. Default location: /codebuild/output/src1098/src/s3/00/jtreg/results/core.4317.%i (max size 107374182400 k). To ensure a full core dump, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /codebuild/output/src1098/src/s3/00/jtreg/results/java_error_4317.log
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
#
    
</failure>
    </testcase>
    <system-out>#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/codebuild/output/src2857/src/git-codecommit.us-west-2.amazonaws.com/v1/repos/openjdk-replica/src/hotspot/share/gc/shenandoah/heuristics/shenandoahGenerationalHeuristics.cpp:228), pid=4317, tid=4335
#  assert(collection_set->is_empty()) failed: Must be empty
#
# JRE version: OpenJDK Runtime Environment (27.0) (fastdebug build 27-root-simplify-aged-region-selection.1f9aefa5b03)
# Java VM: OpenJDK 64-Bit Server VM (fastdebug 27-root-simplify-aged-region-selection.1f9aefa5b03, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, shenandoah gc, linux-amd64)
# Problematic frame:
# V  [libjvm.so+0x1a0c0d8]  ShenandoahGenerationalHeuristics::filter_regions(ShenandoahInPlacePromotionPlanner&, ShenandoahCollectionSet*)+0x5c8
#
# Core dump will be written. Default location: /codebuild/output/src1098/src/s3/00/jtreg/results/core.4317.%i (max size 107374182400 k). To ensure a full core dump, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /codebuild/output/src1098/src/s3/00/jtreg/results/java_error_4317.log
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
#
    </system-out>
    <system-err>    </system-err>
</testsuite>


# JUnit XML Parser Compatibility Test Results

**Successfully Tested: 5/5**

Five parsers fully succeeded, one partially succeeded (ignores properties but doesn't fail):
- **junitparser** (Python, 4M+ downloads) - Perfect parsing
- **Open-MBEE junit-xml-parser** (Java, JAXB-based) - Perfect parsing
- **junit-parser** (Rust) - Perfect parsing
- **go-junit** (Go) - Perfect parsing
- **@techtrain/junit-parser** (Node.js) - Parses successfully but ignores properties

## Test File
`TestRegionSampling_generational.jtr.xml` - JTReg output with 29 properties

## Tested Parsers

### ✅ Python: junitparser (https://pypi.org/project/junitparser/)
**Status: SUCCESS**
- Parsed successfully
- Correctly identified: 1 test, 1 failure, 0 errors
- Correctly parsed all 29 properties
- Correctly extracted test case name and timing
- Correctly identified failure result

**Output:**

======================================================================
Testing: junitparser (https://pypi.org/project/junitparser/)
======================================================================
✓ Parsed successfully
  Suite: gc/shenandoah/TestRegionSampling.java#generational
  Tests: 1, Failures: 1, Errors: 0
  Properties: 29
  Test case: TestRegionSampling_generational, time: 2.887
    Result type: list


### ✅ Rust: junit-parser (https://github.com/borisfaure/junit-parser)
**Status: SUCCESS**
- Parsed successfully
- Correctly identified: 1 test, 1 failure, 0 errors
- Correctly parsed all 29 properties
- Correctly extracted 1 test case

**Output:**

======================================================================
Testing: junit-parser (Rust)
https://github.com/borisfaure/junit-parser
======================================================================
✓ Parsed successfully
  Test suites: 1
  Suite: gc/shenandoah/TestRegionSampling.java#generational
  Tests: 1, Failures: 1, Errors: 0
  Properties: 29
  Test cases: 1


### ✅ Go: go-junit (https://github.com/joshdk/go-junit)
**Status: SUCCESS**
- Parsed successfully
- Correctly identified: 1 test, 1 failure, 0 errors
- Correctly parsed all 29 properties
- Correctly extracted 1 test case

**Output:**

======================================================================
Testing: go-junit (https://github.com/joshdk/go-junit)
======================================================================
✓ Parsed successfully
  Suites: 1
  Suite: gc/shenandoah/TestRegionSampling.java#generational
  Tests: 1, Failures: 1, Errors: 0
  Properties: 29
  Test cases: 1


### ⚠️ Node.js: @techtrain/junit-parser (https://www.npmjs.com/package/@techtrain/junit-parser)
**Status: PARTIAL SUCCESS**
- Parsed successfully without errors
- Correctly identified: 1 test, 1 failure, 0 errors
- **Does NOT extract properties** - properties object is empty
- This parser ignores the `<properties>` section entirely

**Output:**

======================================================================
Testing: @techtrain/junit-parser (Node.js)
https://www.npmjs.com/package/@techtrain/junit-parser
======================================================================
✓ Parsed successfully
  Tests: 1, Failures: 1, Errors: 0
  Suites: 1
  Suite: gc/shenandoah/TestRegionSampling.java#generational
  Properties: 0 (⚠️  Parser does not extract properties)
  Test cases: 1


**Note:** While this parser doesn't extract properties, it also doesn't fail or error on the enhanced XML. The additional properties are simply ignored, demonstrating backward compatibility.

### ✅ Java: Open-MBEE junit-xml-parser (https://github.com/Open-MBEE/junit-xml-parser)
**Status: SUCCESS**
- Parsed successfully with Java 11+ and JAXB dependencies
- Correctly identified: 1 test, 1 failure, 0 errors
- Correctly parsed all 29 properties
- Correctly extracted 1 test case

**Output:**

======================================================================
Testing: Open-MBEE junit-xml-parser
https://github.com/Open-MBEE/junit-xml-parser
======================================================================
✓ Parsed successfully
  Suite: gc/shenandoah/TestRegionSampling.java#generational
  Tests: 1, Failures: 1, Errors: 0
  Properties: 29
  Test cases: 1

-------------

PR Review Comment: https://git.openjdk.org/jtreg/pull/316#discussion_r2818510139

More information about the jtreg-dev mailing list