From ihse at openjdk.org  Tue Jul  1 06:21:51 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Tue, 1 Jul 2025 06:21:51 GMT
Subject: RFR: 8361013: JMH does not export sufficient modules to benchmark
 ImageReader
In-Reply-To: <xGgBGln25JM3Us34SBVKvnoUs09vDqaDi3mTCduEse0=.91815218-6aea-4b60-a7ee-9d81b15fd1dd@github.com>
References: <xGgBGln25JM3Us34SBVKvnoUs09vDqaDi3mTCduEse0=.91815218-6aea-4b60-a7ee-9d81b15fd1dd@github.com>
Message-ID: <DmFUb0n8YFRT0tq33DNdUS3z-LSzjWxlW6ZsE2qqkqg=.b865ab34-4bc5-45dc-974d-411fe76f78fe@github.com>

On Sun, 29 Jun 2025 22:19:18 GMT, David Beaumont <duke at openjdk.org> wrote:

> Added jimage and jrtfs exports.

> FWIW I think these changes to the makefiles are routine enough at this point that we don't need to involve someone from the build team, but they can be approved by any reviewer.

I agree. Just adding addition `--exports` do not require a build system review. With that said, Skara will label any such changes with the `build` tag, so we will get notified. But you don't have to wait for a review from the build team. If I see such changes, I usually add a "build changes trivially fine" just to acknowledge this, but as I said, it's not really a requirement.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26035#issuecomment-3022005628

From ihse at openjdk.org  Tue Jul  1 06:22:46 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Tue, 1 Jul 2025 06:22:46 GMT
Subject: RFR: 8361076: Add benchmark for ImageReader in preparation for
 Valhalla changes [v3]
In-Reply-To: <EPq2BCssdOzDASLBXt1MIab0MbwCeUA1_n9brSSmWSI=.25614944-4553-4518-be89-189453ce7347@github.com>
References: <EMxYqM4tRcDPNaQ_xEpkBi2Zew0HlDwCFXEX0U7lrsY=.4552fbbf-7dd0-491d-b0f5-59ca304b970d@github.com>
 <EPq2BCssdOzDASLBXt1MIab0MbwCeUA1_n9brSSmWSI=.25614944-4553-4518-be89-189453ce7347@github.com>
Message-ID: <sEy5zJyZZgM7ioMx24RWCIlhFUgt29y0e-UwdPgScV8=.2ba693dc-0ce0-449b-8fdb-7583d465a1a1@github.com>

On Mon, 30 Jun 2025 15:08:25 GMT, David Beaumont <duke at openjdk.org> wrote:

>> Initial benchmark to capture at least some comparative measures of ImageReader performance.
>> 
>> Current results on my laptop:
>> 
>> Benchmark                                         Mode  Cnt   Score    Error  Units
>> NewImageBenchmark.warmCache_CountAllNodes         avgt    5   0.785 ?  0.140  ms/op
>> NewImageBenchmark.coldStart_CountOnly               ss    5  34.051 ? 17.662  ms/op
>> NewImageBenchmark.coldStart_InitAndCount            ss    5  31.006 ?  9.674  ms/op
>> NewImageBenchmark.coldStart_LoadJavacInitClasses    ss    5   3.752 ?  6.873  ms/op
>> 
>> The upcoming changes to ImageReader should show significant improvements to these results.
>
> David Beaumont has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Some tidying up relating to the state objects.

Build change trivially fine.

-------------

Marked as reviewed by ihse (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/26044#pullrequestreview-2973675262

From alanb at openjdk.org  Tue Jul  1 07:28:40 2025
From: alanb at openjdk.org (Alan Bateman)
Date: Tue, 1 Jul 2025 07:28:40 GMT
Subject: RFR: 8361076: Add benchmark for ImageReader in preparation for
 Valhalla changes [v3]
In-Reply-To: <EPq2BCssdOzDASLBXt1MIab0MbwCeUA1_n9brSSmWSI=.25614944-4553-4518-be89-189453ce7347@github.com>
References: <EMxYqM4tRcDPNaQ_xEpkBi2Zew0HlDwCFXEX0U7lrsY=.4552fbbf-7dd0-491d-b0f5-59ca304b970d@github.com>
 <EPq2BCssdOzDASLBXt1MIab0MbwCeUA1_n9brSSmWSI=.25614944-4553-4518-be89-189453ce7347@github.com>
Message-ID: <M2qvXA4FkPbt7awGXF1OyvFYP30fgDpvqws5Vm7jJwc=.6a1d2fd4-0b56-4b04-be2f-779bdbec35e6@github.com>

On Mon, 30 Jun 2025 15:08:25 GMT, David Beaumont <duke at openjdk.org> wrote:

>> Initial benchmark to capture at least some comparative measures of ImageReader performance.
>> 
>> Current results on my laptop:
>> 
>> Benchmark                                         Mode  Cnt   Score    Error  Units
>> NewImageBenchmark.warmCache_CountAllNodes         avgt    5   0.785 ?  0.140  ms/op
>> NewImageBenchmark.coldStart_CountOnly               ss    5  34.051 ? 17.662  ms/op
>> NewImageBenchmark.coldStart_InitAndCount            ss    5  31.006 ?  9.674  ms/op
>> NewImageBenchmark.coldStart_LoadJavacInitClasses    ss    5   3.752 ?  6.873  ms/op
>> 
>> The upcoming changes to ImageReader should show significant improvements to these results.
>
> David Beaumont has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Some tidying up relating to the state objects.

test/micro/org/openjdk/bench/jdk/internal/jrtfs/ImageReaderBenchmark.java line 230:

> 228:     // Created by running "java -verbose:class", throwing away anonymous inner
> 229:     // classes and anything without a reliable name, and grouping by the stated
> 230:     // source. It's not perfect, but it's representative.

I don't think this is maintainable. How useful (or not) is this benchmark if the names of all the internal classes (that will change from release to release) are dropped from this?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26044#discussion_r2176636106

From lucy at openjdk.org  Tue Jul  1 09:52:41 2025
From: lucy at openjdk.org (Lutz Schmidt)
Date: Tue, 1 Jul 2025 09:52:41 GMT
Subject: RFR: 8360791: [ubsan] Adjust signal handling [v2]
In-Reply-To: <IMzu0pj3_0YYbxmGyTD1SESnlcJr-ywkJU7K41hcNdA=.6d668bd3-ec77-4bd5-9086-8aa8fee45e62@github.com>
References: <_xiLynxVHloWK4ImZwxEuZ00xOOnPTIdLWPaIUkR6TM=.f9019726-6709-49dd-a6ea-3a98ab288f5c@github.com>
 <IMzu0pj3_0YYbxmGyTD1SESnlcJr-ywkJU7K41hcNdA=.6d668bd3-ec77-4bd5-9086-8aa8fee45e62@github.com>
Message-ID: <7FqJZOWwdy26ixPK6Mz964V91vG1Y1kkiNeKOffw1V0=.8a65fd39-0179-45f3-ab91-1cd5849263fb@github.com>

On Mon, 30 Jun 2025 07:02:31 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

>> A couple of tests e.g. VendorInfoPluginsTest but also some Hotspot like runtime/ErrorHandling/CreateCoredumpOnCrash.java put (write) to special addresses like 0 to provoke crashs.
>> However this does not work well with ubsan-enabled binaries on the clang based platforms (macOS, AIX).
>> The mentioned tests generate a SIGSEGV.
>> 
>> Some other tests generate a SIGBUS, e.g.
>> 
>> runtime/memory/ReserveMemory.java
>> runtime/memory/ReadFromNoaccessArea.java
>> 
>> and this leads to similar issues with ubsan-enabled binaries.
>> 
>> We should adjust the signal handling with the sanitizer options, how to do this is documented here :
>> https://github.com/google/sanitizers/wiki/SanitizerCommonFlags
>
> Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove the clang check

LGTM.
I agree: don't exclude things just because there is a vague suspicion.

-------------

Marked as reviewed by lucy (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/26015#pullrequestreview-2974602300

From mbaesken at openjdk.org  Tue Jul  1 09:58:50 2025
From: mbaesken at openjdk.org (Matthias Baesken)
Date: Tue, 1 Jul 2025 09:58:50 GMT
Subject: Integrated: 8360791: [ubsan] Adjust signal handling
In-Reply-To: <_xiLynxVHloWK4ImZwxEuZ00xOOnPTIdLWPaIUkR6TM=.f9019726-6709-49dd-a6ea-3a98ab288f5c@github.com>
References: <_xiLynxVHloWK4ImZwxEuZ00xOOnPTIdLWPaIUkR6TM=.f9019726-6709-49dd-a6ea-3a98ab288f5c@github.com>
Message-ID: <OwH97aGoBCJ9D1BBZTgM6zSflnFgHxrycySZ7fFCMMY=.0fdb7a3f-3ad2-46c9-a4a3-f489b85dea9b@github.com>

On Fri, 27 Jun 2025 09:30:19 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

> A couple of tests e.g. VendorInfoPluginsTest but also some Hotspot like runtime/ErrorHandling/CreateCoredumpOnCrash.java put (write) to special addresses like 0 to provoke crashs.
> However this does not work well with ubsan-enabled binaries on the clang based platforms (macOS, AIX).
> The mentioned tests generate a SIGSEGV.
> 
> Some other tests generate a SIGBUS, e.g.
> 
> runtime/memory/ReserveMemory.java
> runtime/memory/ReadFromNoaccessArea.java
> 
> and this leads to similar issues with ubsan-enabled binaries.
> 
> We should adjust the signal handling with the sanitizer options, how to do this is documented here :
> https://github.com/google/sanitizers/wiki/SanitizerCommonFlags

This pull request has now been integrated.

Changeset: aeca49e4
Author:    Matthias Baesken <mbaesken at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/aeca49e43fab951c2031895fee32703fb4a19524
Stats:     4 lines in 1 file changed: 3 ins; 0 del; 1 mod

8360791: [ubsan] Adjust signal handling

Reviewed-by: ihse, lucy

-------------

PR: https://git.openjdk.org/jdk/pull/26015

From mbaesken at openjdk.org  Tue Jul  1 09:58:49 2025
From: mbaesken at openjdk.org (Matthias Baesken)
Date: Tue, 1 Jul 2025 09:58:49 GMT
Subject: RFR: 8360791: [ubsan] Adjust signal handling [v2]
In-Reply-To: <IMzu0pj3_0YYbxmGyTD1SESnlcJr-ywkJU7K41hcNdA=.6d668bd3-ec77-4bd5-9086-8aa8fee45e62@github.com>
References: <_xiLynxVHloWK4ImZwxEuZ00xOOnPTIdLWPaIUkR6TM=.f9019726-6709-49dd-a6ea-3a98ab288f5c@github.com>
 <IMzu0pj3_0YYbxmGyTD1SESnlcJr-ywkJU7K41hcNdA=.6d668bd3-ec77-4bd5-9086-8aa8fee45e62@github.com>
Message-ID: <-9sTFThc2fmNldQ10SD2o0Q7su6MityEwdj_oLtE93A=.c3dba2c4-e597-4fd0-81ec-8d52e4ecd689@github.com>

On Mon, 30 Jun 2025 07:02:31 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

>> A couple of tests e.g. VendorInfoPluginsTest but also some Hotspot like runtime/ErrorHandling/CreateCoredumpOnCrash.java put (write) to special addresses like 0 to provoke crashs.
>> However this does not work well with ubsan-enabled binaries on the clang based platforms (macOS, AIX).
>> The mentioned tests generate a SIGSEGV.
>> 
>> Some other tests generate a SIGBUS, e.g.
>> 
>> runtime/memory/ReserveMemory.java
>> runtime/memory/ReadFromNoaccessArea.java
>> 
>> and this leads to similar issues with ubsan-enabled binaries.
>> 
>> We should adjust the signal handling with the sanitizer options, how to do this is documented here :
>> https://github.com/google/sanitizers/wiki/SanitizerCommonFlags
>
> Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove the clang check

Thanks for the reviews !

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26015#issuecomment-3023099250

From ihse at openjdk.org  Tue Jul  1 12:08:23 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Tue, 1 Jul 2025 12:08:23 GMT
Subject: RFR: 8361142: Improve custom hooks for makefiles
Message-ID: <91cuhK8a2VHENkKujTT5Q7CYxYjnK8qV3jULeLXi9jU=.d12fe818-ca41-4448-8a8f-120c60a219d7@github.com>

Add some additional and more fine-grained way in which custom makefiles can be integrated.

-------------

Commit messages:
 - Missed hook for cleaning
 - Turns out these were needed after all...
 - 8361142: Improve custom hooks for makefiles

Changes: https://git.openjdk.org/jdk/pull/26060/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=26060&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8361142
  Stats: 38 lines in 8 files changed: 24 ins; 0 del; 14 mod
  Patch: https://git.openjdk.org/jdk/pull/26060.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/26060/head:pull/26060

PR: https://git.openjdk.org/jdk/pull/26060

From ihse at openjdk.org  Tue Jul  1 13:24:41 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Tue, 1 Jul 2025 13:24:41 GMT
Subject: RFR: 8357979: [JVMCI] jdk.internal.vm.ci should have earlier class
 file version
In-Reply-To: <zBi4B-iJTA87Sg2NI5jhElh0BpVNtV7cEPOkQM9LJ-E=.9ced6224-7254-429c-b4f4-1eebd2f0de57@github.com>
References: <R0hJ349y-7LQd-n5h4qNo1Q3XM8wY5853YpwIAsVXpQ=.24cb515f-3277-4708-b05f-7ef60c188b07@github.com>
 <OIu1fg6FTKDl_Hv2YU_oNrzRfSYns6DCkOHZjsaRd-w=.15a69d94-88d9-4993-b9c4-eb32cf37117a@github.com>
 <zBi4B-iJTA87Sg2NI5jhElh0BpVNtV7cEPOkQM9LJ-E=.9ced6224-7254-429c-b4f4-1eebd2f0de57@github.com>
Message-ID: <zNabyTN96DxRz0Bzp5oobw4T9vYdA17Mz8wbDcwXwtc=.591eae5e-f77f-44b3-afb2-51b0604765d5@github.com>

On Mon, 2 Jun 2025 17:21:21 GMT, Joe Darcy <darcy at openjdk.org> wrote:

>> I don't think this is a good way. We should not call to shell to make calculations in the makefiles; if that were needed, it should be done in configure. However, we are already doing this, or at least something very similar. Have a look at line 398 in boot-jdk.m4. We set up `OLDEST_BOOT_JDK` as the oldest JDK from the list in version-numbers.conf. 
>> 
>> That should, I think, match your expectation of N-1. There is always a bit uncertainty how you want these things to be handled around the version rollover, and I'm not sure about your use case in those situations. But in general, the oldest boot JDK will match the version number actually used in the build as the boot JDK, that is, N-1.
>> 
>> Now, OLDEST_BOOT_JDK is not exported to make. But that is however:
>> 
>> BOOT_JDK_SOURCETARGET="-source $OLDEST_BOOT_JDK -target $OLDEST_BOOT_JDK -Xlint:-options"
>> 
>> which is almost, but not exactly, what you were planning to add. The added `-Xlint:-options` is harmless afaict, and I think was likely added at some point due to option mismatch between N and N-1 javac lint support, so I think it would make sense for you to use it as well.
>> 
>> So, in essense, my suggestion boils down to that you should only add `$(BOOT_JDK_SOURCETARGET)` to your command line.
>
>> I don't think this is a good way. We should not call to shell to make calculations in the makefiles; if that were needed, it should be done in configure. However, we are already doing this, or at least something very similar. Have a look at line 398 in boot-jdk.m4. We set up `OLDEST_BOOT_JDK` as the oldest JDK from the list in version-numbers.conf.
>> 
>> That should, I think, match your expectation of N-1. There is always a bit uncertainty how you want these things to be handled around the version rollover, and I'm not sure about your use case in those situations. But in general, the oldest boot JDK will match the version number actually used in the build as the boot JDK, that is, N-1.
>> 
>> Now, OLDEST_BOOT_JDK is not exported to make. But that is however:
>> 
>> ```
>> BOOT_JDK_SOURCETARGET="-source $OLDEST_BOOT_JDK -target $OLDEST_BOOT_JDK -Xlint:-options"
>> ```
>> 
>> which is almost, but not exactly, what you were planning to add. The added `-Xlint:-options` is harmless afaict, and I think was likely added at some point due to option mismatch between N and N-1 javac lint support, so I think it would make sense for you to use it as well.
>> 
>> So, in essense, my suggestion boils down to that you should only add `$(BOOT_JDK_SOURCETARGET)` to your command line.
> 
> I don't know the particulars here, but using
> 
> `javac -source $OLD -target $OLD ..."
> 
> is _almost_ never more appropriate than using
> 
> `javac --release $OLD ..."
> 
> The `options` warning would be noting the possible use of `--release`.

@jddarcy I tried replacing the `-source/-target` with `--release` for the Boot JDK overall. However, I get this kind of errors:


error: exporting a package from system module java.base is not allowed with --release


I assume this is why we have still been using  `-source/-target`. But maybe we should move to use `--release` instead, and figure out and fix this issue in a different way?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/25494#issuecomment-3024001740

From ihse at openjdk.org  Tue Jul  1 14:04:50 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Tue, 1 Jul 2025 14:04:50 GMT
Subject: RFR: 8357979: [JVMCI] jdk.internal.vm.ci should have earlier class
 file version [v2]
In-Reply-To: <fRD46zX3X5dSrMSp4bD-LEKQ-OIBid54IR-sBgiJg9c=.9f55038a-cdb2-439e-b95d-eeef814b1222@github.com>
References: <R0hJ349y-7LQd-n5h4qNo1Q3XM8wY5853YpwIAsVXpQ=.24cb515f-3277-4708-b05f-7ef60c188b07@github.com>
 <fRD46zX3X5dSrMSp4bD-LEKQ-OIBid54IR-sBgiJg9c=.9f55038a-cdb2-439e-b95d-eeef814b1222@github.com>
Message-ID: <lmIQQUd8Rh1s_Ji0GKbYSqMDQZeRCQp3ZwpD6Fz5HKc=.a63d90a5-2fe5-4827-9879-ba7fa172a803@github.com>

On Mon, 2 Jun 2025 17:00:32 GMT, Doug Simon <dnsimon at openjdk.org> wrote:

>> There are plans to have libgraal be built for JDK master using a version of Native Image running on a JDK one version behind the current JDK. This Native Image execution needs to be able to load the JVMCI classes. As such, the JVMCI classes must have a class file major version of N-1 where N is the major class file version of the current JDK.
>
> Doug Simon has updated the pull request incrementally with one additional commit since the last revision:
> 
>   use BOOT_JDK_SOURCETARGET instead

For another compilation, I got:

error: option --system cannot be used together with --release


I guess that even if the rule "-source/-target is almost never more appropriate than --release" applies to Java developers at large, building the JDK might be that small corner case where the "almost" actually applies.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/25494#issuecomment-3024161108

From duke at openjdk.org  Tue Jul  1 14:18:20 2025
From: duke at openjdk.org (David Beaumont)
Date: Tue, 1 Jul 2025 14:18:20 GMT
Subject: RFR: 8360037: Refactor ImageReader in preparation for Valhalla support
Message-ID: <VtQlJ_3w2k5ewTVyVele6LRy1j6Vp_N-gZ4K0WJEDzE=.9b4ec980-a8a1-4ed2-8da8-0a1e3e58098d@github.com>

Refactoring `ImageReader` to make it easy to add preview mode functionality for Valhalla.

This PR is a large change to `ImageReader` (effectively a rewrite) but reduces the surface area of the API significantly, reduces code complexity and increases performance/memory efficiency. The need for this sort of change comes from the need to support "preview mode" in Valhalla for classes loaded from core modules.

### Preview Mode

In the proposed preview mode support for Valhalla, a resource with the name `/modules/<module>/<path>` may come from one of two locations in the underlying jimage file; `/<module>/<path>` or `/<module>/META-INF/preview/<path>`. Furthermore, directories (represented as directory nodes via the API) will have a potentially different number of child nodes depending on whether preview mode is in effect, and some directories may only exist at all in preview mode.

Furthermore, the directories and symbolic link nodes in `/packages/...` will also be affected by the existence of new package directories. To retain consistency and avoid "surprises" later, all of this needs to be taken into account.

Below is a summary of the main changes for mainline JDK to better support preview mode later:

### 1: Improved encapsulation for preview mode

The current `ImageReader` code exposes the data from the jimage file in several ways; via the `Node` abstraction, but also with methods which return an `ImageLocation` instance directly. In preview mode it would not be acceptable to return the `ImageLocation`, since that would leak the existence of resources in the `META-INF/preview` namespace and lets users see resource nodes with names that don't match the underlying `ImageLocation` from which their contents come.

As such, the PR removes all methods which can leak `ImageLocation` or other "underlying" semantics about the jimage file. Luckily most of these are either used minimally and easily migrated to using nodes, or they were not being used at all. This PR also removes any methods which were already unused across the OpenJDK codebase (if I discover any issues with over-trimming of the API during full CI testing, it will be easy to address).

### 2. Simplification of directory child node handling

The current `ImageReader` code attempts to create parent directories "on demand" for any child nodes it creates. This results in parent directories having a non-empty but incomplete set of child nodes present. This is referred to as the directory being "incomplete" and requires users of the API to understand this and potentially "complete" these nodes by calling back into the `ImageReader` API.

This approach is an attempt to provide lazy node creation (which is a necessary trait since there are ~31,000 nodes in a typical jimage hierarchy) but isn't actually necessary since a child node has no "get parent" functionality and simply not creating parent nodes at the point is functionally equivalent.

The partially complete directory nodes also pose an issue for preview mode support, making it more complex to reason about where and when preview child nodes should be applied.

This PR simplifies directory node handling in two ways:
1. Directory nodes are either empty (incomplete) or have all child nodes present.
2. Directory nodes no longer have an API to return their (possibly incomplete) child nodes to users.

This results in simpler to reason about code for directory management and simpler changes later to support preview mode. Removing the on-demand parent directory creation and the partial child list management also has a noticeable performance improvement (based on the new `ImageReaderBenchmark` in pr/26044).

### 3. More comments and code assertions

Since `ImageReader` is a critical bit of internal tooling for OpenJDK, I made sure to add significant comments explaining the behaviour, as well as adding many new in-code assert statements. The version of `ImageReader` in this PR is 70 lines shorter than the current version, but if you account for new comments, it's really a reduction of almost 40% (640 -> 400 lines of code) and has over 200 more comment lines.

### 4. New tests

I added a new unit test for `ImageReader` since, until now, its API was not really being tested directly. These tests are useful now, but will really help when preview mode is added in Valhalla, since then there will need to be significant edge-case testing. I'm happy to improve or change tests in this PR, but they definitely cover the main cases.

### 5. Adding TODO comments for review discussion.

There are obviously some open questions about the exact design of the APIs, and some questions around behaviour. To make reviewing easier, I've added inline TODOs which are there to illicit feedback. I will account for an remove all of these before the PR is integrated, but I want reviewers to read and comment on them (even if there is no change expected).

### Performance Improvements

On my laptop (not objectively interesting, but good for comparison) I am seeing a significant performance improvement over all benchmarks and a reduction in timing variability. These benchmarks are awkward because of the need to test things in a "cold start" state, so timings will have larger variability than expected, but the performance improvements are consistent and non trivial:

Run with `make test TEST="micro:jdk.internal.jrtfs.ImageReaderBenchmark"` using the benchmark currently being added in `pr/26044`:

Current version:

Benchmark                                            Mode  Cnt   Score    Error  Units
ImageReaderBenchmark.warmCache_CountAllNodes         avgt    5   0.784 ?  0.023  ms/op
ImageReaderBenchmark.coldStart_CountOnly               ss    5  37.910 ? 28.554  ms/op
ImageReaderBenchmark.coldStart_InitAndCount            ss    5  37.471 ? 18.775  ms/op
ImageReaderBenchmark.coldStart_LoadJavacInitClasses    ss    5   4.279 ?  7.959  ms/op


This PR:

Benchmark                                            Mode  Cnt   Score   Error  Units
ImageReaderBenchmark.warmCache_CountAllNodes         avgt   10   0.880 ? 0.086  ms/op
ImageReaderBenchmark.coldStart_CountOnly               ss   10  14.303 ? 3.975  ms/op
ImageReaderBenchmark.coldStart_InitAndCount            ss   10  14.032 ? 3.068  ms/op
ImageReaderBenchmark.coldStart_LoadJavacInitClasses    ss   10   2.530 ? 0.348  ms/op


Shows a >2.5x speedup for traversing all nodes in a "cold start" state, and ~1.7x speedup for loading the core set of classes needed to start javac.

-------------

Depends on: https://git.openjdk.org/jdk/pull/26044

Commit messages:
 - Integrating with ImageReaderBenchmark.
 - Merge branch 'master' into jdk_8355953_reader/squashed
 - undo dependent changes
 - syncing to main branch
 - Newline at EOF.
 - Pretty much ready for review now.
 - WIP - more tests needed.
 - Reapply diffs.

Changes: https://git.openjdk.org/jdk/pull/26054/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=26054&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8360037
  Stats: 2217 lines in 11 files changed: 1613 ins; 342 del; 262 mod
  Patch: https://git.openjdk.org/jdk/pull/26054.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/26054/head:pull/26054

PR: https://git.openjdk.org/jdk/pull/26054

From duke at openjdk.org  Tue Jul  1 14:18:21 2025
From: duke at openjdk.org (David Beaumont)
Date: Tue, 1 Jul 2025 14:18:21 GMT
Subject: RFR: 8360037: Refactor ImageReader in preparation for Valhalla
 support
In-Reply-To: <VtQlJ_3w2k5ewTVyVele6LRy1j6Vp_N-gZ4K0WJEDzE=.9b4ec980-a8a1-4ed2-8da8-0a1e3e58098d@github.com>
References: <VtQlJ_3w2k5ewTVyVele6LRy1j6Vp_N-gZ4K0WJEDzE=.9b4ec980-a8a1-4ed2-8da8-0a1e3e58098d@github.com>
Message-ID: <9xh9VZuuTCyFomE_TV9usc_bVDSPz6ssX5HDmj-hhSU=.18a3192c-57b1-4920-b97b-4cfd37c6fea6@github.com>

On Mon, 30 Jun 2025 23:27:49 GMT, David Beaumont <duke at openjdk.org> wrote:

> Refactoring `ImageReader` to make it easy to add preview mode functionality for Valhalla.
> 
> This PR is a large change to `ImageReader` (effectively a rewrite) but reduces the surface area of the API significantly, reduces code complexity and increases performance/memory efficiency. The need for this sort of change comes from the need to support "preview mode" in Valhalla for classes loaded from core modules.
> 
> ### Preview Mode
> 
> In the proposed preview mode support for Valhalla, a resource with the name `/modules/<module>/<path>` may come from one of two locations in the underlying jimage file; `/<module>/<path>` or `/<module>/META-INF/preview/<path>`. Furthermore, directories (represented as directory nodes via the API) will have a potentially different number of child nodes depending on whether preview mode is in effect, and some directories may only exist at all in preview mode.
> 
> Furthermore, the directories and symbolic link nodes in `/packages/...` will also be affected by the existence of new package directories. To retain consistency and avoid "surprises" later, all of this needs to be taken into account.
> 
> Below is a summary of the main changes for mainline JDK to better support preview mode later:
> 
> ### 1: Improved encapsulation for preview mode
> 
> The current `ImageReader` code exposes the data from the jimage file in several ways; via the `Node` abstraction, but also with methods which return an `ImageLocation` instance directly. In preview mode it would not be acceptable to return the `ImageLocation`, since that would leak the existence of resources in the `META-INF/preview` namespace and lets users see resource nodes with names that don't match the underlying `ImageLocation` from which their contents come.
> 
> As such, the PR removes all methods which can leak `ImageLocation` or other "underlying" semantics about the jimage file. Luckily most of these are either used minimally and easily migrated to using nodes, or they were not being used at all. This PR also removes any methods which were already unused across the OpenJDK codebase (if I discover any issues with over-trimming of the API during full CI testing, it will be easy to address).
> 
> ### 2. Simplification of directory child node handling
> 
> The current `ImageReader` code attempts to create parent directories "on demand" for any child nodes it creates. This results in parent directories having a non-empty but incomplete set of child nodes present. This is referred to as the directory being "incomple...

Anything marked "TODO" is for discussion during the coming review. I don't intend to leave any of these in the code after, but they raise specific issues I'd like address.

With these pre-preview comments, I think this is finally ready for review.

make/test/BuildMicrobenchmark.gmk line 1:

> 1: #

Ignore this file, it's part of the PR to add the benchmark. I'll merge and sort everything out once that's in.

src/java.base/share/classes/jdk/internal/jimage/ImageReader.java line 1:

> 1: /*

I think that this class is best reviewed as if it were a new implementation, rather than trying to reason about the specific changes between the versions. Hopefully the comments will make the design clear and the reduced complexity/lines-of-code will help it be understood in a more stand-alone way.

src/java.base/share/classes/jdk/internal/jimage/ImageReader.java line 95:

> 93:     }
> 94: 
> 95:     // directory management interface

A large number of the methods below were either:
1. Unused or effectively no-ops.
2. Breaking encapsulation and leaking underlying types such as `ImageLocation` (not a big problem now, but prevents cleanly implementing the preview mode logic for Valhalla).

The new API has essentially 3 top level methods here:
1. findNode()
2. getResource(Node)
3. getResourceBuffer(Node)

Any navigation of the node hierarchy is done via `getChildNames()` and unlike now, no user can obtain a reference to an "incomplete" node.

src/java.base/share/classes/jdk/internal/jimage/ImageReader.java line 157:

> 155:     }
> 156: 
> 157:     /**

This method and the `getResourceBuffer()` method below are, I think, only called in one place and could, in theory, be factored out of this class altogether.

src/java.base/share/classes/jdk/internal/jimage/ImageReader.java line 255:

> 253:                 if (openers.isEmpty()) {
> 254:                     close();
> 255:                     // TODO (review note): Should this be synchronized by "this" ??

I genuinely this this might be an existing issue.

src/java.base/share/classes/jdk/internal/jimage/ImageReader.java line 601:

> 599:         }
> 600: 
> 601:         /**

Most of these were either never called or no longer needed in the new implementation.

src/java.base/share/classes/jdk/internal/jrtfs/ExplodedImage.java line 122:

> 120: 
> 121:         @Override
> 122:         public List<Node> getChildren() {

ExplodedImage makes a custom subclass of Node to "fake" the ImageReader behaviour. While this will need to change for preview mode, for now it's enough to just modify the one affected API.

src/java.base/share/classes/jdk/internal/jrtfs/JrtFileAttributes.java line 52:

> 50:     @Override
> 51:     public FileTime creationTime() {
> 52:         return node.creationTime();

There's no benefit to having all these one-line getter methods duplicated in `ImageReader` since it already provides the file attributes object directly.

src/java.base/share/classes/jdk/internal/jrtfs/JrtFileSystem.java line 228:

> 226:         }
> 227:         if (filter == null) {
> 228:             return node.getChildren()

Same logic, just based on the child names instead of the nodes.

src/java.base/share/classes/jdk/internal/jrtfs/SystemImage.java line 61:

> 59:             // open a .jimage and build directory structure
> 60:             final ImageReader image = ImageReader.open(moduleImageFile);
> 61:             image.getRootDirectory();

This method no longer serves any purpose (it used to initialize the root directory entries, but it's not needed now.

src/java.base/share/classes/jdk/internal/module/SystemModuleFinders.java line 224:

> 222:         ImageReader reader = SystemImage.reader();
> 223:         for (String mn : reader.getModuleNames()) {
> 224:             ImageLocation loc = reader.findLocation(mn, "module-info.class");

It is unsafe to let users see the underlying `ImageLocation` for a resource in preview mode, so this code had to be migrated to using a `Node` based view of the data. This is probably the most complex change not part of `ImageReader` itself.

src/java.base/share/classes/jdk/internal/module/SystemModuleFinders.java line 509:

> 507:                     if (node.isDirectory()) {
> 508:                         // build node
> 509:                         ImageReader.Node dir = SystemImage.reader().findNode(name);

This is an example of the fragility of the current "partially complete directory" behaviour. The `node` variable is exactly the same instance as the `dir` variable, and the call to `findNode()` serves only to ensure the directory is "complete" before use. In the new API only the name is available, so `findNode()` *must* be called, ensuring there's no risk of accidentally using an incomplete directory node.

test/jdk/jdk/internal/jimage/ImageReaderTest.java line 78:

> 76: 
> 77:     @Test
> 78:     public void testModuleDirectories() throws IOException {

I could make these more data driven (i.e. parametrized) if people want.

test/jdk/jdk/internal/jimage/ImageReaderTest.java line 217:

> 215: 
> 216:     /// Loads and performs actions on classes stored in a given `ImageReader`.
> 217:     private static class ImageClassLoader extends ClassLoader {

Loading and running class bytes to get a `toString()` value will be super important in the preview mode tests to prove that the correct class bytes were used.

test/jdk/jdk/internal/jimage/JImageReadTest.java line 28:

> 26:  * @summary Unit test for libjimage JIMAGE_Open/Read/Close
> 27:  * @modules java.base/jdk.internal.jimage
> 28:  * @run testng/othervm JImageReadTest

Looking back, I don't recall exactly why this was needed. I will see if it's still necessary, but I think it was.

test/jdk/tools/jimage/ImageReaderDuplicateChildNodesTest.java line 1:

> 1: /*

Just uses child names rather than nodes.

-------------

PR Review: https://git.openjdk.org/jdk/pull/26054#pullrequestreview-2973065956
PR Review: https://git.openjdk.org/jdk/pull/26054#pullrequestreview-2975530216
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177654333
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177702506
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2176142533
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177705221
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177707657
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177710965
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177658457
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177660750
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177664240
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177671307
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177676879
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177687414
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177691003
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177692842
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177694731
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177697691

From ihse at openjdk.org  Tue Jul  1 14:21:43 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Tue, 1 Jul 2025 14:21:43 GMT
Subject: RFR: 8346719: Add relaunchers to the static JDK image for missing
 executables [v3]
In-Reply-To: <CyUgfG09cnYSDOV9IHKDkY4UowM60ej_oKzYoMFKSbg=.f63b99b7-2ed7-4db7-b5a7-b03447a99553@github.com>
References: <GWKPi8D6xcR5Pxkszfod4keJeBQKxp6uRzLl3ObOpJo=.21c2951d-5928-425f-b39e-a5af8659fd05@github.com>
 <CyUgfG09cnYSDOV9IHKDkY4UowM60ej_oKzYoMFKSbg=.f63b99b7-2ed7-4db7-b5a7-b03447a99553@github.com>
Message-ID: <ml1xO8DEABmZgQreHjBi3BQWf0AD2BbQk6yX84j1tLs=.3d6d8b1c-e585-4d6d-a09f-6e21244c1b48@github.com>

On Fri, 27 Jun 2025 16:15:57 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

>> In the static JDK image, a single humongous java executable is generated, and no other launcher, such as javac. This makes it impossible to run our jtreg tests, which assume these are present.
>> 
>> The solution is fortunately simply: we just need to add a bunch of trivial launchers, which are thin wrappers that execute the main java binary, with the proper arguments. This will result in the same behavior as the normal dynamic launchers, only that we will need to take the detour of launching another process instead of calling directly into the JLI library.
>
> Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove problemlisting

Any reviewers on this?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/24380#issuecomment-3024250322

From alanb at openjdk.org  Tue Jul  1 14:35:45 2025
From: alanb at openjdk.org (Alan Bateman)
Date: Tue, 1 Jul 2025 14:35:45 GMT
Subject: RFR: 8360037: Refactor ImageReader in preparation for Valhalla
 support
In-Reply-To: <VtQlJ_3w2k5ewTVyVele6LRy1j6Vp_N-gZ4K0WJEDzE=.9b4ec980-a8a1-4ed2-8da8-0a1e3e58098d@github.com>
References: <VtQlJ_3w2k5ewTVyVele6LRy1j6Vp_N-gZ4K0WJEDzE=.9b4ec980-a8a1-4ed2-8da8-0a1e3e58098d@github.com>
Message-ID: <QPSnXfmDTCKVY47Oh3GplxNEWrpJdvuwXKxz9lvxKSY=.8e5f77ea-1055-4830-9003-cce673330897@github.com>

On Mon, 30 Jun 2025 23:27:49 GMT, David Beaumont <duke at openjdk.org> wrote:

> Refactoring `ImageReader` to make it easy to add preview mode functionality for Valhalla.
> 
> This PR is a large change to `ImageReader` (effectively a rewrite) but reduces the surface area of the API significantly, reduces code complexity and increases performance/memory efficiency. The need for this sort of change comes from the need to support "preview mode" in Valhalla for classes loaded from core modules.
> 
> ### Preview Mode
> 
> In the proposed preview mode support for Valhalla, a resource with the name `/modules/<module>/<path>` may come from one of two locations in the underlying jimage file; `/<module>/<path>` or `/<module>/META-INF/preview/<path>`. Furthermore, directories (represented as directory nodes via the API) will have a potentially different number of child nodes depending on whether preview mode is in effect, and some directories may only exist at all in preview mode.
> 
> Furthermore, the directories and symbolic link nodes in `/packages/...` will also be affected by the existence of new package directories. To retain consistency and avoid "surprises" later, all of this needs to be taken into account.
> 
> Below is a summary of the main changes for mainline JDK to better support preview mode later:
> 
> ### 1: Improved encapsulation for preview mode
> 
> The current `ImageReader` code exposes the data from the jimage file in several ways; via the `Node` abstraction, but also with methods which return an `ImageLocation` instance directly. In preview mode it would not be acceptable to return the `ImageLocation`, since that would leak the existence of resources in the `META-INF/preview` namespace and lets users see resource nodes with names that don't match the underlying `ImageLocation` from which their contents come.
> 
> As such, the PR removes all methods which can leak `ImageLocation` or other "underlying" semantics about the jimage file. Luckily most of these are either used minimally and easily migrated to using nodes, or they were not being used at all. This PR also removes any methods which were already unused across the OpenJDK codebase (if I discover any issues with over-trimming of the API during full CI testing, it will be easy to address).
> 
> ### 2. Simplification of directory child node handling
> 
> The current `ImageReader` code attempts to create parent directories "on demand" for any child nodes it creates. This results in parent directories having a non-empty but incomplete set of child nodes present. This is referred to as the directory being "incomple...

src/java.base/share/classes/jdk/internal/module/SystemModuleFinders.java line 62:

> 60: import jdk.internal.module.ModuleHashes.HashSupplier;
> 61: 
> 62: import static java.util.Objects.requireNonNull;

The existing Objects.requireNonNull are okay, no need to change them all as it is nothing to do with the changes.

src/java.base/share/classes/jdk/internal/module/SystemModuleFinders.java line 251:

> 249:     }
> 250: 
> 251:     private static Stream<ModuleInfo.Attributes> getModuleAttributes() {

Can you rename this to allModuleAttributes and add a method description to match the other methods.

src/java.base/share/classes/jdk/internal/module/SystemModuleFinders.java line 257:

> 255:                 .getChildNames()
> 256:                 .map(mn -> getNode(reader, mn + "/module-info.class"))
> 257:                 // This fails with ISE if the node isn't a resource (corrupt JImage).

Can you drop this comment and check getNode to thrown an Error (ISE isn't right when we have a broken image).

test/micro/org/openjdk/bench/jdk/internal/jrtfs/ImageReaderBenchmark.java line 60:

> 58: @OutputTimeUnit(TimeUnit.MILLISECONDS)
> 59: @Fork(value = 1, jvmArgs = {"--add-exports", "java.base/jdk.internal.jimage=ALL-UNNAMED"})
> 60: public class ImageReaderBenchmark {

This is the benchmark from the other PR, did you mean to include it?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177761299
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177768203
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177772042
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177762265

From duke at openjdk.org  Tue Jul  1 14:40:44 2025
From: duke at openjdk.org (ExE Boss)
Date: Tue, 1 Jul 2025 14:40:44 GMT
Subject: RFR: 8360037: Refactor ImageReader in preparation for Valhalla
 support
In-Reply-To: <VtQlJ_3w2k5ewTVyVele6LRy1j6Vp_N-gZ4K0WJEDzE=.9b4ec980-a8a1-4ed2-8da8-0a1e3e58098d@github.com>
References: <VtQlJ_3w2k5ewTVyVele6LRy1j6Vp_N-gZ4K0WJEDzE=.9b4ec980-a8a1-4ed2-8da8-0a1e3e58098d@github.com>
Message-ID: <wWmD8znj-bwA46ccmxwzDfdHIjLb0Ls3irrN6-o5Scg=.ffc3e9f3-fdc8-4802-b0df-e99d168e6f25@github.com>

On Mon, 30 Jun 2025 23:27:49 GMT, David Beaumont <duke at openjdk.org> wrote:

> Refactoring `ImageReader` to make it easy to add preview mode functionality for Valhalla.
> 
> This PR is a large change to `ImageReader` (effectively a rewrite) but reduces the surface area of the API significantly, reduces code complexity and increases performance/memory efficiency. The need for this sort of change comes from the need to support "preview mode" in Valhalla for classes loaded from core modules.
> 
> ### Preview Mode
> 
> In the proposed preview mode support for Valhalla, a resource with the name `/modules/<module>/<path>` may come from one of two locations in the underlying jimage file; `/<module>/<path>` or `/<module>/META-INF/preview/<path>`. Furthermore, directories (represented as directory nodes via the API) will have a potentially different number of child nodes depending on whether preview mode is in effect, and some directories may only exist at all in preview mode.
> 
> Furthermore, the directories and symbolic link nodes in `/packages/...` will also be affected by the existence of new package directories. To retain consistency and avoid "surprises" later, all of this needs to be taken into account.
> 
> Below is a summary of the main changes for mainline JDK to better support preview mode later:
> 
> ### 1: Improved encapsulation for preview mode
> 
> The current `ImageReader` code exposes the data from the jimage file in several ways; via the `Node` abstraction, but also with methods which return an `ImageLocation` instance directly. In preview mode it would not be acceptable to return the `ImageLocation`, since that would leak the existence of resources in the `META-INF/preview` namespace and lets users see resource nodes with names that don't match the underlying `ImageLocation` from which their contents come.
> 
> As such, the PR removes all methods which can leak `ImageLocation` or other "underlying" semantics about the jimage file. Luckily most of these are either used minimally and easily migrated to using nodes, or they were not being used at all. This PR also removes any methods which were already unused across the OpenJDK codebase (if I discover any issues with over-trimming of the API during full CI testing, it will be easy to address).
> 
> ### 2. Simplification of directory child node handling
> 
> The current `ImageReader` code attempts to create parent directories "on demand" for any child nodes it creates. This results in parent directories having a non-empty but incomplete set of child nodes present. This is referred to as the directory being "incomple...

src/java.base/share/classes/jdk/internal/jimage/ImageReader.java line 207:

> 205:             this.nodes = new HashMap<>();
> 206:             // TODO (review note): These should exist under all circumstances, but there's
> 207:             //  probably a more robust way of getting it these offsets.

Suggestion:

            //  probably a more robust way of getting these offsets.

src/java.base/share/classes/jdk/internal/jimage/ImageReader.java line 211:

> 209:             this.packagesStringOffset = findLocation("/packages/java.lang").getModuleOffset();
> 210: 
> 211:             // Node creation is very lazy, se can just make the top-level directories

Typo:
Suggestion:

            // Node creation is very lazy, so can just make the top-level directories

src/java.base/share/classes/jdk/internal/jimage/ImageReader.java line 544:

> 542: 
> 543:         // A node is completed when all its direct children have been built. As
> 544:         // such, non-directory nodes are always complete.

I?think this?might flow?better:
Suggestion:

        // A node is completed when all its direct children have been built.
        // As such, non-directory nodes are always complete.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177769091
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177768332
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177779216

From alanb at openjdk.org  Tue Jul  1 14:40:45 2025
From: alanb at openjdk.org (Alan Bateman)
Date: Tue, 1 Jul 2025 14:40:45 GMT
Subject: RFR: 8360037: Refactor ImageReader in preparation for Valhalla
 support
In-Reply-To: <VtQlJ_3w2k5ewTVyVele6LRy1j6Vp_N-gZ4K0WJEDzE=.9b4ec980-a8a1-4ed2-8da8-0a1e3e58098d@github.com>
References: <VtQlJ_3w2k5ewTVyVele6LRy1j6Vp_N-gZ4K0WJEDzE=.9b4ec980-a8a1-4ed2-8da8-0a1e3e58098d@github.com>
Message-ID: <CdlBFucDBOCgehmRdZSlhW14zhZ4VdLQ8kU3ecHXqpI=.a3a94675-f4c4-4614-b665-7466f3d31345@github.com>

On Mon, 30 Jun 2025 23:27:49 GMT, David Beaumont <duke at openjdk.org> wrote:

> Refactoring `ImageReader` to make it easy to add preview mode functionality for Valhalla.
> 
> This PR is a large change to `ImageReader` (effectively a rewrite) but reduces the surface area of the API significantly, reduces code complexity and increases performance/memory efficiency. The need for this sort of change comes from the need to support "preview mode" in Valhalla for classes loaded from core modules.
> 
> ### Preview Mode
> 
> In the proposed preview mode support for Valhalla, a resource with the name `/modules/<module>/<path>` may come from one of two locations in the underlying jimage file; `/<module>/<path>` or `/<module>/META-INF/preview/<path>`. Furthermore, directories (represented as directory nodes via the API) will have a potentially different number of child nodes depending on whether preview mode is in effect, and some directories may only exist at all in preview mode.
> 
> Furthermore, the directories and symbolic link nodes in `/packages/...` will also be affected by the existence of new package directories. To retain consistency and avoid "surprises" later, all of this needs to be taken into account.
> 
> Below is a summary of the main changes for mainline JDK to better support preview mode later:
> 
> ### 1: Improved encapsulation for preview mode
> 
> The current `ImageReader` code exposes the data from the jimage file in several ways; via the `Node` abstraction, but also with methods which return an `ImageLocation` instance directly. In preview mode it would not be acceptable to return the `ImageLocation`, since that would leak the existence of resources in the `META-INF/preview` namespace and lets users see resource nodes with names that don't match the underlying `ImageLocation` from which their contents come.
> 
> As such, the PR removes all methods which can leak `ImageLocation` or other "underlying" semantics about the jimage file. Luckily most of these are either used minimally and easily migrated to using nodes, or they were not being used at all. This PR also removes any methods which were already unused across the OpenJDK codebase (if I discover any issues with over-trimming of the API during full CI testing, it will be easy to address).
> 
> ### 2. Simplification of directory child node handling
> 
> The current `ImageReader` code attempts to create parent directories "on demand" for any child nodes it creates. This results in parent directories having a non-empty but incomplete set of child nodes present. This is referred to as the directory being "incomple...

src/java.base/share/classes/jdk/internal/module/SystemModuleFinders.java line 456:

> 454:             Optional<ImageReader.Node> node = Optional.ofNullable(reader.findNode(nodeName));
> 455:             if (node.isPresent() && !node.get().isResource()) {
> 456:                 throw new IllegalStateException("Not a resource node: " + node.get());

The IllegalStateException is problematic here, maybe you mean to add an assert?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2177784419

From duke at openjdk.org  Tue Jul  1 15:09:12 2025
From: duke at openjdk.org (Anton Artemov)
Date: Tue, 1 Jul 2025 15:09:12 GMT
Subject: RFR: 8320353: Reenable stringop-overflow warnings
Message-ID: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>

Hi, please consider the following changes:

this PR addresses the issue of stringop-overflow warnings produced by GCC. The compiler does think that the thread pointer returned by `JavaThread::current()` can be null, though it cant. The thread pointer ends up being an argument in `__atomic_load`, and the compiler reports the warning related to argument of that method. 

The patch adds a hint to the compiler by means of `__builtin_unreachable()` intrinsic, which tells the compiler that certain piece of code will never be reached (case of thread pointer being null). This solves the issue. 

Tested in tiers 1-3 and GHA.

-------------

Commit messages:
 - 8320353: Fixed comment
 - 8320353: Fixed whitespace error
 - 8320353:  Fiixed build problem
 - 8320353: Added debug only case
 - 8320353: More work
 - 8320353: More work
 - 8320353: Reenabled stringop-overflow warning for linux zero/fastdebug

Changes: https://git.openjdk.org/jdk/pull/26067/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=26067&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8320353
  Stats: 25 lines in 3 files changed: 20 ins; 4 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/26067.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/26067/head:pull/26067

PR: https://git.openjdk.org/jdk/pull/26067

From duke at openjdk.org  Tue Jul  1 16:52:40 2025
From: duke at openjdk.org (David Beaumont)
Date: Tue, 1 Jul 2025 16:52:40 GMT
Subject: RFR: 8361076: Add benchmark for ImageReader in preparation for
 Valhalla changes [v3]
In-Reply-To: <M2qvXA4FkPbt7awGXF1OyvFYP30fgDpvqws5Vm7jJwc=.6a1d2fd4-0b56-4b04-be2f-779bdbec35e6@github.com>
References: <EMxYqM4tRcDPNaQ_xEpkBi2Zew0HlDwCFXEX0U7lrsY=.4552fbbf-7dd0-491d-b0f5-59ca304b970d@github.com>
 <EPq2BCssdOzDASLBXt1MIab0MbwCeUA1_n9brSSmWSI=.25614944-4553-4518-be89-189453ce7347@github.com>
 <M2qvXA4FkPbt7awGXF1OyvFYP30fgDpvqws5Vm7jJwc=.6a1d2fd4-0b56-4b04-be2f-779bdbec35e6@github.com>
Message-ID: <E4dW0zQJNkVozaAP9rjlBDv-wTEtRKMl0OotVJY-9Tg=.d96625fd-17be-4418-96ad-6de5746ea9b7@github.com>

On Tue, 1 Jul 2025 07:25:35 GMT, Alan Bateman <alanb at openjdk.org> wrote:

>> David Beaumont has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Some tidying up relating to the state objects.
>
> test/micro/org/openjdk/bench/jdk/internal/jrtfs/ImageReaderBenchmark.java line 230:
> 
>> 228:     // Created by running "java -verbose:class", throwing away anonymous inner
>> 229:     // classes and anything without a reliable name, and grouping by the stated
>> 230:     // source. It's not perfect, but it's representative.
> 
> I don't think this is maintainable. How useful (or not) is this benchmark if the names of all the internal classes (that will change from release to release) are dropped from this?

Debatable. It's obviously going to scale any results somewhat based on the size of the resources and number of classes. It's kind nice to see "this change removes at least N micro/milli seconds of time spent" since that's a minimum set of classes we expect to always be needed, so any time saved is a lower bound.

I'd say maybe leave it as is for now with a note saying "if this keeps breaking, make the list less fragile".
I'm also assuming this is only run manually, and not a part of any CI pipeline ... so please let me know if I'm wrong about that.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26044#discussion_r2178080151

From duke at openjdk.org  Tue Jul  1 17:02:27 2025
From: duke at openjdk.org (David Beaumont)
Date: Tue, 1 Jul 2025 17:02:27 GMT
Subject: RFR: 8361076: Add benchmark for ImageReader in preparation for
 Valhalla changes [v4]
In-Reply-To: <EMxYqM4tRcDPNaQ_xEpkBi2Zew0HlDwCFXEX0U7lrsY=.4552fbbf-7dd0-491d-b0f5-59ca304b970d@github.com>
References: <EMxYqM4tRcDPNaQ_xEpkBi2Zew0HlDwCFXEX0U7lrsY=.4552fbbf-7dd0-491d-b0f5-59ca304b970d@github.com>
Message-ID: <HL9e08KU4wHaANdRVoesH2BLEHKh4BYZldySxG21WT4=.aa87928e-775d-4d7e-8489-300b1abddbf3@github.com>

> Initial benchmark to capture at least some comparative measures of ImageReader performance.
> 
> Current results on my laptop:
> 
> Benchmark                                         Mode  Cnt   Score    Error  Units
> NewImageBenchmark.warmCache_CountAllNodes         avgt    5   0.785 ?  0.140  ms/op
> NewImageBenchmark.coldStart_CountOnly               ss    5  34.051 ? 17.662  ms/op
> NewImageBenchmark.coldStart_InitAndCount            ss    5  31.006 ?  9.674  ms/op
> NewImageBenchmark.coldStart_LoadJavacInitClasses    ss    5   3.752 ?  6.873  ms/op
> 
> The upcoming changes to ImageReader should show significant improvements to these results.

David Beaumont has updated the pull request incrementally with one additional commit since the last revision:

  Adding comment about maintainability.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/26044/files
  - new: https://git.openjdk.org/jdk/pull/26044/files/089b68e2..965abec0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=26044&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=26044&range=02-03

  Stats: 7 lines in 1 file changed: 7 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/26044.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/26044/head:pull/26044

PR: https://git.openjdk.org/jdk/pull/26044

From ihse at openjdk.org  Tue Jul  1 17:31:49 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Tue, 1 Jul 2025 17:31:49 GMT
Subject: RFR: 8357979: Compile jdk.internal.vm.ci targeting the Boot JDK
 version
Message-ID: <2CLx0WU6ZAmW7ewzkvL_Lq6ELqc_AvibM9bWZjc8OcY=.86e6519f-3153-45f3-89cf-df30c8a1de29@github.com>

>From the bug description:

There are plans to build libgraal in JDK master using a version of Native Image running on a JDK one version behind JDK master. This Native Image execution needs to be able to load the JVMCI classes as they are built into the libgraal image. As such, the JVMCI classes must have a class file major version of N-1 where N is the major class file version of JDK master. 

This PR is basically the same change as published by Doug in https://github.com/openjdk/jdk/pull/25494, but it contains some cleanup and adds and uses the `TARGET_RELEASE` argument that Doug correctly realized was needed.

-------------

Commit messages:
 - 8357979: Compile jdk.internal.vm.ci targeting the Boot JDK version

Changes: https://git.openjdk.org/jdk/pull/26069/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=26069&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8357979
  Stats: 12 lines in 5 files changed: 4 ins; 3 del; 5 mod
  Patch: https://git.openjdk.org/jdk/pull/26069.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/26069/head:pull/26069

PR: https://git.openjdk.org/jdk/pull/26069

From rriggs at openjdk.org  Tue Jul  1 18:57:44 2025
From: rriggs at openjdk.org (Roger Riggs)
Date: Tue, 1 Jul 2025 18:57:44 GMT
Subject: RFR: 8360037: Refactor ImageReader in preparation for Valhalla
 support
In-Reply-To: <VtQlJ_3w2k5ewTVyVele6LRy1j6Vp_N-gZ4K0WJEDzE=.9b4ec980-a8a1-4ed2-8da8-0a1e3e58098d@github.com>
References: <VtQlJ_3w2k5ewTVyVele6LRy1j6Vp_N-gZ4K0WJEDzE=.9b4ec980-a8a1-4ed2-8da8-0a1e3e58098d@github.com>
Message-ID: <nnLiULvlgT9T3NE_bVbt1wOrepPdbNtHfy20P5UtPLE=.8c7c120e-fb4e-49c8-8c39-80506f3b8980@github.com>

On Mon, 30 Jun 2025 23:27:49 GMT, David Beaumont <duke at openjdk.org> wrote:

> Refactoring `ImageReader` to make it easy to add preview mode functionality for Valhalla.
> 
> This PR is a large change to `ImageReader` (effectively a rewrite) but reduces the surface area of the API significantly, reduces code complexity and increases performance/memory efficiency. The need for this sort of change comes from the need to support "preview mode" in Valhalla for classes loaded from core modules.
> 
> ### Preview Mode
> 
> In the proposed preview mode support for Valhalla, a resource with the name `/modules/<module>/<path>` may come from one of two locations in the underlying jimage file; `/<module>/<path>` or `/<module>/META-INF/preview/<path>`. Furthermore, directories (represented as directory nodes via the API) will have a potentially different number of child nodes depending on whether preview mode is in effect, and some directories may only exist at all in preview mode.
> 
> Furthermore, the directories and symbolic link nodes in `/packages/...` will also be affected by the existence of new package directories. To retain consistency and avoid "surprises" later, all of this needs to be taken into account.
> 
> Below is a summary of the main changes for mainline JDK to better support preview mode later:
> 
> ### 1: Improved encapsulation for preview mode
> 
> The current `ImageReader` code exposes the data from the jimage file in several ways; via the `Node` abstraction, but also with methods which return an `ImageLocation` instance directly. In preview mode it would not be acceptable to return the `ImageLocation`, since that would leak the existence of resources in the `META-INF/preview` namespace and lets users see resource nodes with names that don't match the underlying `ImageLocation` from which their contents come.
> 
> As such, the PR removes all methods which can leak `ImageLocation` or other "underlying" semantics about the jimage file. Luckily most of these are either used minimally and easily migrated to using nodes, or they were not being used at all. This PR also removes any methods which were already unused across the OpenJDK codebase (if I discover any issues with over-trimming of the API during full CI testing, it will be easy to address).
> 
> ### 2. Simplification of directory child node handling
> 
> The current `ImageReader` code attempts to create parent directories "on demand" for any child nodes it creates. This results in parent directories having a non-empty but incomplete set of child nodes present. This is referred to as the directory being "incomple...

src/java.base/share/classes/jdk/internal/jimage/ImageReader.java line 555:

> 553: 
> 554:         /**
> 555:          * Returns the JRY file system compatible name of this node (e.g.

Typo:  "JRY"

src/java.base/share/classes/jdk/internal/module/SystemModuleFinders.java line 292:

> 290:         @Override
> 291:         public Optional<ModuleReference> find(String name) {
> 292:             requireNonNull(name);

Personal preference is avoid static imports making the code readable by retaining the "Objects." qualifier otherwise the non-local method call appears out of thin air.  YMMV.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2178300589
PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2178311050

From rriggs at openjdk.org  Tue Jul  1 18:57:45 2025
From: rriggs at openjdk.org (Roger Riggs)
Date: Tue, 1 Jul 2025 18:57:45 GMT
Subject: RFR: 8360037: Refactor ImageReader in preparation for Valhalla
 support
In-Reply-To: <9xh9VZuuTCyFomE_TV9usc_bVDSPz6ssX5HDmj-hhSU=.18a3192c-57b1-4920-b97b-4cfd37c6fea6@github.com>
References: <VtQlJ_3w2k5ewTVyVele6LRy1j6Vp_N-gZ4K0WJEDzE=.9b4ec980-a8a1-4ed2-8da8-0a1e3e58098d@github.com>
 <9xh9VZuuTCyFomE_TV9usc_bVDSPz6ssX5HDmj-hhSU=.18a3192c-57b1-4920-b97b-4cfd37c6fea6@github.com>
Message-ID: <7jq-fF6TKn2LVG7na2cKpZ_uR-c-bSZe1xhqt3vaXa4=.397585d5-c923-42de-9431-8a23645df224@github.com>

On Tue, 1 Jul 2025 14:02:11 GMT, David Beaumont <duke at openjdk.org> wrote:

>> Refactoring `ImageReader` to make it easy to add preview mode functionality for Valhalla.
>> 
>> This PR is a large change to `ImageReader` (effectively a rewrite) but reduces the surface area of the API significantly, reduces code complexity and increases performance/memory efficiency. The need for this sort of change comes from the need to support "preview mode" in Valhalla for classes loaded from core modules.
>> 
>> ### Preview Mode
>> 
>> In the proposed preview mode support for Valhalla, a resource with the name `/modules/<module>/<path>` may come from one of two locations in the underlying jimage file; `/<module>/<path>` or `/<module>/META-INF/preview/<path>`. Furthermore, directories (represented as directory nodes via the API) will have a potentially different number of child nodes depending on whether preview mode is in effect, and some directories may only exist at all in preview mode.
>> 
>> Furthermore, the directories and symbolic link nodes in `/packages/...` will also be affected by the existence of new package directories. To retain consistency and avoid "surprises" later, all of this needs to be taken into account.
>> 
>> Below is a summary of the main changes for mainline JDK to better support preview mode later:
>> 
>> ### 1: Improved encapsulation for preview mode
>> 
>> The current `ImageReader` code exposes the data from the jimage file in several ways; via the `Node` abstraction, but also with methods which return an `ImageLocation` instance directly. In preview mode it would not be acceptable to return the `ImageLocation`, since that would leak the existence of resources in the `META-INF/preview` namespace and lets users see resource nodes with names that don't match the underlying `ImageLocation` from which their contents come.
>> 
>> As such, the PR removes all methods which can leak `ImageLocation` or other "underlying" semantics about the jimage file. Luckily most of these are either used minimally and easily migrated to using nodes, or they were not being used at all. This PR also removes any methods which were already unused across the OpenJDK codebase (if I discover any issues with over-trimming of the API during full CI testing, it will be easy to address).
>> 
>> ### 2. Simplification of directory child node handling
>> 
>> The current `ImageReader` code attempts to create parent directories "on demand" for any child nodes it creates. This results in parent directories having a non-empty but incomplete set of child nodes present. This is re...
>
> test/jdk/jdk/internal/jimage/ImageReaderTest.java line 78:
> 
>> 76: 
>> 77:     @Test
>> 78:     public void testModuleDirectories() throws IOException {
> 
> I could make these more data driven (i.e. parametrized) if people want.

yes, please, it will make failures easier to identity.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26054#discussion_r2178321323

From ihse at openjdk.org  Tue Jul  1 19:04:37 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Tue, 1 Jul 2025 19:04:37 GMT
Subject: RFR: 8320353: Reenable stringop-overflow warnings
In-Reply-To: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
References: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
Message-ID: <bxWyyE5VCVY0EhzEwu3fgOwoK1g1eH-1ar3VEE2RbHw=.30e71e59-041e-4280-8b11-ce40bf154990@github.com>

On Tue, 1 Jul 2025 12:25:04 GMT, Anton Artemov <duke at openjdk.org> wrote:

> Hi, please consider the following changes:
> 
> this PR addresses the issue of stringop-overflow warnings produced by GCC. The compiler does think that the thread pointer returned by `JavaThread::current()` can be null, though it cant. The thread pointer ends up being an argument in `__atomic_load`, and the compiler reports the warning related to argument of that method. 
> 
> The patch adds a hint to the compiler by means of `__builtin_unreachable()` intrinsic, which tells the compiler that certain piece of code will never be reached (case of thread pointer being null). This solves the issue. 
> 
> Tested in tiers 1-3 and GHA.

Build changes look good. Someone else will have to review the hotspot changes.

-------------

Marked as reviewed by ihse (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/26067#pullrequestreview-2976656855

From bpb at openjdk.org  Tue Jul  1 23:26:42 2025
From: bpb at openjdk.org (Brian Burkhalter)
Date: Tue, 1 Jul 2025 23:26:42 GMT
Subject: RFR: 8337143: (fc, fs) Move filesystem-related native objects from
 libnio to libjava [v15]
In-Reply-To: <TNlAuJBeBcTH1PE3w2bgVm-OFNoQcN84ICcym6yX3hs=.b134d04f-caff-49bf-87f1-de893bbcd6ef@github.com>
References: <TNlAuJBeBcTH1PE3w2bgVm-OFNoQcN84ICcym6yX3hs=.b134d04f-caff-49bf-87f1-de893bbcd6ef@github.com>
Message-ID: <ufWpflFhdrPm4cpC-J2_5GYBXqeZoNnOd7NWZMLCc5U=.e2abe1f8-1f0c-4e54-8fbc-9c493cd99887@github.com>

> This proposed change would move the native objects required for NIO file interaction from the libnio native library to the libjava native library on Linux, macOS, and Windows.

Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 15 commits:

 - Merge
 - Merge
 - Merge
 - Merge
 - Merge
 - Merge
 - Merge
 - 8337143: Minor makefile tweak
 - 8337143: Clean up to address reviewer comments
 - Merge
 - ... and 5 more: https://git.openjdk.org/jdk/compare/9d518b32...ffa3c469

-------------

Changes: https://git.openjdk.org/jdk/pull/20317/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=20317&range=14
  Stats: 1539 lines in 93 files changed: 774 ins; 668 del; 97 mod
  Patch: https://git.openjdk.org/jdk/pull/20317.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20317/head:pull/20317

PR: https://git.openjdk.org/jdk/pull/20317

From dholmes at openjdk.org  Wed Jul  2 06:57:40 2025
From: dholmes at openjdk.org (David Holmes)
Date: Wed, 2 Jul 2025 06:57:40 GMT
Subject: RFR: 8320353: Reenable stringop-overflow warnings
In-Reply-To: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
References: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
Message-ID: <LIm3kaxNVCwLe6hk7kwoQ8yoiL9EJOiqI3c0U7GSY1s=.3088b118-a78c-47f1-9d84-97b3ac823cc8@github.com>

On Tue, 1 Jul 2025 12:25:04 GMT, Anton Artemov <duke at openjdk.org> wrote:

> Hi, please consider the following changes:
> 
> this PR addresses the issue of stringop-overflow warnings produced by GCC. The compiler does think that the thread pointer returned by `JavaThread::current()` can be null, though it cant. The thread pointer ends up being an argument in `__atomic_load`, and the compiler reports the warning related to argument of that method. 
> 
> The patch adds a hint to the compiler by means of `__builtin_unreachable()` intrinsic, which tells the compiler that certain piece of code will never be reached (case of thread pointer being null). This solves the issue. 
> 
> Tested in tiers 1-3 and GHA.

Sorry but I really dislike seeing this compiler-specific pollution in shared code. It is even worse that you have to put it in two places (what is so special about the jvmci code to require this?) just because gcc is "too dumb" keep track of things.

Also IIUC from JBS the problem was only seen building Zero, so maybe we can do something there that is Zero specific?

Sorry.

-------------

PR Review: https://git.openjdk.org/jdk/pull/26067#pullrequestreview-2977963403

From dnsimon at openjdk.org  Wed Jul  2 08:58:41 2025
From: dnsimon at openjdk.org (Doug Simon)
Date: Wed, 2 Jul 2025 08:58:41 GMT
Subject: RFR: 8320353: Reenable stringop-overflow warnings
In-Reply-To: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
References: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
Message-ID: <wMHIciMZ4QYUToxmNokc5_kGX8T_fnwXxmbHlQXt7Qg=.d0cb5bfa-64d4-42b0-b76d-6b5bd81a2956@github.com>

On Tue, 1 Jul 2025 12:25:04 GMT, Anton Artemov <duke at openjdk.org> wrote:

> Hi, please consider the following changes:
> 
> this PR addresses the issue of stringop-overflow warnings produced by GCC. The compiler does think that the thread pointer returned by `JavaThread::current()` can be null, though it cant. The thread pointer ends up being an argument in `__atomic_load`, and the compiler reports the warning related to argument of that method. 
> 
> The patch adds a hint to the compiler by means of `__builtin_unreachable()` intrinsic, which tells the compiler that certain piece of code will never be reached (case of thread pointer being null). This solves the issue. 
> 
> Tested in tiers 1-3 and GHA.

src/hotspot/share/jvmci/jvmciCodeInstaller.cpp line 929:

> 927:   if (thread == nullptr) {
> 928:     // This is to prevent --stringop-overflow warning from GCC on linux/fastdebug.
> 929:     // GCC does believe that JavaThread::current() can return nullptr,

`does believe` -> `believes`

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26067#discussion_r2179510064

From jwaters at openjdk.org  Wed Jul  2 09:07:44 2025
From: jwaters at openjdk.org (Julian Waters)
Date: Wed, 2 Jul 2025 09:07:44 GMT
Subject: RFR: 8320353: Reenable stringop-overflow warnings
In-Reply-To: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
References: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
Message-ID: <T6noB03P0f3RK2RfhSkBqKzxwv8Nvgilrp-8KUPfkNo=.1398d88c-5e1a-4a17-84f1-7e98d9875eaf@github.com>

On Tue, 1 Jul 2025 12:25:04 GMT, Anton Artemov <duke at openjdk.org> wrote:

> Hi, please consider the following changes:
> 
> this PR addresses the issue of stringop-overflow warnings produced by GCC. The compiler does think that the thread pointer returned by `JavaThread::current()` can be null, though it cant. The thread pointer ends up being an argument in `__atomic_load`, and the compiler reports the warning related to argument of that method. 
> 
> The patch adds a hint to the compiler by means of `__builtin_unreachable()` intrinsic, which tells the compiler that certain piece of code will never be reached (case of thread pointer being null). This solves the issue. 
> 
> Tested in tiers 1-3 and GHA.

src/hotspot/share/runtime/javaThread.hpp line 1070:

> 1068:   // Returns the running thread as a JavaThread
> 1069:   static JavaThread* current() {
> 1070:     auto result = JavaThread::cast(Thread::current());

Why auto when the type is known?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26067#discussion_r2179528150

From jwaters at openjdk.org  Wed Jul  2 09:12:40 2025
From: jwaters at openjdk.org (Julian Waters)
Date: Wed, 2 Jul 2025 09:12:40 GMT
Subject: RFR: 8320353: Reenable stringop-overflow warnings
In-Reply-To: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
References: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
Message-ID: <0_bgDyCHArbkXPSDNQdNWLbrhY2AOJT_BjFTMD38tuY=.9ca01026-bf7e-4d59-8326-d5c4537acc17@github.com>

On Tue, 1 Jul 2025 12:25:04 GMT, Anton Artemov <duke at openjdk.org> wrote:

> Hi, please consider the following changes:
> 
> this PR addresses the issue of stringop-overflow warnings produced by GCC. The compiler does think that the thread pointer returned by `JavaThread::current()` can be null, though it cant. The thread pointer ends up being an argument in `__atomic_load`, and the compiler reports the warning related to argument of that method. 
> 
> The patch adds a hint to the compiler by means of `__builtin_unreachable()` intrinsic, which tells the compiler that certain piece of code will never be reached (case of thread pointer being null). This solves the issue. 
> 
> Tested in tiers 1-3 and GHA.

A less messy solution could be to use PRAGMA_STRINGOP_OVERFLOW_IGNORED instead. The conditionals for the PRAGMA IGNORED macros are already all handled for you, so there's no need to put conditionalizations at the use sites. As an example, simply do the following:


PRAGMA_DIAG_PUSH
PRAGMA_STRINGOP_OVERFLOW_IGNORED
// Problem code here
PRAGMA_DIAG_POP


The first 2 macros are placed directly in front of the problem code, and the last goes behind.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26067#issuecomment-3027075027

From kbarrett at openjdk.org  Wed Jul  2 14:01:41 2025
From: kbarrett at openjdk.org (Kim Barrett)
Date: Wed, 2 Jul 2025 14:01:41 GMT
Subject: RFR: 8320353: Reenable stringop-overflow warnings
In-Reply-To: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
References: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
Message-ID: <LLpd1-mLP3Q6EoLyOgM65uXPooSxUuw51szHOjR6bF0=.8a18dc2a-6b9a-49a6-b3f9-fad520377a7c@github.com>

On Tue, 1 Jul 2025 12:25:04 GMT, Anton Artemov <duke at openjdk.org> wrote:

> Hi, please consider the following changes:
> 
> this PR addresses the issue of stringop-overflow warnings produced by GCC. The compiler does think that the thread pointer returned by `JavaThread::current()` can be null, though it cant. The thread pointer ends up being an argument in `__atomic_load`, and the compiler reports the warning related to argument of that method. 
> 
> The patch adds a hint to the compiler by means of `__builtin_unreachable()` intrinsic, which tells the compiler that certain piece of code will never be reached (case of thread pointer being null). This solves the issue. 
> 
> Tested in tiers 1-3 and GHA.

Changes requested by kbarrett (Reviewer).

src/hotspot/share/jvmci/jvmciCodeInstaller.cpp line 926:

> 924: JVMCI::CodeInstallResult CodeInstaller::initialize_buffer(JVMCIObject compiled_code, CodeBuffer& buffer, HotSpotCompiledCodeStream* stream, u1 code_flags, JVMCI_TRAPS) {
> 925:   JavaThread* thread = stream->thread();
> 926: #if defined(__GNUC__) && !defined(__clang__) && !defined(PRODUCT)

Note that `!PRODUCT` does not imply `ASSERT`. There is also the "optimized" build type, which
defines neither `PRODUCT` nor `ASSERT`. I have no idea whether the false-positive gcc warnings
occur in that build configuration. If so, then my suggestion of using an assertion to give the compiler
the clue it needs won't work. But I'm guessing that isn't a problem, in which case the macro check
should be `defined(ASSERT)`.

src/hotspot/share/jvmci/jvmciCodeInstaller.cpp line 936:

> 934:     __builtin_unreachable();
> 935:   }
> 936: #endif

I think this could be replaced with asserting non-null.  A failed assert calls a noreturn reporting
function.

src/hotspot/share/runtime/javaThread.hpp line 1079:

> 1077:   }
> 1078: #endif
> 1079:     return result;

Similarly here, I think this could be replaced with asserting non-null.

-------------

PR Review: https://git.openjdk.org/jdk/pull/26067#pullrequestreview-2979254552
PR Review Comment: https://git.openjdk.org/jdk/pull/26067#discussion_r2180140733
PR Review Comment: https://git.openjdk.org/jdk/pull/26067#discussion_r2180123147
PR Review Comment: https://git.openjdk.org/jdk/pull/26067#discussion_r2180120768

From naoto at openjdk.org  Wed Jul  2 16:25:44 2025
From: naoto at openjdk.org (Naoto Sato)
Date: Wed, 2 Jul 2025 16:25:44 GMT
Subject: Withdrawn: 8360774: Use text representation of normalization data
 files
In-Reply-To: <TOEIN-RxBEAKv4Vleaa0WtyMijKUwSGuF_31bQA-pCo=.88f82538-a7ec-4dc7-b6fe-744906fbc48b@github.com>
References: <TOEIN-RxBEAKv4Vleaa0WtyMijKUwSGuF_31bQA-pCo=.88f82538-a7ec-4dc7-b6fe-744906fbc48b@github.com>
Message-ID: <qwMi_YD5Yow-pNEOATyrNKv_ANutIt16RfCk7_AXN3Q=.2078aed1-5a95-4587-ac45-b221019bd0ea@github.com>

On Fri, 27 Jun 2025 20:45:14 GMT, Naoto Sato <naoto at openjdk.org> wrote:

> The ICU4J component currently stores binary data files directly in the repository. This change replaces them with base64-encoded text files and converts them to binary during the build process

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/26027

From jkratochvil at openjdk.org  Wed Jul  2 17:20:48 2025
From: jkratochvil at openjdk.org (Jan Kratochvil)
Date: Wed, 2 Jul 2025 17:20:48 GMT
Subject: RFR: 8361288: Fix build of JTReg: wget exited with exit code 4
Message-ID: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>

https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689
[build.sh][INFO] Downloading https://archive.apache.org/dist/ant/binaries/apache-ant-1.10.8-bin.zip to /home/runner/work/jdk21u-dev/jdk21u-dev/jtreg/src/make/../build/deps/apache-ant-1.10.8-bin.zip
Error: sh][ERROR] wget exited with exit code 4
Error: Process completed with exit code 1.

It is enough to make an empty commit "retry CI" and push it to re-run the Github CI.
It happened for me twice:
https://github.com/openjdk/jdk21u-dev/pull/1663#event-18416642257
-> (I did not save the log, unaware how to access it now)
https://github.com/openjdk/jdk21u-dev/pull/1664#commits-pushed-c00e2ab
-> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689

-------------

Commit messages:
 - 8361288: Fix build of JTReg: wget exited with exit code 4

Changes: https://git.openjdk.org/jdk/pull/26100/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=26100&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8361288
  Stats: 8 lines in 1 file changed: 7 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/26100.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/26100/head:pull/26100

PR: https://git.openjdk.org/jdk/pull/26100

From kbarrett at openjdk.org  Thu Jul  3 01:32:53 2025
From: kbarrett at openjdk.org (Kim Barrett)
Date: Thu, 3 Jul 2025 01:32:53 GMT
Subject: RFR: 8320353: Reenable stringop-overflow warnings
In-Reply-To: <LLpd1-mLP3Q6EoLyOgM65uXPooSxUuw51szHOjR6bF0=.8a18dc2a-6b9a-49a6-b3f9-fad520377a7c@github.com>
References: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
 <LLpd1-mLP3Q6EoLyOgM65uXPooSxUuw51szHOjR6bF0=.8a18dc2a-6b9a-49a6-b3f9-fad520377a7c@github.com>
Message-ID: <fN-YSbHIJgK0sJ_O9fEtRgXSPyzrVT10D7_Cds5zvn0=.9a06b83c-984c-4f87-bce8-46639c458e4f@github.com>

On Wed, 2 Jul 2025 13:51:24 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:

>> Hi, please consider the following changes:
>> 
>> this PR addresses the issue of stringop-overflow warnings produced by GCC. The compiler does think that the thread pointer returned by `JavaThread::current()` can be null, though it cant. The thread pointer ends up being an argument in `__atomic_load`, and the compiler reports the warning related to argument of that method. 
>> 
>> The patch adds a hint to the compiler by means of `__builtin_unreachable()` intrinsic, which tells the compiler that certain piece of code will never be reached (case of thread pointer being null). This solves the issue. 
>> 
>> Tested in tiers 1-3 and GHA.
>
> src/hotspot/share/jvmci/jvmciCodeInstaller.cpp line 936:
> 
>> 934:     __builtin_unreachable();
>> 935:   }
>> 936: #endif
> 
> I think this could be replaced with asserting non-null.  A failed assert calls a noreturn reporting
> function.

Although I wonder why the assert in Thread::current() isn't sufficient? Maybe non-null info isn't
propagating through the cast to JavaThread?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26067#discussion_r2181311538

From kbarrett at openjdk.org  Thu Jul  3 01:52:41 2025
From: kbarrett at openjdk.org (Kim Barrett)
Date: Thu, 3 Jul 2025 01:52:41 GMT
Subject: RFR: 8320353: Reenable stringop-overflow warnings
In-Reply-To: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
References: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
Message-ID: <SPURRbg_QDlnGackOfs8p6LKFH3f9AdHgRZu_s_bUkA=.183c6e9d-48be-4bf4-8f85-9923da9da039@github.com>

On Tue, 1 Jul 2025 12:25:04 GMT, Anton Artemov <duke at openjdk.org> wrote:

> Hi, please consider the following changes:
> 
> this PR addresses the issue of stringop-overflow warnings produced by GCC. The compiler does think that the thread pointer returned by `JavaThread::current()` can be null, though it cant. The thread pointer ends up being an argument in `__atomic_load`, and the compiler reports the warning related to argument of that method. 
> 
> The patch adds a hint to the compiler by means of `__builtin_unreachable()` intrinsic, which tells the compiler that certain piece of code will never be reached (case of thread pointer being null). This solves the issue. 
> 
> Tested in tiers 1-3 and GHA.

make/hotspot/lib/CompileJvm.gmk line 199:

> 197:     DISABLED_WARNINGS_gcc_macroAssembler_ppc_sha.cpp := unused-const-variable, \
> 198:     DISABLED_WARNINGS_gcc_postaloc.cpp := address, \
> 199:     DISABLED_WARNINGS_gcc_shenandoahLock.cpp := stringop-overflow, \

What about this one in shenandoahLock.cpp?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26067#discussion_r2181357398

From kbarrett at openjdk.org  Thu Jul  3 05:36:42 2025
From: kbarrett at openjdk.org (Kim Barrett)
Date: Thu, 3 Jul 2025 05:36:42 GMT
Subject: RFR: 8320353: Reenable stringop-overflow warnings
In-Reply-To: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
References: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
Message-ID: <M6RgvhXwuZ6Rl3z62kcTVsbRMAlOVGuodqbsKyKHzFo=.bca79592-4adc-4027-88c3-1d70b7f86090@github.com>

On Tue, 1 Jul 2025 12:25:04 GMT, Anton Artemov <duke at openjdk.org> wrote:

> Hi, please consider the following changes:
> 
> this PR addresses the issue of stringop-overflow warnings produced by GCC. The compiler does think that the thread pointer returned by `JavaThread::current()` can be null, though it cant. The thread pointer ends up being an argument in `__atomic_load`, and the compiler reports the warning related to argument of that method. 
> 
> The patch adds a hint to the compiler by means of `__builtin_unreachable()` intrinsic, which tells the compiler that certain piece of code will never be reached (case of thread pointer being null). This solves the issue. 
> 
> Tested in tiers 1-3 and GHA.

My suggestion of using assertions instead doesn't seem to work, still getting
bogus warnings. This is with gcc14.2 on linux-aarch64, which also uses
__atomic_load. That is mysteriously weird. The noreturn reporting function
should be roughly equivalent to calling an ordinary function with an
"unreachable" following it. But then, this whole issue is mysteriously weird.

But that's okay; I wasn't really all that keen on adding assertions to
suppress bogus compiler warnings anyway.

At this point I think this change should just not be made. It's a compiler
bug. The existing suppression of the warnings is fine with me.

I think what's really needed is for someone to file a gcc bug.  It's too bad
nobody did that when the issue was first noticed.  There's already a bug that
is similar, and might even be the same, even though it involves sanitizers and
ours doesn't:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113775

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26067#issuecomment-3030839463

From duke at openjdk.org  Thu Jul  3 07:47:40 2025
From: duke at openjdk.org (Anton Artemov)
Date: Thu, 3 Jul 2025 07:47:40 GMT
Subject: RFR: 8320353: Reenable stringop-overflow warnings
In-Reply-To: <M6RgvhXwuZ6Rl3z62kcTVsbRMAlOVGuodqbsKyKHzFo=.bca79592-4adc-4027-88c3-1d70b7f86090@github.com>
References: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
 <M6RgvhXwuZ6Rl3z62kcTVsbRMAlOVGuodqbsKyKHzFo=.bca79592-4adc-4027-88c3-1d70b7f86090@github.com>
Message-ID: <YunUc3j5XRipL9egR5B7dqs0ANTJ6ETSL2udnFcBXDY=.ea2ee97d-caf9-4a06-b02a-f63aebe00f8a@github.com>

On Thu, 3 Jul 2025 05:34:27 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:

> My suggestion of using assertions instead doesn't seem to work, still getting bogus warnings. This is with gcc14.2 on linux-aarch64, which also uses __atomic_load. That is mysteriously weird. The noreturn reporting function should be roughly equivalent to calling an ordinary function with an "unreachable" following it. But then, this whole issue is mysteriously weird.
> 
> But that's okay; I wasn't really all that keen on adding assertions to suppress bogus compiler warnings anyway.
> 
> At this point I think this change should just not be made. It's a compiler bug. The existing suppression of the warnings is fine with me.
> 
> I think what's really needed is for someone to file a gcc bug. It's too bad nobody did that when the issue was first noticed. There's already a bug that is similar, and might even be the same, even though it involves sanitizers and ours doesn't: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113775

Thanks @kimbarrett, after looking at your findings I agree that it looks like a GCC bug, and therefore we should not do this change. I will try to make the smallest reproducible example and submit a bug report.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26067#issuecomment-3031232698

From ihse at openjdk.org  Thu Jul  3 10:21:40 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Thu, 3 Jul 2025 10:21:40 GMT
Subject: RFR: 8361288: Fix build of JTReg: wget exited with exit code 4
In-Reply-To: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
References: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
Message-ID: <ssABLvpNGyapgy8zINfDtAe_pwZJs0dF3hdEzdhCgGY=.8488ec82-79d7-433e-b4ec-bb3a32e31b98@github.com>

On Wed, 2 Jul 2025 16:52:39 GMT, Jan Kratochvil <jkratochvil at openjdk.org> wrote:

> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689
> [build.sh][INFO] Downloading https://archive.apache.org/dist/ant/binaries/apache-ant-1.10.8-bin.zip to /home/runner/work/jdk21u-dev/jdk21u-dev/jtreg/src/make/../build/deps/apache-ant-1.10.8-bin.zip
> Error: sh][ERROR] wget exited with exit code 4
> Error: Process completed with exit code 1.
> 
> It is enough to make an empty commit "retry CI" and push it to re-run the Github CI.
> It happened for me twice:
> https://github.com/openjdk/jdk21u-dev/pull/1663#event-18416642257
> -> (I did not save the log, unaware how to access it now)
> https://github.com/openjdk/jdk21u-dev/pull/1664#commits-pushed-c00e2ab
> -> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689

It's really a bummer that we need to build jtreg each time we run a GHA test. :-(

.github/actions/build-jtreg/action.yml line 59:

> 57:         while ! bash make/build.sh --jdk "$JAVA_HOME_17_X64"; do
> 58:           retry=$[$retry+1]
> 59:           if [ $retry -gt 20 ];then

20 retries seems excessive. There is likely a persistent problem if it retries that many times.

Suggestion:

          if [ $retry -gt 3 ];then

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26100#issuecomment-3031719129
PR Review Comment: https://git.openjdk.org/jdk/pull/26100#discussion_r2182416593

From jkratochvil at openjdk.org  Thu Jul  3 11:00:28 2025
From: jkratochvil at openjdk.org (Jan Kratochvil)
Date: Thu, 3 Jul 2025 11:00:28 GMT
Subject: RFR: 8361288: Fix build of JTReg: wget exited with exit code 4
 [v2]
In-Reply-To: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
References: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
Message-ID: <ZthIWvRqf0DmKfAutS3vzzGb6pjZuN0SBmyQszCQn_8=.d32a9909-b091-4b7d-baba-ad16d9fb1906@github.com>

> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689
> [build.sh][INFO] Downloading https://archive.apache.org/dist/ant/binaries/apache-ant-1.10.8-bin.zip to /home/runner/work/jdk21u-dev/jdk21u-dev/jtreg/src/make/../build/deps/apache-ant-1.10.8-bin.zip
> Error: sh][ERROR] wget exited with exit code 4
> Error: Process completed with exit code 1.
> 
> It is enough to make an empty commit "retry CI" and push it to re-run the Github CI.
> It happened for me twice:
> https://github.com/openjdk/jdk21u-dev/pull/1663#event-18416642257
> -> (I did not save the log, unaware how to access it now)
> https://github.com/openjdk/jdk21u-dev/pull/1664#commits-pushed-c00e2ab
> -> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689

Jan Kratochvil has updated the pull request incrementally with one additional commit since the last revision:

  retries 20->3
  add a sleep 30

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/26100/files
  - new: https://git.openjdk.org/jdk/pull/26100/files/45c41735..be68526a

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=26100&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=26100&range=00-01

  Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/26100.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/26100/head:pull/26100

PR: https://git.openjdk.org/jdk/pull/26100

From jkratochvil at openjdk.org  Thu Jul  3 11:00:29 2025
From: jkratochvil at openjdk.org (Jan Kratochvil)
Date: Thu, 3 Jul 2025 11:00:29 GMT
Subject: RFR: 8361288: Fix build of JTReg: wget exited with exit code 4
 [v2]
In-Reply-To: <ssABLvpNGyapgy8zINfDtAe_pwZJs0dF3hdEzdhCgGY=.8488ec82-79d7-433e-b4ec-bb3a32e31b98@github.com>
References: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
 <ssABLvpNGyapgy8zINfDtAe_pwZJs0dF3hdEzdhCgGY=.8488ec82-79d7-433e-b4ec-bb3a32e31b98@github.com>
Message-ID: <Phkwnea9ouVtG3l3BG3ygzjwEzKZXlxYW_Gr8RwVSt8=.8168e50e-1905-43ae-9b52-3bb0dd1f635b@github.com>

On Thu, 3 Jul 2025 10:18:47 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

>> Jan Kratochvil has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   retries 20->3
>>   add a sleep 30
>
> .github/actions/build-jtreg/action.yml line 59:
> 
>> 57:         while ! bash make/build.sh --jdk "$JAVA_HOME_17_X64"; do
>> 58:           retry=$[$retry+1]
>> 59:           if [ $retry -gt 20 ];then
> 
> 20 retries seems excessive. There is likely a persistent problem if it retries that many times.
> 
> Suggestion:
> 
>           if [ $retry -gt 3 ];then

Done. I have also added a sleep 30.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26100#discussion_r2182489712

From ihse at openjdk.org  Thu Jul  3 11:01:22 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Thu, 3 Jul 2025 11:01:22 GMT
Subject: RFR: 8361306: jdk.compiler-gendata needs to depend on
 java.base-launchers
Message-ID: <Ypwb4ZKxOf-WZPxp3_ECLpTTwfwlyqNGBo5AX20jlig=.f273a4f0-1072-4022-80ef-85d6b987bf54@github.com>

A recent run in the Oracle CI produced the following:

Creating ct.sym classes
/bin/bash: $WS/build/linux-aarch64-open/jdk/bin/java: No such file or directory
modules/jdk.compiler/Gendata.gmk:72: recipe for target '$WS/build/linux-aarch64-open/support/symbols/ct.sym' failed
make[3]: *** [$WS/build/linux-aarch64-open/support/symbols/ct.sym] Error 127
make[2]: *** [jdk.compiler-gendata] Error 2
make[2]: *** Waiting for unfinished jobs....
make/Main.gmk:136: recipe for target 'jdk.compiler-gendata' failed 


`jdk.compiler/Gendata.gmk` calls `$(BUILD_JAVA_SMALL)`, which is created by java.base-launchers (unless cross-compiling). This dependency is not recorded, and it has just worked due to pure luck since jdk/bin/java is usually produced much earlier in the build process.

-------------

Commit messages:
 - 8361306: jdk.compiler-gendata needs to depend on java.base-launchers

Changes: https://git.openjdk.org/jdk/pull/26113/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=26113&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8361306
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/26113.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/26113/head:pull/26113

PR: https://git.openjdk.org/jdk/pull/26113

From duke at openjdk.org  Thu Jul  3 12:04:44 2025
From: duke at openjdk.org (Anton Artemov)
Date: Thu, 3 Jul 2025 12:04:44 GMT
Subject: Withdrawn: 8320353: Reenable stringop-overflow warnings
In-Reply-To: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
References: <AWiF73BlsZ3NptMul6doFVw_XDpAyXSyIk5zOtGLrig=.75ff3d22-524e-423b-998a-9a5a7bb5098f@github.com>
Message-ID: <ifHIvUSdpFXevUEdGdw4H2NlTNegBILudXcalRwNBpQ=.7f952342-0bcb-4ba4-be49-6618d4b6755d@github.com>

On Tue, 1 Jul 2025 12:25:04 GMT, Anton Artemov <duke at openjdk.org> wrote:

> Hi, please consider the following changes:
> 
> this PR addresses the issue of stringop-overflow warnings produced by GCC. The compiler does think that the thread pointer returned by `JavaThread::current()` can be null, though it cant. The thread pointer ends up being an argument in `__atomic_load`, and the compiler reports the warning related to argument of that method. 
> 
> The patch adds a hint to the compiler by means of `__builtin_unreachable()` intrinsic, which tells the compiler that certain piece of code will never be reached (case of thread pointer being null). This solves the issue. 
> 
> Tested in tiers 1-3 and GHA.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/26067

From shade at openjdk.org  Thu Jul  3 12:41:41 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 3 Jul 2025 12:41:41 GMT
Subject: RFR: 8361288: Fix build of JTReg: wget exited with exit code 4
 [v2]
In-Reply-To: <ZthIWvRqf0DmKfAutS3vzzGb6pjZuN0SBmyQszCQn_8=.d32a9909-b091-4b7d-baba-ad16d9fb1906@github.com>
References: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
 <ZthIWvRqf0DmKfAutS3vzzGb6pjZuN0SBmyQszCQn_8=.d32a9909-b091-4b7d-baba-ad16d9fb1906@github.com>
Message-ID: <fC2gyog4CSP7TrrVg-Uxhnq1vRt-UkC8nmkHTrmwzHE=.d0d11be3-0abc-4a42-9139-1c04e7426d8a@github.com>

On Thu, 3 Jul 2025 11:00:28 GMT, Jan Kratochvil <jkratochvil at openjdk.org> wrote:

>> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689
>> [build.sh][INFO] Downloading https://archive.apache.org/dist/ant/binaries/apache-ant-1.10.8-bin.zip to /home/runner/work/jdk21u-dev/jdk21u-dev/jtreg/src/make/../build/deps/apache-ant-1.10.8-bin.zip
>> Error: sh][ERROR] wget exited with exit code 4
>> Error: Process completed with exit code 1.
>> 
>> It is enough to make an empty commit "retry CI" and push it to re-run the Github CI.
>> It happened for me twice:
>> https://github.com/openjdk/jdk21u-dev/pull/1663#event-18416642257
>> -> (I did not save the log, unaware how to access it now)
>> https://github.com/openjdk/jdk21u-dev/pull/1664#commits-pushed-c00e2ab
>> -> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689
>
> Jan Kratochvil has updated the pull request incrementally with one additional commit since the last revision:
> 
>   retries 20->3
>   add a sleep 30

We tried this before: https://bugs.openjdk.org/browse/JDK-8343190 -- and it did not work, so we have backed out. It looked as if the rate limiter basically bans the access for hours, so short retries do not work.

-------------

Changes requested by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/26100#pullrequestreview-2983003303

From shade at openjdk.org  Thu Jul  3 12:41:42 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 3 Jul 2025 12:41:42 GMT
Subject: RFR: 8361288: Fix build of JTReg: wget exited with exit code 4
In-Reply-To: <ssABLvpNGyapgy8zINfDtAe_pwZJs0dF3hdEzdhCgGY=.8488ec82-79d7-433e-b4ec-bb3a32e31b98@github.com>
References: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
 <ssABLvpNGyapgy8zINfDtAe_pwZJs0dF3hdEzdhCgGY=.8488ec82-79d7-433e-b4ec-bb3a32e31b98@github.com>
Message-ID: <4YYMKL3XwB2-GPNyX2lHSbrLN3SUBn5FbLkzBXaKHBE=.76922fd2-ac7a-4b76-a3fa-73c9888d20df@github.com>

On Thu, 3 Jul 2025 10:19:15 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

> It's really a bummer that we need to build jtreg each time we run a GHA test. :-(

Not each time, once per PR, then it gets cached :) It is a small consolation, though. It would be better once I have enough mental fortitude to complete [JDK-8343546](https://bugs.openjdk.org/browse/JDK-8343546) -- then jtreg would be cached across different PRs.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26100#issuecomment-3032117946

From jkratochvil at openjdk.org  Thu Jul  3 13:42:48 2025
From: jkratochvil at openjdk.org (Jan Kratochvil)
Date: Thu, 3 Jul 2025 13:42:48 GMT
Subject: RFR: 8361288: Fix build of JTReg: wget exited with exit code 4
 [v2]
In-Reply-To: <fC2gyog4CSP7TrrVg-Uxhnq1vRt-UkC8nmkHTrmwzHE=.d0d11be3-0abc-4a42-9139-1c04e7426d8a@github.com>
References: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
 <ZthIWvRqf0DmKfAutS3vzzGb6pjZuN0SBmyQszCQn_8=.d32a9909-b091-4b7d-baba-ad16d9fb1906@github.com>
 <fC2gyog4CSP7TrrVg-Uxhnq1vRt-UkC8nmkHTrmwzHE=.d0d11be3-0abc-4a42-9139-1c04e7426d8a@github.com>
Message-ID: <yCNXlMd_6oMgqFhKaTXbr2t81DNTAKb82agLwSZwPOw=.04f55d01-5a5c-46df-9171-b68880794bc6@github.com>

On Thu, 3 Jul 2025 12:37:28 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> It looked as if the rate limiter basically bans the access for hours, so short retries do not work. I see no point in trying it again.

OK so closing this one, it really looks the same.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26100#issuecomment-3032320997

From jkratochvil at openjdk.org  Thu Jul  3 13:42:49 2025
From: jkratochvil at openjdk.org (Jan Kratochvil)
Date: Thu, 3 Jul 2025 13:42:49 GMT
Subject: Withdrawn: 8361288: Fix build of JTReg: wget exited with exit code 4
In-Reply-To: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
References: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
Message-ID: <nPAL4dscxOQDJi7vMNebSYIXTS3b_ecrTber6PPAZ4M=.06e47c9c-3266-4a47-b306-381728ffa183@github.com>

On Wed, 2 Jul 2025 16:52:39 GMT, Jan Kratochvil <jkratochvil at openjdk.org> wrote:

> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689
> [build.sh][INFO] Downloading https://archive.apache.org/dist/ant/binaries/apache-ant-1.10.8-bin.zip to /home/runner/work/jdk21u-dev/jdk21u-dev/jtreg/src/make/../build/deps/apache-ant-1.10.8-bin.zip
> Error: sh][ERROR] wget exited with exit code 4
> Error: Process completed with exit code 1.
> 
> It is enough to make an empty commit "retry CI" and push it to re-run the Github CI.
> It happened for me twice:
> https://github.com/openjdk/jdk21u-dev/pull/1663#event-18416642257
> -> (I did not save the log, unaware how to access it now)
> https://github.com/openjdk/jdk21u-dev/pull/1664#commits-pushed-c00e2ab
> -> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/26100

From shade at openjdk.org  Thu Jul  3 13:48:40 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 3 Jul 2025 13:48:40 GMT
Subject: RFR: 8361306: jdk.compiler-gendata needs to depend on
 java.base-launchers
In-Reply-To: <Ypwb4ZKxOf-WZPxp3_ECLpTTwfwlyqNGBo5AX20jlig=.f273a4f0-1072-4022-80ef-85d6b987bf54@github.com>
References: <Ypwb4ZKxOf-WZPxp3_ECLpTTwfwlyqNGBo5AX20jlig=.f273a4f0-1072-4022-80ef-85d6b987bf54@github.com>
Message-ID: <qCvIaXUdwQ9BW3Nz1ERwvNYjWXOQdItrBIpYa_ao6Ig=.55266d0b-69a4-4b35-894a-f88f24bc8ca5@github.com>

On Thu, 3 Jul 2025 10:55:50 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

> A recent run in the Oracle CI produced the following:
> 
> Creating ct.sym classes
> /bin/bash: $WS/build/linux-aarch64-open/jdk/bin/java: No such file or directory
> modules/jdk.compiler/Gendata.gmk:72: recipe for target '$WS/build/linux-aarch64-open/support/symbols/ct.sym' failed
> make[3]: *** [$WS/build/linux-aarch64-open/support/symbols/ct.sym] Error 127
> make[2]: *** [jdk.compiler-gendata] Error 2
> make[2]: *** Waiting for unfinished jobs....
> make/Main.gmk:136: recipe for target 'jdk.compiler-gendata' failed 
> 
> 
> `jdk.compiler/Gendata.gmk` calls `$(BUILD_JAVA_SMALL)`, which is created by java.base-launchers (unless cross-compiling). This dependency is not recorded, and it has just worked due to pure luck since jdk/bin/java is usually produced much earlier in the build process.

Looks okay, but I also wonder if `jdk.javadoc-gendata` needs to depend on `java.base-launchers` too? I guess it only needs `bin/javadoc`, and that's why it works?

Alternatively, maybe we make `jdk.compiler-launchers` depend on `java.base-launchers`? IMO sounds reasonable that component-specific launcher depends on the "base" launcher like this.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/26113#pullrequestreview-2983266065

From sgehwolf at openjdk.org  Thu Jul  3 13:54:44 2025
From: sgehwolf at openjdk.org (Severin Gehwolf)
Date: Thu, 3 Jul 2025 13:54:44 GMT
Subject: RFR: 8361288: Fix build of JTReg: wget exited with exit code 4
 [v2]
In-Reply-To: <fC2gyog4CSP7TrrVg-Uxhnq1vRt-UkC8nmkHTrmwzHE=.d0d11be3-0abc-4a42-9139-1c04e7426d8a@github.com>
References: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
 <ZthIWvRqf0DmKfAutS3vzzGb6pjZuN0SBmyQszCQn_8=.d32a9909-b091-4b7d-baba-ad16d9fb1906@github.com>
 <fC2gyog4CSP7TrrVg-Uxhnq1vRt-UkC8nmkHTrmwzHE=.d0d11be3-0abc-4a42-9139-1c04e7426d8a@github.com>
Message-ID: <TUeczGuRLgQxMkCxqwsDH_i7OyBWDPqt9PWPNUsUSyY=.350e6a27-0382-4ebd-aec1-a2d6f7162975@github.com>

On Thu, 3 Jul 2025 12:37:28 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> We tried the retries like this before: https://bugs.openjdk.org/browse/JDK-8343190 -- and it did not work, so we have backed out. It looked as if the rate limiter basically bans the access for hours, so short retries do not work. I see no point in trying it again.

Yes. This hits archive.apache.org which has this warning:


Do note that heavy use of this service will result in immediate throttling of your download speeds to either 12 or 6
mbps for the remainder of the day, depending on severity. Continuous abuse (to the tune of more than 40 GB 
downloaded per week) will cause an [automatic ban](https://infra.apache.org/infra-ban.html), so please tune your 
services to this fact.


It's conceivable that GHA runners run into some of those limits.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26100#issuecomment-3032363126

From ihse at openjdk.org  Thu Jul  3 14:36:41 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Thu, 3 Jul 2025 14:36:41 GMT
Subject: RFR: 8361306: jdk.compiler-gendata needs to depend on
 java.base-launchers
In-Reply-To: <Ypwb4ZKxOf-WZPxp3_ECLpTTwfwlyqNGBo5AX20jlig=.f273a4f0-1072-4022-80ef-85d6b987bf54@github.com>
References: <Ypwb4ZKxOf-WZPxp3_ECLpTTwfwlyqNGBo5AX20jlig=.f273a4f0-1072-4022-80ef-85d6b987bf54@github.com>
Message-ID: <Di-cCuOpanItfHXSEi1pWTAvwA6D4zvFELlk7oLyeYQ=.496ebb6f-34a0-4d3f-aaba-6ac65c337bcb@github.com>

On Thu, 3 Jul 2025 10:55:50 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

> A recent run in the Oracle CI produced the following:
> 
> Creating ct.sym classes
> /bin/bash: $WS/build/linux-aarch64-open/jdk/bin/java: No such file or directory
> modules/jdk.compiler/Gendata.gmk:72: recipe for target '$WS/build/linux-aarch64-open/support/symbols/ct.sym' failed
> make[3]: *** [$WS/build/linux-aarch64-open/support/symbols/ct.sym] Error 127
> make[2]: *** [jdk.compiler-gendata] Error 2
> make[2]: *** Waiting for unfinished jobs....
> make/Main.gmk:136: recipe for target 'jdk.compiler-gendata' failed 
> 
> 
> `jdk.compiler/Gendata.gmk` calls `$(BUILD_JAVA_SMALL)`, which is created by java.base-launchers (unless cross-compiling). This dependency is not recorded, and it has just worked due to pure luck since jdk/bin/java is usually produced much earlier in the build process.

I did check jdk.javadoc/Gendata.gmk, and it only relies on the boot JDK. `jdk-compiler-launchers` is for creating `javac`, and it has not much to do with the current situation. We don't need to rely on having created the `java` launcher for that, and it is also the gendata step is performed before building launchers, so even if we put a dependency to the java.base-launcher there, there would still be a missing dependency for the gendata phase.

(The launchers do need libjli, but I am pretty sure this dependency is already covered. They do not need any other launcher, they are all independent.)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26113#issuecomment-3032504486

From ihse at openjdk.org  Thu Jul  3 16:06:45 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Thu, 3 Jul 2025 16:06:45 GMT
Subject: RFR: 8361288: Fix build of JTReg: wget exited with exit code 4
 [v2]
In-Reply-To: <ZthIWvRqf0DmKfAutS3vzzGb6pjZuN0SBmyQszCQn_8=.d32a9909-b091-4b7d-baba-ad16d9fb1906@github.com>
References: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
 <ZthIWvRqf0DmKfAutS3vzzGb6pjZuN0SBmyQszCQn_8=.d32a9909-b091-4b7d-baba-ad16d9fb1906@github.com>
Message-ID: <-z0tD9_QTa8h4jGagnGB9mtA0d6WlhvE0rXmASN1e8I=.2a9600cd-ed00-4f81-bfe9-104e3d85c47a@github.com>

On Thu, 3 Jul 2025 11:00:28 GMT, Jan Kratochvil <jkratochvil at openjdk.org> wrote:

>> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689
>> [build.sh][INFO] Downloading https://archive.apache.org/dist/ant/binaries/apache-ant-1.10.8-bin.zip to /home/runner/work/jdk21u-dev/jdk21u-dev/jtreg/src/make/../build/deps/apache-ant-1.10.8-bin.zip
>> Error: sh][ERROR] wget exited with exit code 4
>> Error: Process completed with exit code 1.
>> 
>> It is enough to make an empty commit "retry CI" and push it to re-run the Github CI.
>> It happened for me twice:
>> https://github.com/openjdk/jdk21u-dev/pull/1663#event-18416642257
>> -> (I did not save the log, unaware how to access it now)
>> https://github.com/openjdk/jdk21u-dev/pull/1664#commits-pushed-c00e2ab
>> -> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689
>
> Jan Kratochvil has updated the pull request incrementally with one additional commit since the last revision:
> 
>   retries 20->3
>   add a sleep 30

I feel we're not being very good net citizens here if we overload the apache infrastructure, regardless of if we get throttled or not... :-( Is there some other cache we can get the stuff we need from instead?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26100#issuecomment-3032811186

From sgehwolf at openjdk.org  Thu Jul  3 16:29:44 2025
From: sgehwolf at openjdk.org (Severin Gehwolf)
Date: Thu, 3 Jul 2025 16:29:44 GMT
Subject: RFR: 8361288: Fix build of JTReg: wget exited with exit code 4
 [v2]
In-Reply-To: <-z0tD9_QTa8h4jGagnGB9mtA0d6WlhvE0rXmASN1e8I=.2a9600cd-ed00-4f81-bfe9-104e3d85c47a@github.com>
References: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
 <ZthIWvRqf0DmKfAutS3vzzGb6pjZuN0SBmyQszCQn_8=.d32a9909-b091-4b7d-baba-ad16d9fb1906@github.com>
 <-z0tD9_QTa8h4jGagnGB9mtA0d6WlhvE0rXmASN1e8I=.2a9600cd-ed00-4f81-bfe9-104e3d85c47a@github.com>
Message-ID: <JxPAxjU9HpRWqi_Ra7aiPdaly9p8LqZq9FDGkgylZ7k=.592eb9a8-63e9-473c-908b-0ed671c34ceb@github.com>

On Thu, 3 Jul 2025 16:04:20 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

> I feel we're not being very good net citizens here if we overload the apache infrastructure, regardless of if we get throttled or not... :-( Is there some other cache we can get the stuff we need from instead?

I think if we were able to host a copy of ant 1.10.8 (the default version that jtreg uses) somewhere then we could switch to that instead when building jtreg.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26100#issuecomment-3032868300

From shade at openjdk.org  Thu Jul  3 18:43:44 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 3 Jul 2025 18:43:44 GMT
Subject: RFR: 8361288: Fix build of JTReg: wget exited with exit code 4
In-Reply-To: <4YYMKL3XwB2-GPNyX2lHSbrLN3SUBn5FbLkzBXaKHBE=.76922fd2-ac7a-4b76-a3fa-73c9888d20df@github.com>
References: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
 <ssABLvpNGyapgy8zINfDtAe_pwZJs0dF3hdEzdhCgGY=.8488ec82-79d7-433e-b4ec-bb3a32e31b98@github.com>
 <4YYMKL3XwB2-GPNyX2lHSbrLN3SUBn5FbLkzBXaKHBE=.76922fd2-ac7a-4b76-a3fa-73c9888d20df@github.com>
Message-ID: <Gmc6kt3CNDtJiD2DxPdZMfxMgMppN5FNQv-nsdcdXzw=.76374691-91ac-4237-a512-b5cffc3c1a1c@github.com>

On Thu, 3 Jul 2025 12:39:23 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> Not each time, once per PR, then it gets cached :) It is a small consolation, though. It would be better once I have enough mental fortitude to complete [JDK-8343546](https://bugs.openjdk.org/browse/JDK-8343546) -- then jtreg would be cached across different PRs.

FWIW, I now have a version that works, but it is somewhat ugly: https://github.com/openjdk/jdk/compare/master...shipilev:JDK-8343546-gha-master-cache :)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26100#issuecomment-3033206979

From ihse at openjdk.org  Fri Jul  4 12:21:43 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Fri, 4 Jul 2025 12:21:43 GMT
Subject: Integrated: 8361306: jdk.compiler-gendata needs to depend on
 java.base-launchers
In-Reply-To: <Ypwb4ZKxOf-WZPxp3_ECLpTTwfwlyqNGBo5AX20jlig=.f273a4f0-1072-4022-80ef-85d6b987bf54@github.com>
References: <Ypwb4ZKxOf-WZPxp3_ECLpTTwfwlyqNGBo5AX20jlig=.f273a4f0-1072-4022-80ef-85d6b987bf54@github.com>
Message-ID: <BwMQgrViXfJVnjnLFDTUdPr-OU8M8TYWZ2GEiP3GIYQ=.f007d4f8-3690-446d-8070-62eb39d99abc@github.com>

On Thu, 3 Jul 2025 10:55:50 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

> A recent run in the Oracle CI produced the following:
> 
> Creating ct.sym classes
> /bin/bash: $WS/build/linux-aarch64-open/jdk/bin/java: No such file or directory
> modules/jdk.compiler/Gendata.gmk:72: recipe for target '$WS/build/linux-aarch64-open/support/symbols/ct.sym' failed
> make[3]: *** [$WS/build/linux-aarch64-open/support/symbols/ct.sym] Error 127
> make[2]: *** [jdk.compiler-gendata] Error 2
> make[2]: *** Waiting for unfinished jobs....
> make/Main.gmk:136: recipe for target 'jdk.compiler-gendata' failed 
> 
> 
> `jdk.compiler/Gendata.gmk` calls `$(BUILD_JAVA_SMALL)`, which is created by java.base-launchers (unless cross-compiling). This dependency is not recorded, and it has just worked due to pure luck since jdk/bin/java is usually produced much earlier in the build process.

This pull request has now been integrated.

Changeset: fba74f79
Author:    Magnus Ihse Bursie <ihse at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/fba74f796eeeb42accc60ecab444c3d933b73e70
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

8361306: jdk.compiler-gendata needs to depend on java.base-launchers

Reviewed-by: shade

-------------

PR: https://git.openjdk.org/jdk/pull/26113

From shade at openjdk.org  Fri Jul  4 13:27:18 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 4 Jul 2025 13:27:18 GMT
Subject: RFR: 8343546: GHA: Cache required dependencies in master-branch
 workflow
Message-ID: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>

In our current GHA workflows, we only run workflows in branches in personal forks. GHA isolation rules say that workflow caches from the parent branches can be used by descendant branches. For our branches, the usual parent is `master`. Since we do not run workflows on `master`, this means every time we create a new branch, GHA would start with logically empty caches for it. Only the next trigger on the same branch would use the caches, saved from the first workflow run.

This means we put additional load on shared infrastructure with pulling JDKs, building jtreg (and pulling its dependencies), bootstrapping sysroots, etc. All these steps also fail intermittently every so often. It also means everyone carries lots of caches around, segregated by branch (look into your https://github.com/<github-name>/caches) only relying on cache cleanups when it starts to hit 10 GB. With 200+ contributors, this is easily 2 TB of cloud space we effectively waste in GHA clouds.

We can make all this more reliable, if we manage to run a master-branch workflow that bootstraps all required dependencies and caches them. These dependencies can then be used by PR branches, as "master" branch is their effective parent. 

This PR introduces the notion of "dry run", which does everything _except_ the actual builds and tests. Therefore, it verifies whether all dependencies are done properly for JDK configure to pass. This is useful in itself for future GHA debugging of dependencies. Workflow can be dispatched with additional "dry run" parameter now.

What makes master-branch caching possible is the second part of the PR that hooks up dry runs to master/stabilization branch pushes. These would make the dry-run workflow run every time you update your personal fork's master/stabilization branch. That dry run would likely finish very quickly if all caches are already in place. It would populate caches in master/stabilization branch in your personal fork, if not. 

The expected net result is that actual PRs that are branched off the personal fork master would be able to use the caches from that branch. (If you want to make this experiment in current GHA, trigger the existing workflow on `master` branch in your fork, it would do roughly the same, but with all builds/tests).

A sample "dry-run" can be seen here: https://github.com/shipilev/jdk/actions/runs/16074619302. The most heavy-weight part is MSYS2 unpacking in Windows builds, and that part is beyond our reach; the relevant GH action maintained by MSYS2 folks is responsible for managing caches for it.

-------------

Commit messages:
 - Single-script dry run
 - Initial version: dry-run from a separate file

Changes: https://git.openjdk.org/jdk/pull/26134/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=26134&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8343546
  Stats: 99 lines in 7 files changed: 96 ins; 2 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/26134.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/26134/head:pull/26134

PR: https://git.openjdk.org/jdk/pull/26134

From shade at openjdk.org  Fri Jul  4 13:27:49 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 4 Jul 2025 13:27:49 GMT
Subject: RFR: 8361288: Fix build of JTReg: wget exited with exit code 4
 [v2]
In-Reply-To: <ZthIWvRqf0DmKfAutS3vzzGb6pjZuN0SBmyQszCQn_8=.d32a9909-b091-4b7d-baba-ad16d9fb1906@github.com>
References: <Z4JNzVY5QX-5MLknUK0mIP36juSH0paaRRvMfJpRfRc=.6ceab9e8-765e-416a-9d48-d403707cc89c@github.com>
 <ZthIWvRqf0DmKfAutS3vzzGb6pjZuN0SBmyQszCQn_8=.d32a9909-b091-4b7d-baba-ad16d9fb1906@github.com>
Message-ID: <kpTwOMG97oTkMrgQu7bhcC4LCcScRnzjsrw7VQJDDps=.d9508249-69df-4fb7-809e-ef4f440e2b3c@github.com>

On Thu, 3 Jul 2025 11:00:28 GMT, Jan Kratochvil <jkratochvil at openjdk.org> wrote:

>> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689
>> [build.sh][INFO] Downloading https://archive.apache.org/dist/ant/binaries/apache-ant-1.10.8-bin.zip to /home/runner/work/jdk21u-dev/jdk21u-dev/jtreg/src/make/../build/deps/apache-ant-1.10.8-bin.zip
>> Error: sh][ERROR] wget exited with exit code 4
>> Error: Process completed with exit code 1.
>> 
>> It is enough to make an empty commit "retry CI" and push it to re-run the Github CI.
>> It happened for me twice:
>> https://github.com/openjdk/jdk21u-dev/pull/1663#event-18416642257
>> -> (I did not save the log, unaware how to access it now)
>> https://github.com/openjdk/jdk21u-dev/pull/1664#commits-pushed-c00e2ab
>> -> https://github.com/jankratochvil/jdk21u-dev/actions/runs/16026488095/job/45215623689
>
> Jan Kratochvil has updated the pull request incrementally with one additional commit since the last revision:
> 
>   retries 20->3
>   add a sleep 30

> > Not each time, once per PR, then it gets cached :) It is a small consolation, though. It would be better once I have enough mental fortitude to complete [JDK-8343546](https://bugs.openjdk.org/browse/JDK-8343546) -- then jtreg would be cached across different PRs.
> 
> FWIW, I now have a version that works, but it is somewhat ugly: [master...shipilev:JDK-8343546-gha-master-cache](https://github.com/openjdk/jdk/compare/master...shipilev:JDK-8343546-gha-master-cache) :)

Got a much cleaner version here: https://github.com/openjdk/jdk/pull/26134

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26100#issuecomment-3036278152

From ihse at openjdk.org  Fri Jul  4 14:26:39 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Fri, 4 Jul 2025 14:26:39 GMT
Subject: RFR: 8343546: GHA: Cache required dependencies in master-branch
 workflow
In-Reply-To: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>
References: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>
Message-ID: <QheMs1i087lV9bjBO0W0-G24ghXqWyd8xdH0n9lI-NI=.1a849775-5450-4464-844b-c4f896296ddb@github.com>

On Fri, 4 Jul 2025 13:21:40 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> In our current GHA workflows, we only run workflows in branches in personal forks. GHA isolation rules say that workflow caches from the parent branches can be used by descendant branches. For our branches, the usual parent is `master`. Since we do not run workflows on `master`, this means every time we create a new branch, GHA would start with logically empty caches for it. Only the next trigger on the same branch would use the caches, saved from the first workflow run.
> 
> This means we put additional load on shared infrastructure with pulling JDKs, building jtreg (and pulling its dependencies), bootstrapping sysroots, etc. All these steps also fail intermittently every so often. It also means everyone carries lots of caches around, segregated by branch and repo (look into your https://github.com/your-github-name/jdk/actions/caches, for example) only relying on cache cleanups when it starts to hit 10 GB. With hundreds of contributors, this easily wastes terabytes of cloud storage space.
> 
> We can make all this more efficient and reliable, if we manage to run a master-branch workflow that bootstraps all required dependencies and caches them. These dependencies can then be used by PR branches, as "master" branch is their effective parent. 
> 
> This PR introduces the notion of "dry run", which does everything _except_ the actual builds and tests. Therefore, it verifies whether all dependencies are done properly for JDK configure to pass. This is useful in itself for future GHA debugging of dependencies. Workflow can be dispatched with additional "dry run" parameter now.
> 
> What makes master-branch caching possible is the second part of the PR that hooks up dry runs to master/stabilization branch pushes. These would make the dry-run workflow run every time you update your personal fork's master/stabilization branch. That dry run would likely finish very quickly if all caches are already in place. It would populate caches in master/stabilization branch in your personal fork, if not. 
> 
> The expected net result is that actual PRs that are branched off the personal fork master would be able to use the caches from that master workflow run. (If you want to make this experiment in current GHA, trigger the existing workflow on `master` branch in your fork, it would do roughly the same, but with all builds/tests).
> 
> A sample "dry-run" can be seen here: https://github.com/shipilev/jdk/actions/runs/16074619302. The most heavy-weight part is MSYS2 unpacking in Windows builds, and t...

It looks like you managed to get this down to a clear and nice improvement. Just remove the debug code and you're good to go :)

.github/workflows/main.yml line 172:

> 170:               fi
> 171: 
> 172:               # FOR TESTING, REMOVE BEFORE INTEGRATION

^^^

-------------

Marked as reviewed by ihse (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/26134#pullrequestreview-2987438070
PR Review Comment: https://git.openjdk.org/jdk/pull/26134#discussion_r2185492128

From sgehwolf at openjdk.org  Fri Jul  4 14:26:39 2025
From: sgehwolf at openjdk.org (Severin Gehwolf)
Date: Fri, 4 Jul 2025 14:26:39 GMT
Subject: RFR: 8343546: GHA: Cache required dependencies in master-branch
 workflow
In-Reply-To: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>
References: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>
Message-ID: <KbkZ7CJ_QoXhxZVCDfGH8oFU7jOPEraFUPf7CKGRRw8=.8a2ec9d5-b1e2-4782-a1b2-c91c78c5fdf6@github.com>

On Fri, 4 Jul 2025 13:21:40 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> GHA isolation rules say that workflow caches from the parent branches can be used by descendant branches.

Is a corollary of that that one ought to keep one's fork `master` branch in sync with the main repo's (`openjdk/jdk`) `master` branch? I usually only do this locally (on my work machine), but not on the github fork. Just making sure I understand this right.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26134#issuecomment-3036477280

From ihse at openjdk.org  Fri Jul  4 14:35:38 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Fri, 4 Jul 2025 14:35:38 GMT
Subject: RFR: 8343546: GHA: Cache required dependencies in master-branch
 workflow
In-Reply-To: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>
References: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>
Message-ID: <aD5CufKm825mQqupJEz7D3Tb38QmLByMmfKSFz1NqrY=.adf368d0-e066-4959-9136-3a5c60724a1f@github.com>

On Fri, 4 Jul 2025 13:21:40 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> In our current GHA workflows, we only run workflows in branches in personal forks. GHA isolation rules say that workflow caches from the parent branches can be used by descendant branches. For our branches, the usual parent is `master`. Since we do not run workflows on `master`, this means every time we create a new branch, GHA would start with logically empty caches for it. Only the next trigger on the same branch would use the caches, saved from the first workflow run.
> 
> This means we put additional load on shared infrastructure with pulling JDKs, building jtreg (and pulling its dependencies), bootstrapping sysroots, etc. All these steps also fail intermittently every so often. It also means everyone carries lots of caches around, segregated by branch and repo (look into your https://github.com/your-github-name/jdk/actions/caches, for example) only relying on cache cleanups when it starts to hit 10 GB. With hundreds of contributors, this easily wastes terabytes of cloud storage space.
> 
> We can make all this more efficient and reliable, if we manage to run a master-branch workflow that bootstraps all required dependencies and caches them. These dependencies can then be used by PR branches, as "master" branch is their effective parent. 
> 
> This PR introduces the notion of "dry run", which does everything _except_ the actual builds and tests. Therefore, it verifies whether all dependencies are done properly for JDK configure to pass. This is useful in itself for future GHA debugging of dependencies. Workflow can be dispatched with additional "dry run" parameter now.
> 
> What makes master-branch caching possible is the second part of the PR that hooks up dry runs to master/stabilization branch pushes. These would make the dry-run workflow run every time you update your personal fork's master/stabilization branch. That dry run would likely finish very quickly if all caches are already in place. It would populate caches in master/stabilization branch in your personal fork, if not. 
> 
> The expected net result is that actual PRs that are branched off the personal fork master would be able to use the caches from that master workflow run. (If you want to make this experiment in current GHA, trigger the existing workflow on `master` branch in your fork, it would do roughly the same, but with all builds/tests).
> 
> A sample "dry-run" can be seen here: https://github.com/shipilev/jdk/actions/runs/16074619302. The most heavy-weight part is MSYS2 unpacking in Windows builds, and t...

That seems like a reasonable interpretation, yes. At least if you want to utilize this optimization. Otoh, Github makes it really easy nowadays with a simple "sync with upstream" button (or whatever it's called).

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26134#issuecomment-3036519454

From sgehwolf at openjdk.org  Fri Jul  4 14:39:38 2025
From: sgehwolf at openjdk.org (Severin Gehwolf)
Date: Fri, 4 Jul 2025 14:39:38 GMT
Subject: RFR: 8343546: GHA: Cache required dependencies in master-branch
 workflow
In-Reply-To: <aD5CufKm825mQqupJEz7D3Tb38QmLByMmfKSFz1NqrY=.adf368d0-e066-4959-9136-3a5c60724a1f@github.com>
References: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>
 <aD5CufKm825mQqupJEz7D3Tb38QmLByMmfKSFz1NqrY=.adf368d0-e066-4959-9136-3a5c60724a1f@github.com>
Message-ID: <u-SjQY0vclikPvO2f32OjTcgWDyHyHSLCSOmpk84P0M=.317dc6a5-d09a-4090-9792-6e0fcf7a82b2@github.com>

On Fri, 4 Jul 2025 14:32:44 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

> That seems like a reasonable interpretation, yes. At least if you want to utilize this optimization. Otoh, Github makes it really easy nowadays with a simple "sync with upstream" button (or whatever it's called).

Thanks. It's not a problem for me. Nevertheless developers need to know that there is an opt-in of sorts for this caching to work for them.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26134#issuecomment-3036534250

From shade at openjdk.org  Fri Jul  4 15:30:56 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 4 Jul 2025 15:30:56 GMT
Subject: RFR: 8343546: GHA: Cache required dependencies in master-branch
 workflow [v2]
In-Reply-To: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>
References: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>
Message-ID: <OG3z7OgCLfpmd9YLycYZ3D06er1LvAiAvnDxNRkeNTU=.74204cc7-3bfa-4c47-8f5f-3134a579996a@github.com>

> In our current GHA workflows, we only run workflows in branches in personal forks. GHA isolation rules say that workflow caches from the parent branches can be used by descendant branches. For our branches, the usual parent is `master`. Since we do not run workflows on `master`, this means every time we create a new branch, GHA would start with logically empty caches for it. Only the next trigger on the same branch would use the caches, saved from the first workflow run.
> 
> This means we put additional load on shared infrastructure with pulling JDKs, building jtreg (and pulling its dependencies), bootstrapping sysroots, etc. All these steps also fail intermittently every so often. It also means everyone carries lots of caches around, segregated by branch and repo (look into your https://github.com/your-github-name/jdk/actions/caches, for example) only relying on cache cleanups when it starts to hit 10 GB. With hundreds of contributors, this easily wastes terabytes of cloud storage space.
> 
> We can make all this more efficient and reliable, if we manage to run a master-branch workflow that bootstraps all required dependencies and caches them. These dependencies can then be used by PR branches, as "master" branch is their effective parent. 
> 
> This PR introduces the notion of "dry run", which does everything _except_ the actual builds and tests. Therefore, it verifies whether all dependencies are done properly for JDK configure to pass. This is useful in itself for future GHA debugging of dependencies. Workflow can be dispatched with additional "dry run" parameter now.
> 
> What makes master-branch caching possible is the second part of the PR that hooks up dry runs to master/stabilization branch pushes. These would make the dry-run workflow run every time you update your personal fork's master/stabilization branch. That dry run would likely finish very quickly if all caches are already in place. It would populate caches in master/stabilization branch in your personal fork, if not. 
> 
> The expected net result is that actual PRs that are branched off the personal fork master would be able to use the caches from that master workflow run. (If you want to make this experiment in current GHA, trigger the existing workflow on `master` branch in your fork, it would do roughly the same, but with all builds/tests).
> 
> A sample "dry-run" can be seen here: https://github.com/shipilev/jdk/actions/runs/16074619302. The most heavy-weight part is MSYS2 unpacking in Windows builds, and t...

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Final touches

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/26134/files
  - new: https://git.openjdk.org/jdk/pull/26134/files/bc087942..67b5be3d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=26134&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=26134&range=00-01

  Stats: 8 lines in 1 file changed: 1 ins; 6 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/26134.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/26134/head:pull/26134

PR: https://git.openjdk.org/jdk/pull/26134

From shade at openjdk.org  Fri Jul  4 15:30:56 2025
From: shade at openjdk.org (Aleksey Shipilev)
Date: Fri, 4 Jul 2025 15:30:56 GMT
Subject: RFR: 8343546: GHA: Cache required dependencies in master-branch
 workflow
In-Reply-To: <KbkZ7CJ_QoXhxZVCDfGH8oFU7jOPEraFUPf7CKGRRw8=.8a2ec9d5-b1e2-4782-a1b2-c91c78c5fdf6@github.com>
References: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>
 <KbkZ7CJ_QoXhxZVCDfGH8oFU7jOPEraFUPf7CKGRRw8=.8a2ec9d5-b1e2-4782-a1b2-c91c78c5fdf6@github.com>
Message-ID: <OkHvq4tQGbtpKQ0cSKg72slF1sR3_tSZjUnIJMiA-Mw=.a67cf4a6-ba35-44a1-835e-c043199360a8@github.com>

On Fri, 4 Jul 2025 14:24:00 GMT, Severin Gehwolf <sgehwolf at openjdk.org> wrote:

> > GHA isolation rules say that workflow caches from the parent branches can be used by descendant branches.
> 
> Is a corollary of that that one ought to keep one's fork `master` branch in sync with the main repo's (`openjdk/jdk`) `master` branch? I usually only do this locally (on my work machine), but not on the github fork. Just making sure I understand this right.

Yes, I think that's right. If personal fork `master` is not updated on GH side, the caches would be missing/stale, and the "old" caching scheme takes over like before: the caches would be per branch. In other words, caching stuff in remote `master` is completely opportunistic.

Keeping in perfect sync is not needed, IMO, as GHA caches are keyed by `.github` contents hash. So caches would become stale only when GHA code is changed.

FWIW, I prefer to sync `master` in fork in my master every so often, and keep it in sync with upstream `master`. Usually through a magic button in UI, sometimes with fetch/push. The optimization like this would probably push towards the similar workflow. We should probably write it down somewhere, but I don't quite see where...

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26134#issuecomment-3036683454

From sgehwolf at openjdk.org  Fri Jul  4 15:54:38 2025
From: sgehwolf at openjdk.org (Severin Gehwolf)
Date: Fri, 4 Jul 2025 15:54:38 GMT
Subject: RFR: 8343546: GHA: Cache required dependencies in master-branch
 workflow
In-Reply-To: <OkHvq4tQGbtpKQ0cSKg72slF1sR3_tSZjUnIJMiA-Mw=.a67cf4a6-ba35-44a1-835e-c043199360a8@github.com>
References: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>
 <KbkZ7CJ_QoXhxZVCDfGH8oFU7jOPEraFUPf7CKGRRw8=.8a2ec9d5-b1e2-4782-a1b2-c91c78c5fdf6@github.com>
 <OkHvq4tQGbtpKQ0cSKg72slF1sR3_tSZjUnIJMiA-Mw=.a67cf4a6-ba35-44a1-835e-c043199360a8@github.com>
Message-ID: <N4z9XvsJakM1QeSzDgpuTV8BFo7nIAeLNj9WwS-Y5BA=.95f61d12-8419-4b6b-99f3-5ac9fc1fb911@github.com>

On Fri, 4 Jul 2025 15:23:04 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> We should probably write it down somewhere, but I don't quite see where...

Maybe a heads-up to jdk-dev list would do and we have it in the list archives once this is in.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26134#issuecomment-3036754009

From ihse at openjdk.org  Fri Jul  4 20:35:38 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Fri, 4 Jul 2025 20:35:38 GMT
Subject: RFR: 8343546: GHA: Cache required dependencies in master-branch
 workflow [v2]
In-Reply-To: <OG3z7OgCLfpmd9YLycYZ3D06er1LvAiAvnDxNRkeNTU=.74204cc7-3bfa-4c47-8f5f-3134a579996a@github.com>
References: <QSffQ_1OmAouwwG-s9ASR1HXZd3HMbfNfYOYC_eMRBA=.3b781511-aca4-48ff-aa7d-ce177702d050@github.com>
 <OG3z7OgCLfpmd9YLycYZ3D06er1LvAiAvnDxNRkeNTU=.74204cc7-3bfa-4c47-8f5f-3134a579996a@github.com>
Message-ID: <IqTfW1NLl6mNpBryHm9wVlakcPvWdDCpfzzQvu-SkbA=.b3a4909b-1144-4108-9fae-505df8794e81@github.com>

On Fri, 4 Jul 2025 15:30:56 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> In our current GHA workflows, we only run workflows in branches in personal forks. GHA isolation rules say that workflow caches from the parent branches can be used by descendant branches. For our branches, the usual parent is `master`. Since we do not run workflows on `master`, this means every time we create a new branch, GHA would start with logically empty caches for it. Only the next trigger on the same branch would use the caches, saved from the first workflow run.
>> 
>> This means we put additional load on shared infrastructure with pulling JDKs, building jtreg (and pulling its dependencies), bootstrapping sysroots, etc. All these steps also fail intermittently every so often. It also means everyone carries lots of caches around, segregated by branch and repo (look into your https://github.com/your-github-name/jdk/actions/caches, for example) only relying on cache cleanups when it starts to hit 10 GB. With hundreds of contributors, this easily wastes terabytes of cloud storage space.
>> 
>> We can make all this more efficient and reliable, if we manage to run a master-branch workflow that bootstraps all required dependencies and caches them. These dependencies can then be used by PR branches, as "master" branch is their effective parent. 
>> 
>> This PR introduces the notion of "dry run", which does everything _except_ the actual builds and tests. Therefore, it verifies whether all dependencies are done properly for JDK configure to pass. This is useful in itself for future GHA debugging of dependencies. Workflow can be dispatched with additional "dry run" parameter now.
>> 
>> What makes master-branch caching possible is the second part of the PR that hooks up dry runs to master/stabilization branch pushes. These would make the dry-run workflow run every time you update your personal fork's master/stabilization branch. That dry run would likely finish very quickly if all caches are already in place. It would populate caches in master/stabilization branch in your personal fork, if not. 
>> 
>> The expected net result is that actual PRs that are branched off the personal fork master would be able to use the caches from that master workflow run. (If you want to make this experiment in current GHA, trigger the existing workflow on `master` branch in your fork, it would do roughly the same, but with all builds/tests).
>> 
>> A sample "dry-run" can be seen here: https://github.com/shipilev/jdk/actions/runs/16074619302. The most heavy-weight part is MSYS2 unpackin...
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Final touches

Yeah, a mail to jdk-dev would probably be good.

-------------

Marked as reviewed by ihse (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/26134#pullrequestreview-2988297474