RFR: 8347755: Support static library in jmod
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`. The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool. Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu. ------------- Commit messages: - Fix julong format on MacOS - Add --static-libs to jmod Changes: https://git.openjdk.org/leyden/pull/46/files Webrev: https://webrevs.openjdk.org/?repo=leyden&pr=46&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8347755 Stats: 95 lines in 6 files changed: 79 ins; 0 del; 16 mod Patch: https://git.openjdk.org/leyden/pull/46.diff Fetch: git fetch https://git.openjdk.org/leyden.git pull/46/head:pull/46 PR: https://git.openjdk.org/leyden/pull/46
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision: Use JULONG_FORMAT_X to suit various platforms ------------- Changes: - all: https://git.openjdk.org/leyden/pull/46/files - new: https://git.openjdk.org/leyden/pull/46/files/bfd37c0a..fc47dee8 Webrevs: - full: https://webrevs.openjdk.org/?repo=leyden&pr=46&range=01 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=46&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/leyden/pull/46.diff Fetch: git fetch https://git.openjdk.org/leyden.git pull/46/head:pull/46 PR: https://git.openjdk.org/leyden/pull/46
On Tue, 4 Mar 2025 04:20:48 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision:
Use JULONG_FORMAT_X to suit various platforms
test/jdk/tools/jmod/JmodTest.java line 959:
957: } 958: 959: static void assertMagic(Path file, int minorVersion) throws IOException {
testLibs and testStaticLibs checking the magic number is good, just might be better to have the two call assertMagic with both the expected major and minor version rather than just the minor version. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1979081747
On Tue, 4 Mar 2025 04:20:48 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision:
Use JULONG_FORMAT_X to suit various platforms
The updates to the JMOD format and jmod tool looks good, as is bumping the version when the JMOD includes a static-libs section. Doesn't have to be this PR but jmod.md will need to be updated if the overall feature is eventually proposed for main line. ------------- PR Review: https://git.openjdk.org/leyden/pull/46#pullrequestreview-2656793519
On Tue, 4 Mar 2025 10:11:55 GMT, Alan Bateman <alanb@openjdk.org> wrote:
Doesn't have to be this PR but jmod.md will need to be updated if the overall feature is eventually proposed for main line.
Also CSR would be needed. ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2698218693
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision: Update jmod.md ------------- Changes: - all: https://git.openjdk.org/leyden/pull/46/files - new: https://git.openjdk.org/leyden/pull/46/files/fc47dee8..42923fb1 Webrevs: - full: https://webrevs.openjdk.org/?repo=leyden&pr=46&range=02 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=46&range=01-02 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod Patch: https://git.openjdk.org/leyden/pull/46.diff Fetch: git fetch https://git.openjdk.org/leyden.git pull/46/head:pull/46 PR: https://git.openjdk.org/leyden/pull/46
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision: support major version validation ------------- Changes: - all: https://git.openjdk.org/leyden/pull/46/files - new: https://git.openjdk.org/leyden/pull/46/files/42923fb1..95a037b8 Webrevs: - full: https://webrevs.openjdk.org/?repo=leyden&pr=46&range=03 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=46&range=02-03 Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.org/leyden/pull/46.diff Fetch: git fetch https://git.openjdk.org/leyden.git pull/46/head:pull/46 PR: https://git.openjdk.org/leyden/pull/46
On Tue, 4 Mar 2025 16:23:36 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision:
support major version validation
src/java.base/share/classes/jdk/internal/jmod/JmodFile.java line 195:
193: } else { 194: os.write(JMOD_MAGIC_NO_STATIC_LIB); 195: }
This conditionality seems a bit strange. Isn't the idea for the magic to be changed when the format got extended to do more things? I.e. shouldn't this be micro version `0x01` in both cases? Then when dealing with version JMOD `1.1` version you'd be checking if static-libs is there. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1981396924
On Wed, 5 Mar 2025 13:29:35 GMT, Severin Gehwolf <sgehwolf@openjdk.org> wrote:
Henry Jen has updated the pull request incrementally with one additional commit since the last revision:
support major version validation
src/java.base/share/classes/jdk/internal/jmod/JmodFile.java line 195:
193: } else { 194: os.write(JMOD_MAGIC_NO_STATIC_LIB); 195: }
This conditionality seems a bit strange. Isn't the idea for the magic to be changed when the format got extended to do more things? I.e. shouldn't this be micro version `0x01` in both cases? Then when dealing with version JMOD `1.1` version you'd be checking if static-libs is there.
The idea is without static lib, it's essential the same as the old format, using the old version so that old tools can be used if static is not included. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1981636419
On Wed, 5 Mar 2025 15:25:49 GMT, Henry Jen <henryjen@openjdk.org> wrote:
src/java.base/share/classes/jdk/internal/jmod/JmodFile.java line 195:
193: } else { 194: os.write(JMOD_MAGIC_NO_STATIC_LIB); 195: }
This conditionality seems a bit strange. Isn't the idea for the magic to be changed when the format got extended to do more things? I.e. shouldn't this be micro version `0x01` in both cases? Then when dealing with version JMOD `1.1` version you'd be checking if static-libs is there.
The idea is without static lib, it's essential the same as the old format, using the old version so that old tools can be used if static is not included.
I thought the idea was that the jmod format was left intentionally undefined, to not think you could rely on anything else than the corresponding JDK tools to process it? That would mean backwards compatibility should not be a concern. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1983551278
On Thu, 6 Mar 2025 15:18:55 GMT, Magnus Ihse Bursie <ihse@openjdk.org> wrote:
The idea is without static lib, it's essential the same as the old format, using the old version so that old tools can be used if static is not included.
I thought the idea was that the jmod format was left intentionally undefined, to not think you could rely on anything else than the corresponding JDK tools to process it? That would mean backwards compatibility should not be a concern.
(Just to be clear, this is about the JMOD format, not the jimage format. The jimage format is JDK internal and we are free to change it from release to release, even build to build, without compatibility concerns.) You are right that the JMOD format is not documented but care is required when rev'ing the format because there are JDK tools that produce and consume this format. It should be possible for a project to produce a module in JMOD format with the `jmod` tool from JDK N, and put the package module on the module path when creating a run-time image with JDK N+1 `jlink`. So what Henry has looks right. It means the `jmod` tool will produce a 1.0 format when creating a package module that doesn't have static libs, and a JMOD in the new format when the module has static libs. This seems preferable to adding a `--release` option. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1983719080
On Thu, 6 Mar 2025 16:47:39 GMT, Alan Bateman <alanb@openjdk.org> wrote:
I thought the idea was that the jmod format was left intentionally undefined, to not think you could rely on anything else than the corresponding JDK tools to process it? That would mean backwards compatibility should not be a concern.
(Just to be clear, this is about the JMOD format, not the jimage format. The jimage format is JDK internal and we are free to change it from release to release, even build to build, without compatibility concerns.)
You are right that the JMOD format is not documented but care is required when rev'ing the format because there are JDK tools that produce and consume this format. It should be possible for a project to produce a module in JMOD format with the `jmod` tool from JDK N, and put the package module on the module path when creating a run-time image with JDK N+1 `jlink`. So what Henry has looks right. It means the `jmod` tool will produce a 1.0 format when creating a package module that doesn't have static libs, and a JMOD in the new format when the module has static libs. This seems preferable to adding a `--release` option.
I guess its worth asking how would the format evolve in the future if more than `/static-lib` were added? Just as a strawman: say `/docs` and `/sources` were added in some sequence. Would this grow to say if (<nothing new>) { // 1.0 } else if (<just static-lib>) { // 1.1 } else { // 1.2 } What if `/sources` (and corresponding generation of `src.zip`) were added first and `/docs` second? if (<nothing new>) { // 1.0 } else if (<just static-lib>) { // 1.1 } else if (<just sources +/- static-lib>) { // 1.2 } else { // 1.3 } Or would, at that point, versioning move to be closer to what is done for classfiles. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1986138478
On Sat, 8 Mar 2025 19:20:13 GMT, Ethan McCue <duke@openjdk.org> wrote:
I guess its worth asking how would the format evolve in the future if more than `/static-lib` were added?
Unlike the JAR format, the JMOD format requires an update when new sections are added. We don't need to get hung up on any of the details around this now, most of the work to support this will be in jlink, not the jmod tool. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1986247305
On Sun, 9 Mar 2025 08:08:33 GMT, Alan Bateman <alanb@openjdk.org> wrote:
I guess its worth asking how would the format evolve in the future if more than `/static-lib` were added? Just as a strawman: say `/docs` and `/sources` were added in some sequence. Would this grow to say
if (<nothing new>) { // 1.0 } else if (<just static-lib>) { // 1.1 } else { // 1.2 }
What if `/sources` (and corresponding generation of `src.zip`) were added first and `/docs` second?
if (<nothing new>) { // 1.0 } else if (<just static-lib>) { // 1.1 } else if (<just sources +/- static-lib>) { // 1.2 } else { // 1.3 }
Or would, at that point, versioning move to be closer to what is done for classfiles.
I guess its worth asking how would the format evolve in the future if more than `/static-lib` were added?
Unlike the JAR format, the JMOD format requires an update when new sections are added. We don't need to get hung up on any of the details around this now, most of the work to support this will be in jlink, not the jmod tool.
I'm ok with this for the leyden branch. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1986918228
On Mon, 10 Mar 2025 09:31:17 GMT, Severin Gehwolf <sgehwolf@openjdk.org> wrote:
I guess its worth asking how would the format evolve in the future if more than `/static-lib` were added?
Unlike the JAR format, the JMOD format requires an update when new sections are added. We don't need to get hung up on any of the details around this now, most of the work to support this will be in jlink, not the jmod tool.
I'm ok with this for the leyden branch.
I am thinking just to support -1 version. So 1.2 jmod can produce 1.2 with latest feature and 1.1 for previous version. Anyway, as Alan pointed out, we don't need to figure this out right now. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1994038253
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: - Merge - Support `make static-jmods` to create static jmods - support major version validation - Update jmod.md - Use JULONG_FORMAT_X to suit various platforms - Fix julong format on MacOS - Add --static-libs to jmod ------------- Changes: - all: https://git.openjdk.org/leyden/pull/46/files - new: https://git.openjdk.org/leyden/pull/46/files/95a037b8..242e9e7a Webrevs: - full: https://webrevs.openjdk.org/?repo=leyden&pr=46&range=04 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=46&range=03-04 Stats: 25322 lines in 856 files changed: 10200 ins; 10637 del; 4485 mod Patch: https://git.openjdk.org/leyden/pull/46.diff Fetch: git fetch https://git.openjdk.org/leyden.git pull/46/head:pull/46 PR: https://git.openjdk.org/leyden/pull/46
On Thu, 13 Mar 2025 17:46:43 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
- Merge - Support `make static-jmods` to create static jmods - support major version validation - Update jmod.md - Use JULONG_FORMAT_X to suit various platforms - Fix julong format on MacOS - Add --static-libs to jmod
Latest commit to support creating of static-jmods with `make static-jmods`, thanks @magicus for help. Result on MacOS, ls -l build/macosx-x86_64-server-release/images/static-jmods total 184152 -rw-r--r-- 1 hjen staff 21778174 Mar 13 10:35 java.base.jmod -rw-r--r-- 1 hjen staff 140500 Mar 13 10:35 java.compiler.jmod -rw-r--r-- 1 hjen staff 58934 Mar 13 10:35 java.datatransfer.jmod -rw-r--r-- 1 hjen staff 16688299 Mar 13 10:35 java.desktop.jmod -rw-r--r-- 1 hjen staff 68824 Mar 13 10:35 java.instrument.jmod ... -rw-r--r-- 1 hjen staff 49946 Mar 13 10:35 jdk.xml.dom.jmod -rw-r--r-- 1 hjen staff 112583 Mar 13 10:35 jdk.zipfs.jmod ./build/macosx-x86_64-server-release/jdk/bin/jmod list build/macosx-x86_64-server-release/images/static-jmods/java.base.jmod | grep static-lib static-lib/libjava.a static-lib/libjimage.a static-lib/libjli.a static-lib/libjsig.a static-lib/libnet.a static-lib/libnio.a static-lib/libosxsecurity.a static-lib/libsyslookup.a static-lib/libverify.a static-lib/libzip.a static-lib/module-included-libs.txt ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2722247186
On Thu, 13 Mar 2025 17:55:01 GMT, Henry Jen <henryjen@openjdk.org> wrote:
./build/macosx-x86_64-server-release/jdk/bin/jmod list build/macosx-x86_64-server-release/images/static-jmods/java.base.jmod | grep static-lib static-lib/libjava.a static-lib/libjimage.a static-lib/libjli.a static-lib/libjsig.a static-lib/libnet.a static-lib/libnio.a static-lib/libosxsecurity.a static-lib/libsyslookup.a static-lib/libverify.a static-lib/libzip.a static-lib/module-included-libs.txt
I think `libjvm.a` should be included in java.base.jmod as well. ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2722343210
On Thu, 13 Mar 2025 18:29:19 GMT, Jiangli Zhou <jiangli@openjdk.org> wrote:
I think `libjvm.a` should be included in java.base.jmod as well.
You are right. Looks libjvm.a is not build as a dependency, and I didn't try to copy that file either. Looks to me, it is only built for static launcher, I'll try to add that for java.base-static-jmod. ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2723149831
On Thu, 13 Mar 2025 17:46:43 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
- Merge - Support `make static-jmods` to create static jmods - support major version validation - Update jmod.md - Use JULONG_FORMAT_X to suit various platforms - Fix julong format on MacOS - Add --static-libs to jmod
Verified with `make java.base-static-jmod`, and libjvm.a is included. ./build/macosx-x86_64-server-release/jdk/bin/jmod list build/macosx-x86_64-server-release/images/static-jmods/java.base.jmod | grep static-lib static-lib/libjava.a static-lib/libjimage.a static-lib/libjli.a static-lib/libjsig.a static-lib/libjvm.a static-lib/libnet.a static-lib/libnio.a static-lib/libosxsecurity.a static-lib/libsyslookup.a static-lib/libverify.a static-lib/libzip.a static-lib/module-included-libs.txt ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2723458595
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision: Add hotspot static libs to java.base-static-jmod ------------- Changes: - all: https://git.openjdk.org/leyden/pull/46/files - new: https://git.openjdk.org/leyden/pull/46/files/242e9e7a..0a16f914 Webrevs: - full: https://webrevs.openjdk.org/?repo=leyden&pr=46&range=05 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=46&range=04-05 Stats: 15 lines in 2 files changed: 15 ins; 0 del; 0 mod Patch: https://git.openjdk.org/leyden/pull/46.diff Fetch: git fetch https://git.openjdk.org/leyden.git pull/46/head:pull/46 PR: https://git.openjdk.org/leyden/pull/46
On Fri, 14 Mar 2025 04:42:49 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision:
Add hotspot static libs to java.base-static-jmod
`static-lib/module-included-libs.txt` should not be included. It is a built-time support file. I guess this happens since you re-used an existing directory structure that was only used for build support files before. The proper fix is to make sure this file ends up in a different directory. ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2724209876
On Fri, 14 Mar 2025 10:00:00 GMT, Magnus Ihse Bursie <ihse@openjdk.org> wrote:
`static-lib/module-included-libs.txt` should not be included. It is a built-time support file. I guess this happens since you re-used an existing directory structure that was only used for build support files before. The proper fix is to make sure this file ends up in a different directory.
I didn't particularly filter-out or move this file because I wonder if this file serve as a manifest. Otherwise, we could do something like how we process other folders like modules_libs. `jmod --exclude` could be a solution as well. ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2724987885
On Fri, 14 Mar 2025 15:09:11 GMT, Henry Jen <henryjen@openjdk.org> wrote:
I didn't particularly filter-out or move this file because I wonder if this file serve as a manifest.
No, it doesn't, it's just an implementation specific detail of the build system.
Otherwise, we could do something like how we process other folders like modules_libs. `jmod --exclude` could be a solution as well.
No, no. We should just don't mix temporary support files with the proper output files. ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2725402561
On Fri, 14 Mar 2025 18:01:35 GMT, Magnus Ihse Bursie <ihse@openjdk.org> wrote:
I didn't particularly filter-out or move this file because I wonder if this file serve as a manifest.
No, it doesn't, it's just an implementation specific detail of the build system.
I know the original intention of this is not for that. What I am asking is if we should have such a manifest or hints for user? Similar to pkg-config.
Otherwise, we could do something like how we process other folders like modules_libs. `jmod --exclude` could be a solution as well.
No, no. We should just don't mix temporary support files with the proper output files.
That I agree. ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2725414257
On Fri, 14 Mar 2025 18:07:05 GMT, Henry Jen <henryjen@openjdk.org> wrote:
Similar to pkg-config.
Maybe we just wrote past each other here... As I said, my plan is to actually include a pkg-config file. This is a canonical way to describe exactly what is needed to utilize a library. There is no reason to do "something almost like pkg-config but not actually pkg-config". :-) ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2725420077
On Fri, 14 Mar 2025 18:07:05 GMT, Henry Jen <henryjen@openjdk.org> wrote:
I didn't particularly filter-out or move this file because I wonder if this file serve as a manifest.
No, it doesn't, it's just an implementation specific detail of the build system.
Otherwise, we could do something like how we process other folders like modules_libs. `jmod --exclude` could be a solution as well.
No, no. We should just don't mix temporary support files with the proper output files.
I didn't particularly filter-out or move this file because I wonder if this file serve as a manifest.
No, it doesn't, it's just an implementation specific detail of the build system.
I know the original intention of this is not for that. What I am asking is if we should have such a manifest or hints for user? Similar to pkg-config.
Otherwise, we could do something like how we process other folders like modules_libs. `jmod --exclude` could be a solution as well.
No, no. We should just don't mix temporary support files with the proper output files.
That I agree.
@slowhog This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!
@slowhog Could you please add a comment to make sure your PR is not closed? Thanks ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2814127775
On Fri, 14 Mar 2025 04:42:49 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision:
Add hotspot static libs to java.base-static-jmod
make/CreateJmods.gmk line 104:
102: endif 103: 104: # Copy hotspot libs, currently only libjvm.a
I just realized that we also need the launcher's `main.o` in the static `java.base.jmod`. Creating a `liblauncher.a` with the `main.o` and including the `liblauncher.a` in `java.base.jmod` seems to be cleaner than including `main.o` directly in the jmod. When jlinking the final image for the application, if user doesn't provide customized launcher, `liblauncher.a` is used with other needed JDK/VM static libraries for linking the executable using the native linker (e.g. `ld`). If customized launcher is used, the object file(s) or the static library for the customized launcher is then used when linking the executable. `liblauncher.a` is not used in that case. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1996018727
On Fri, 14 Mar 2025 17:48:30 GMT, Jiangli Zhou <jiangli@openjdk.org> wrote:
Henry Jen has updated the pull request incrementally with one additional commit since the last revision:
Add hotspot static libs to java.base-static-jmod
make/CreateJmods.gmk line 104:
102: endif 103: 104: # Copy hotspot libs, currently only libjvm.a
I just realized that we also need the launcher's `main.o` in the static `java.base.jmod`. Creating a `liblauncher.a` with the `main.o` and including the `liblauncher.a` in `java.base.jmod` seems to be cleaner than including `main.o` directly in the jmod.
When jlinking the final image for the application, if user doesn't provide customized launcher, `liblauncher.a` is used with other needed JDK/VM static libraries for linking the executable using the native linker (e.g. `ld`). If customized launcher is used, the object file(s) or the static library for the customized launcher is then used when linking the executable. `liblauncher.a` is not used in that case.
Should we really, though? That includes a `main` function. If the user wants to use the static libs from the jmod to create their own application, that will cause a symbol conflict. Sorry @jianglizhou, I did not properly read the second part of your suggestion, to create a `liblauncher.o` out of `main.o`. Well, tbh, this is just JDK-8351367 all over again. If we should include "optional" libraries in the jmods, we need to figure out first how to handle them. How should they be marked/classified as optional or required? How should the user tell jlink to include or exclude them? If we do get a good answer to that question, then we can introduce an optional `liblauncher.a` as well alongside `libjsig.a`. But until then, we should just stay clear of "optional" static libraries. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1996034651 PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1996053235
On Fri, 14 Mar 2025 18:00:26 GMT, Magnus Ihse Bursie <ihse@openjdk.org> wrote:
make/CreateJmods.gmk line 104:
102: endif 103: 104: # Copy hotspot libs, currently only libjvm.a
I just realized that we also need the launcher's `main.o` in the static `java.base.jmod`. Creating a `liblauncher.a` with the `main.o` and including the `liblauncher.a` in `java.base.jmod` seems to be cleaner than including `main.o` directly in the jmod.
When jlinking the final image for the application, if user doesn't provide customized launcher, `liblauncher.a` is used with other needed JDK/VM static libraries for linking the executable using the native linker (e.g. `ld`). If customized launcher is used, the object file(s) or the static library for the customized launcher is then used when linking the executable. `liblauncher.a` is not used in that case.
Should we really, though? That includes a `main` function. If the user wants to use the static libs from the jmod to create their own application, that will cause a symbol conflict.
I think that worth a discussion, IIRC, the most needed processing should be in libjli already. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1996036735
On Fri, 14 Mar 2025 18:02:17 GMT, Henry Jen <henryjen@openjdk.org> wrote:
Should we really, though? That includes a `main` function. If the user wants to use the static libs from the jmod to create their own application, that will cause a symbol conflict.
I think that worth a discussion, IIRC, the most needed processing should be in libjli already.
Yes, main.c is basically a thin wrapper that parses command line arguments and calls `JLI_Launch`. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1996047658
On Fri, 14 Mar 2025 18:14:04 GMT, Magnus Ihse Bursie <ihse@openjdk.org> wrote:
make/CreateJmods.gmk line 104:
102: endif 103: 104: # Copy hotspot libs, currently only libjvm.a
I just realized that we also need the launcher's `main.o` in the static `java.base.jmod`. Creating a `liblauncher.a` with the `main.o` and including the `liblauncher.a` in `java.base.jmod` seems to be cleaner than including `main.o` directly in the jmod.
When jlinking the final image for the application, if user doesn't provide customized launcher, `liblauncher.a` is used with other needed JDK/VM static libraries for linking the executable using the native linker (e.g. `ld`). If customized launcher is used, the object file(s) or the static library for the customized launcher is then used when linking the executable. `liblauncher.a` is not used in that case.
Sorry @jianglizhou, I did not properly read the second part of your suggestion, to create a `liblauncher.o` out of `main.o`. Well, tbh, this is just JDK-8351367 all over again. If we should include "optional" libraries in the jmods, we need to figure out first how to handle them. How should they be marked/classified as optional or required? How should the user tell jlink to include or exclude them? If we do get a good answer to that question, then we can introduce an optional `liblauncher.a` as well alongside `libjsig.a`. But until then, we should just stay clear of "optional" static libraries.
I think for common cases, there would be no C/C++ `main()` function provided from user code with Java applications. The JDK `java` launcher's `main()` is the C/C++ entry point. So we do need to provide a static library with the C/C++ `main` function for linking the executable. With the customized launcher cases, the C/C++ `main()` is implemented by the customized launcher. Then, the JDK provided `main` entry point is not needed. @magicus, yes, this involves with the optional static library in the jmods, as you pointed out. I think for C/C++ `main`, supporting that is necessary. An alternative is to require Java application implement C/C++ `main()`. That doesn't seem to be an applaudable approach. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1996151267
On Fri, 14 Mar 2025 19:20:40 GMT, Jiangli Zhou <jiangli@openjdk.org> wrote:
Sorry @jianglizhou, I did not properly read the second part of your suggestion, to create a `liblauncher.o` out of `main.o`. Well, tbh, this is just JDK-8351367 all over again. If we should include "optional" libraries in the jmods, we need to figure out first how to handle them. How should they be marked/classified as optional or required? How should the user tell jlink to include or exclude them? If we do get a good answer to that question, then we can introduce an optional `liblauncher.a` as well alongside `libjsig.a`. But until then, we should just stay clear of "optional" static libraries.
I think for common cases, there would be no C/C++ `main()` function provided from user code with Java applications. The JDK `java` launcher's `main()` is the C/C++ entry point. So we do need to provide a static library with the C/C++ `main` function for linking the executable.
With the customized launcher cases, the C/C++ `main()` is implemented by the customized launcher. Then, the JDK provided `main` entry point is not needed.
@magicus, yes, this involves with the optional static library in the jmods, as you pointed out. I think for C/C++ `main`, supporting that is necessary.
An alternative is to require Java application implement C/C++ `main()`. That doesn't seem to be an applaudable approach.
I would prefer that thin wrapper is to be provided by the end tools(like jlink) which use the jmod to link an executable. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1996336751
On Fri, 14 Mar 2025 21:57:23 GMT, Henry Jen <henryjen@openjdk.org> wrote:
I would prefer that thin wrapper is to be provided by the end tools(like jlink) which use the jmod to link an executable.
@slowhog Can you provide details on that? Do you suggest there would be a separate executable that provides the C/C++ `main` entry point as a thin wrapper? The binary that statically linked with the `libjvm.a`, `libjli.a` and etc would contain no C/C++ `main`? Is that binary linked as a shred library? Or, your proposal is to generate some C/C++ code with the `main()` function using jlink, then compile the C/C++ code and use that to link the executable? The issue that we want to solve here is to provide the C/C++ `main()` entry point for the executable for the final image. ------------- PR Review Comment: https://git.openjdk.org/leyden/pull/46#discussion_r1996411727
On Fri, 14 Mar 2025 04:42:49 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision:
Add hotspot static libs to java.base-static-jmod
However, regarding manifests, there *is* one thing that really should be included with the static libs, and that is a `pkg-config` file, that tells the end users what linker flags to use to be able to properly use the static libraries. I have been planning to add this to the static-jdk-image but never gotten around to it. (Can't find the JBS number now either.) But I guess that there is no real conflict between that and this PR; if I just plop the file down alongside the .a files, it will be automatically picked up by jmod. ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2725408153
On Fri, 14 Mar 2025 04:42:49 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision:
Add hotspot static libs to java.base-static-jmod
We agreed that to add a liblauncher to facilitate the `main` entry point. However, as I mentioned before, this is really a depends on situation and should be answered at link time. One obvious question is how do we deliver javaw(with WinMain) symbol for Windows? Also all the tunable configuration at build time such as MAIN_CLASS and JAVA_ARGS for an executable? Or are we simply thinking of `java` here? ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2814148602
On Thu, 17 Apr 2025 22:42:10 GMT, Henry Jen <henryjen@openjdk.org> wrote:
We agreed that to add a liblauncher to facilitate the `main` entry point. However, as I mentioned before, this is really a depends on situation and should be answered at link time. One obvious question is how do we deliver javaw(with WinMain) symbol for Windows?
Also all the tunable configuration at build time such as MAIN_CLASS and JAVA_ARGS for an executable? Or are we simply thinking of `java` here?
The default Java launcher `main.o` (and packaged as `liblauncher` static library) is produced by https://github.com/openjdk/jdk/blob/ad7c475fb1e23f583a33d58f0bd73ea0fb56740c...: $(eval $(call SetupBuildLauncher, java, \ ENABLE_ARG_FILES := true, \ EXPAND_CLASSPATH_WILDCARDS := true, \ EXTRA_RCFLAGS := $(JAVA_RCFLAGS), \ VERSION_INFO_RESOURCE := $(JAVA_VERSION_INFO_RESOURCE), \ OPTIMIZATION := HIGH, \ )) That should be used to link the launcher executable when jlink creates the final image, if no custom launcher is used. With the https://github.com/openjdk/leyden/blob/hermetic-java-runtime/src/jdk.jlink/s... prototype, we could pass additional linking flags to to form the complete native linking command (when integrating with the static .jmod support). With with that, I think we can pass the default `liblauncher.a` as part of the extra linking flags. Then, that may help simplify things for us by not having to include `liblauncher` in `java.base` static .jmod and not have to deal with optionally using the default `liblauncher` from the .jmod for linking. We could go with that. The default launcher and custom launcher would be handled similarly then, by passing via the extra linking flag. ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2814220035
On Thu, 17 Apr 2025 23:56:07 GMT, Jiangli Zhou <jiangli@openjdk.org> wrote:
The default Java launcher `main.o` (and packaged as `liblauncher` static library) is produced by https://github.com/openjdk/jdk/blob/ad7c475fb1e23f583a33d58f0bd73ea0fb56740c...:
``` $(eval $(call SetupBuildLauncher, java, \ ENABLE_ARG_FILES := true, \ EXPAND_CLASSPATH_WILDCARDS := true, \ EXTRA_RCFLAGS := $(JAVA_RCFLAGS), \ VERSION_INFO_RESOURCE := $(JAVA_VERSION_INFO_RESOURCE), \ OPTIMIZATION := HIGH, \ )) ```
That should be used to link the launcher executable when jlink creates the final image, if no custom launcher is used.
With the https://github.com/openjdk/leyden/blob/hermetic-java-runtime/src/jdk.jlink/s... prototype, we could pass additional linking flags to to form the complete native linking command (when integrating with the static .jmod support). With with that, I think we can pass the default `liblauncher.a` as part of the extra linking flags. Then, that may help simplify things for us by not having to include `liblauncher` in `java.base` static .jmod and not have to deal with optionally using the default `liblauncher` from the .jmod for linking. We could go with that.
The default launcher and custom launcher would be handled similarly then, by passing via the extra linking flag.
Yes, that's just for the `java` launcher. If that's the only executable for java.base static mod targeted for. `javaw` would be another executable in java.base, so we will need another one for that. Regard to other executable, this approach would expect they would need to build a separate lib in their own corresponding static jmod to link with. ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2814230978
On Fri, 18 Apr 2025 00:09:14 GMT, Henry Jen <henryjen@openjdk.org> wrote:
The default Java launcher `main.o` (and packaged as `liblauncher` static library) is produced by https://github.com/openjdk/jdk/blob/ad7c475fb1e23f583a33d58f0bd73ea0fb56740c...: ``` $(eval $(call SetupBuildLauncher, java, \ ENABLE_ARG_FILES := true, \ EXPAND_CLASSPATH_WILDCARDS := true, \ EXTRA_RCFLAGS := $(JAVA_RCFLAGS), \ VERSION_INFO_RESOURCE := $(JAVA_VERSION_INFO_RESOURCE), \ OPTIMIZATION := HIGH, \ )) ```
That should be used to link the launcher executable when jlink creates the final image, if no custom launcher is used. With the https://github.com/openjdk/leyden/blob/hermetic-java-runtime/src/jdk.jlink/s... prototype, we could pass additional linking flags to to form the complete native linking command (when integrating with the static .jmod support). With with that, I think we can pass the default `liblauncher.a` as part of the extra linking flags. Then, that may help simplify things for us by not having to include `liblauncher` in `java.base` static .jmod and not have to deal with optionally using the default `liblauncher` from the .jmod for linking. We could go with that. The default launcher and custom launcher would be handled similarly then, by passing via the extra linking flag.
Yes, that's just for the `java` launcher. If that's the only executable for java.base static mod targeted for. `javaw` would be another executable in java.base, so we will need another one for that.
`javaw` is Windows only, right? It would only need to be included for Windows build.
Regard to other executable, this approach would expect they would need to build a separate lib in their own corresponding static jmod to link with.
Yes. Or they could just directly pass the user supplied static libraries or .o files to jlink via the extra linking flag. ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2814259013
On Fri, 14 Mar 2025 04:42:49 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision:
Add hotspot static libs to java.base-static-jmod
Marked as reviewed by rpressler (Committer). ------------- PR Review: https://git.openjdk.org/leyden/pull/46#pullrequestreview-2784824577
On Fri, 14 Mar 2025 04:42:49 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision:
Add hotspot static libs to java.base-static-jmod
Marked as reviewed by sgehwolf (Reviewer). ------------- PR Review: https://git.openjdk.org/leyden/pull/46#pullrequestreview-2784828103
On Fri, 14 Mar 2025 04:42:49 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
Henry Jen has updated the pull request incrementally with one additional commit since the last revision:
Add hotspot static libs to java.base-static-jmod
I'm looking to using the static libs from the static jmods for linking launcher executable. I just pushed https://github.com/openjdk/leyden/commit/664d3070dcc8040dacbb2adc20245e9cd58... with following changes for static jmods. Do we want to start review process for all changes to the leyden/hermetic-java-runtime branch now? - Build static-jmods as a dependency for static-jdk. - Copy the static jmods into static-jdk/jmods, instead of copying from the regular JDK. The shared libraries (`.so`) are still included in the static jmods. I think only the static libraries (`.a`) are needed in the static jmods. $ jmod list ./linux-x86_64-server-fastdebug/images/static-jdk/jmods/java.base.jmod | grep libjvm lib/server/libjvm.so static-lib/libjvm.a ------------- PR Comment: https://git.openjdk.org/leyden/pull/46#issuecomment-2892431418
On Tue, 4 Mar 2025 02:56:54 GMT, Henry Jen <henryjen@openjdk.org> wrote:
This PR add --static-libs option to the jmod tool, it's basically mirroring --libs option to support adding a new section of static archive into the jmod file under the new section `static-lib`.
The JMOD magic header contains a MAJOR and MINOR version, we bump up the MINOR version if the --static-libs option is specified; otherwise, keep the MINOR version as 1. This allow created JMOD file without the newly supported static-lib section to continue be consumed by earlier version of jmod tool.
Also fix the formatting for _hermetic_jdk_jimage_offset, which is a julong(uint64_t) as unsigned long long on MacOS, thus need to use %llu.
This pull request has now been integrated. Changeset: 4180d204 Author: Henry Jen <henryjen@openjdk.org> URL: https://git.openjdk.org/leyden/commit/4180d204b6b7ee5f7fe96591a007b80a751318... Stats: 176 lines in 11 files changed: 147 ins; 0 del; 29 mod 8347755: Support static library in jmod Reviewed-by: rpressler, sgehwolf ------------- PR: https://git.openjdk.org/leyden/pull/46
participants (7)
-
Alan Bateman
-
Ethan McCue
-
Henry Jen
-
Jiangli Zhou
-
Magnus Ihse Bursie
-
Ron Pressler
-
Severin Gehwolf