RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v8]
Severin Gehwolf
sgehwolf at openjdk.org
Thu Nov 23 18:30:07 UTC 2023
On Thu, 16 Nov 2023 19:00:57 GMT, Mandy Chung <mchung at openjdk.org> wrote:
> It'd be helpful if you write down your proposed behavior for each plugin for discussion. Otherwise, we will have to find out from the code.
Here it goes. This was a *very* useful exercise and uncovered some bugs. Fundamentally, each plugin's `Category` is what I was looking at.
Unique categories (for pre-existing plug-ins):
--------------------------------
- FILTER
- TRANSFORMER
- ADDER
- METADATA_ADDER
- SORTER
- PROCESSOR
- COMPRESSOR
SORTER, PROCESSOR and COMPRESSOR. Those plugins affect the final image only so
they work the same as when using packaged modules. There is nothing to leak
from a previous link as the output directory will contain a new JDK image. Note
that classes.jsa and classes_nocoops.jsa are not part of the jmod files, nor
are they classes or resources in the jimage. Thus, they are not tracked with
`add-run-image-resources` and don't end up in the final JDK image when performing
a run-time based link.
ADDER, METADATA_ADDER plugins add files to the final jimage and/or JDK image
('release' file), which aren't there without those plugins. They get appropriately
filtered at link time, so there isn't any oberservable difference. Exception:
save-jlink-arg-file. The arg files are processed too early so they carry forward
for the link, but not into the final image of that link. Therefore I've marked them
as equivalent.
FILTER, TRANSFORMER. Those change class files or native resources or exclude
specific resources. Since those changes are then in effect in the resulting
runtime image, and the link is performed from such an image, they carry over to
the derived image. Certain transformers can be overridden by the final run-time
image based link (e.g. vendor-version plugin). Filters remove things from the
image, so those changes carry forward. Exceptions: system-modules and
generate-jli-classes. Both behave the same as in packaged-modules case.
For each individual plug-in the planned behaviour is in the "Current behaviour in run-time image link mode" column.
Plugin name | Description | Current behaviour in run-time | Notes
| | image link mode |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
add-options |Prepend the specified <options> string, which may | Same as with packaged-modules. | Category: ADDER
|include whitespace, before any other options when | /java.base/jdk/internal/vm/ |
|invoking the virtual machine in the resulting | options file is being filtered.|
|image. | | Adds a file in jimage:
| | | /java.base/jdk/internal/vm/options
-------------------------------------------------------------------------------------------------------------------------------------------------------------
compress |Compression to use in compressing resources: | Same as with packaged-modules | Category: COMPRESSOR
|Accepted values are: | |
|zip-[0-9], where zip-0 provides no compression, | | Combination of 'zip'
|and zip-9 provides the best compression. | | and 'compact-cp'
|Default is zip-6. | |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
dedup-legal-notices |De-duplicate all legal notices. | Accumulate with base run-time |
|If error-if-not-same-content is specified then | image (if any). |Category: TRANSFORMER
|it will be an error if two files of the same | |
|filename are different. | |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
exclude-files | | Accumulate with base run-time | Category: FILTER
| Specify files to exclude. | image (if any). |
| e.g.: **.java,glob:/java.base/lib/client/** | |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
exclude-jmod-section | | Accumulate with base run-time | Category: FILTER
|Specify a JMOD section to exclude. | image (if any). |
|Where <section-name> is "man" or "headers". | | Excludes resources of type man/header
-------------------------------------------------------------------------------------------------------------------------------------------------------------
exclude-resources |Specify resources to exclude. | Accumulate with base run-time | Category: FILTER
|e.g.: **.jcov,glob:**/META-INF/** | image (if any). |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
generate-cds-archive |Generate CDS archive if the runtime image supports | Same as with packaged-modules | Category: PROCESSOR
|the CDS feature. | | Generates classes.jsa classes_nocoops.jsa
| | | in $JAVA_HOME
-------------------------------------------------------------------------------------------------------------------------------------------------------------
generate-jli-classes |Specify a file listing the java.lang.invoke | Same as with packaged-modules | Category: TRANSFORMER
|classes to pre-generate. By default, this plugin | |
|may use a builtin list of classes to pre-generate. | Potential existing BMH.Species |
|If this plugin runs on a different runtime version | classes are filtered to match | Generates BoundMethodHandle$Species*
|than the image being created then code generation | java.base jmod | classes.
|will be disabled by default to guarantee | |
|correctness add ignore-version=true | |
|to override this. | |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
include-locales |BCP 47 language tags separated by a comma, | Accumulate with base run-time | Category: FILTER
|allowing | image (if any). |
|locale matching defined in RFC 4647. | | Filters specific locales in the target
|e.g.: en,ja,*-IN | | image.
-------------------------------------------------------------------------------------------------------------------------------------------------------------
order-resources |Order resources. | Same as with packaged-modules | Category: SORTER
|e.g.: **/module-info.class, at classlist, | |
|/java.base/java/lang/** | |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
release-info |<file> option is to load release properties from | Same as with packaged-modules | Category: METAINFO_ADDER
|the supplied file. | |
|add: is to add properties to the release file. | | Adds a file 'release' to the final image
|Any number of <key>=<value> pairs can be passed. | | in the top folder. Since that file is not
|del: is to delete the list of keys in release file. | | in packaged modules nor in the jimage,
| | | it doesn't get carried forward
-------------------------------------------------------------------------------------------------------------------------------------------------------------
save-jlink-argfiles |Save the specified argument files that contain | If specified, options in an | Category: ADDER
|the arguments to be prepended to the execution of | existing file are replaced by |
|jlink in the output image. <filenames> is a | the new options. Otherwise, | Adds a file to the jdk.jlink module,
|: (; on Windows) separated list of command-line | options from the base image | /jdk.jlink/jdk/tools/jlink/internal/options
|argument files. | affect the run, but don't |
| | carry forward. |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
strip-debug |Strip debug information from the output image | Accumulate with base run-time | Category: TRANSFORMER
| | image (if any). | 'strip-java-debug-attributes' and
| | | 'strip-native-debug-symbols' combined
-------------------------------------------------------------------------------------------------------------------------------------------------------------
strip-java-debug-attributes |Strip Java debug attributes from | Accumulate with base run-time | Category: TRANSFORMER
|classes in the output image | image (if any). |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
strip-native-commands |Exclude native commands (such as java/java.exe) | Accumulate with base run-time | Category: FILTER
|from the image. | image (if any). |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
strip-native-debug-symbols |Strip debug symbols from native libraries (if any). | Accumulate with base run-time | Category: TRANSFORMER
|This plugin requires at least one option: | image (if any). |
| objcopy: The path to the objcopy binary. | |
| Defaults to objcopy in PATH. | |
| exclude-debuginfo-files: Exclude debug info | |
| files. Defaults to true. | |
| keep-debuginfo-files[=<ext>]: Keep debug info | |
| files in <file>.<ext>. | |
| Defaults to <file>.debuginfo | |
|Examples: --strip-native-debug-symbols | |
| keep-debuginfo-files:objcopy=OBJPATH | |
| --strip-native-debug-symbols | |
| exclude-debuginfo-files | |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
system-modules |The batch size specifies the maximum number of | Same as with packaged modules. | Category: TRANSFORMER
|modules be handled in one method to workaround if | |
|the generated bytecode exceeds the method size | Generated classes get filtered | Changes SystemModulesMap.class and adds
|limit. The default batch size is 75. | so they don't populate. | SystemModules${<N>,all,default}.class
-------------------------------------------------------------------------------------------------------------------------------------------------------------
vendor-bug-url |Override the vendor bug URL baked into the build. | Accumulate with base run-time | Category: TRANSFORMER
|The value of the system property | image (if any). |
|"java.vendor.url.bug" will be <vendor-url-bug>. | |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
vendor-version |Override the vendor version string baked into the | Accumulate with base run-time | Category: TRANSFORMER
|build,if any. The value of the system property | image (if any). |
|"java.vendor.version" will be <vendor-version>. | |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
vendor-vm-bug-url |Override the vendor VM bug URL baked | Accumulate with base run-time | Category: TRANSFORMER
|into the build. The URL displayed in VM error | image (if any). |
|logs will be <vendor-vm-bug-url>. | |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
vm |Select the HotSpot VM in the output image. | Accumulate with base run-time | Category: FILTER
|Default is all | image (if any). |
-------------------------------------------------------------------------------------------------------------------------------------------------------------
> Regarding the name of the `runimage_resources` file, this file is only used by jlink and so it can be a resource file in `jdk.jlink` module like `jdk.jlink/jdk/tools/jlink/internal/runtime/$MODULE_resources` (e.g. `jdk.jlink/jdk/tools/jlink/internal/runtime/java.base_resources`)
That's a good observation. Thanks. I've modified the patch and CSR to mention that. Those resource files now live in `/jdk.jlink/jdk/tools/jlink/internal/runlink_$MODULE_resources`.
The option `--unlock-run-image` to get equivalence between a link using packaged modules and run-time based link is now hidden (similar to `--keep-packaged-modules`). This is really advanced usage and doesn't need a prominent place in `jlink --help`.
Other things of note:
------------------
`jlink` orders plugin's by their `Category`. So in general, the processing pipeline is: `FILTER->ADDER->TRANSFORMER->MODULEINFO_TRANSFORMER->SORTER->METAINFO_ADDER->COMPRESSOR->VERIFIER->PROCESSOR->PACKAGER`. In this updated patch I've used this info in order to re-establish the invariant of certain classes and resources not being in packaged modules. Things like `SystemModules$*.class` and `BMH$Species_*.class`. Those are now being filtered (using `exclude-resource`) prior it being processed by the `TRANSFORMER` or `ADDER` phases. Furthermore, it prevents those classes to populate if plug-ins that generated them aren't used. This also removes the need to actually modify any of the existing plugins.
More thoughts?
-------------
PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-1824804506
More information about the core-libs-dev
mailing list