RFR: 8231640: (prop) Canonical property storage

Jaikiran Pai jpai at openjdk.java.net
Sun Sep 5 12:59:54 UTC 2021

On Sat, 4 Sep 2021 15:25:59 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> The commit in this PR implements the proposal for enhancement that was discussed in the core-libs-dev mailing list recently[1], for https://bugs.openjdk.java.net/browse/JDK-8231640
> At a high level - the `store()` APIs in `Properties` have been modified to now look for the `SOURCE_DATE_EPOCH` environment variable[2]. If that env variable is set, then instead of writing out the current date time as a date comment, the `store()` APIs instead will use the value set for this env variable to parse it to a `Date` and write out the string form of such a date. The implementation here uses the `d MMM yyyy HH:mm:ss 'GMT'` date format and `Locale.ROOT` to format and write out such a date. This should provide reproducibility whenever the `SOURCE_DATE_EPOCH` is set. Furthermore, intentionally, no changes in the date format of the "current date" have been done.
> These  modified `store()` APIs work in the presence of the `SecurityManager` too. The caller is expected to have a read permission on the `SOURCE_DATE_EPOCH` environment variable. If the caller doesn't have that permission, then the implementation of these `store()` APIs will write out the "current date" and will ignore any value that has been set for the `SOURCE_DATE_EPOCH` env variable. This should allow for backward compatibility of existing applications, where, when they run under a `SecurityManager` and perhaps with an existing restrictive policy file, the presence of `SOURCE_DATE_EPOCH` shouldn't impact their calls to the `store()` APIs.
> The modified `store()` APIs will also ignore any value for `SOURCE_DATE_EPOCH` that  cannot be parsed to an `long` value. In such cases, the `store()` APIs will write out the "current date" and ignore the value set for this environment variable. No exceptions will be thrown for such invalid values. This is an additional backward compatibility precaution to prevent any rogue value for `SOURCE_DATE_EPOCH` from breaking applications.
> An additional change in the implementation of these `store()` APIs and unrelated to the date comment, is that these APIs will now write out the property keys in a deterministic order. The keys will be written out in the natural ordering as specified by `java.lang.String#compareTo()` API.
> The combination of the ordering of the property keys when written out and the usage of `SOURCE_DATE_EPOCH` environment value to determine the date comment should together allow for reproducibility of the output generated by these `store()` APIs.
> New jtreg test classes have been introduced to verify these changes. The primary focus of `PropertiesStoreTest` is the ordering aspects of the property keys that are written out. On the other hand `StoreReproducibilityTest` focuses on the reproducibility of the output generated by these APIs.  The `StoreReproducibilityTest` runs these tests both in the presence and absence of `SecurityManager`. Plus, in the presence of SecurityManager, it tests both the scenarios where the caller is granted the requisite permission and in other case not granted that permission.
> These new tests and existing tests under `test/jdk/java/util/Properties/` pass with these changes.
> [1] https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-August/080758.html
> [2] https://reproducible-builds.org/specs/source-date-epoch/

Off topic - the `tier1` failures that are being reported in GitHub action jobs seem genuine but unrelated to this change. The failure is consistently in a single test `InfoOptsTest` and it fails with the following error when running the `testUniqueInfoOpts` test case. Looks like a recent issue in some commit. I'll sort that out separately.
Here's the error from the logs:

2021-09-04T16:37:24.7528210Z Running test testUniqueInfoOpts
2021-09-04T16:37:24.7630470Z Main [--help, --help] []
2021-09-04T16:37:24.7635480Z rc:0
2021-09-04T16:37:24.7736940Z javac/DIRECT:
2021-09-04T16:37:24.7762440Z Usage: javac <options> <source files>
2021-09-04T16:37:24.7863980Z where possible options include:
2021-09-04T16:37:24.7892480Z   @<filename>                  Read options and filenames from file
2021-09-04T16:37:24.7986700Z   -Akey[=value]                Options to pass to annotation processors
2021-09-04T16:37:24.8003100Z   --add-modules <module>(,<module>)*
2021-09-04T16:37:24.8030720Z         Root modules to resolve in addition to the initial modules, or all modules
2021-09-04T16:37:24.8106550Z         on the module path if <module> is ALL-MODULE-PATH.
2021-09-04T16:37:24.8108330Z   --boot-class-path <path>, -bootclasspath <path>
2021-09-04T16:37:24.8110830Z         Override location of bootstrap class files
2021-09-04T16:37:24.8112470Z   --class-path <path>, -classpath <path>, -cp <path>
2021-09-04T16:37:24.8113590Z         Specify where to find user class files and annotation processors
2021-09-04T16:37:24.8115140Z   -d <directory>               Specify where to place generated class files
2021-09-04T16:37:24.8116530Z   -deprecation
2021-09-04T16:37:24.8117490Z         Output source locations where deprecated APIs are used
2021-09-04T16:37:24.8118770Z   --enable-preview
2021-09-04T16:37:24.8120330Z         Enable preview language features. To be used in conjunction with either -source or --release.
2021-09-04T16:37:24.8129810Z   -encoding <encoding>         Specify character encoding used by source files
2021-09-04T16:37:24.8136440Z   -endorseddirs <dirs>         Override location of endorsed standards path
2021-09-04T16:37:24.8239190Z   -extdirs <dirs>              Override location of installed extensions
2021-09-04T16:37:24.8251400Z   -g                           Generate all debugging info
2021-09-04T16:37:24.8290250Z   -g:{lines,vars,source}       Generate only some debugging info
2021-09-04T16:37:24.8291910Z   -g:none                      Generate no debugging info
2021-09-04T16:37:24.8293200Z   -h <directory>
2021-09-04T16:37:24.8294170Z         Specify where to place generated native header files
2021-09-04T16:37:24.8296270Z   --help, -help, -?            Print this help message
2021-09-04T16:37:24.8297720Z   --help-extra, -X             Print help on extra options
2021-09-04T16:37:24.8299160Z   -implicit:{none,class}
2021-09-04T16:37:24.8300250Z         Specify whether or not to generate class files for implicitly referenced files
2021-09-04T16:37:24.8301820Z   -J<flag>                     Pass <flag> directly to the runtime system
2021-09-04T16:37:24.8303230Z   --limit-modules <module>(,<module>)*
2021-09-04T16:37:24.8304210Z         Limit the universe of observable modules
2021-09-04T16:37:24.8305570Z   --module <module>(,<module>)*, -m <module>(,<module>)*
2021-09-04T16:37:24.8306630Z         Compile only the specified module(s), check timestamps
2021-09-04T16:37:24.8307980Z   --module-path <path>, -p <path>
2021-09-04T16:37:24.8309050Z         Specify where to find application modules
2021-09-04T16:37:24.8310500Z   --module-source-path <module-source-path>
2021-09-04T16:37:24.8311660Z         Specify where to find input source files for multiple modules
2021-09-04T16:37:24.8313050Z   --module-version <version>
2021-09-04T16:37:24.8314040Z         Specify version of modules that are being compiled
2021-09-04T16:37:24.8315390Z   -nowarn                      Generate no warnings
2021-09-04T16:37:24.8316640Z   -parameters
2021-09-04T16:37:24.8317620Z         Generate metadata for reflection on method parameters
2021-09-04T16:37:24.8361300Z   -proc:{none,only}
2021-09-04T16:37:24.8362510Z         Control whether annotation processing and/or compilation is done.
2021-09-04T16:37:24.8364150Z   -processor <class1>[,<class2>,<class3>...]
2021-09-04T16:37:24.8365410Z         Names of the annotation processors to run; bypasses default discovery process
2021-09-04T16:37:24.8366910Z   --processor-module-path <path>
2021-09-04T16:37:24.8367990Z         Specify a module path where to find annotation processors
2021-09-04T16:37:24.8369540Z   --processor-path <path>, -processorpath <path>
2021-09-04T16:37:24.8370620Z         Specify where to find annotation processors
2021-09-04T16:37:24.8371840Z   -profile <profile>
2021-09-04T16:37:24.8372800Z         Check that API used is available in the specified profile
2021-09-04T16:37:24.8374080Z   --release <release>
2021-09-04T16:37:24.8375110Z         Compile for the specified Java SE release. Supported releases: 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18
2021-09-04T16:37:24.8377060Z   -s <directory>               Specify where to place generated source files
2021-09-04T16:37:24.8378940Z   --source <release>, -source <release>
2021-09-04T16:37:24.8380150Z         Provide source compatibility with the specified Java SE release. Supported releases: 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18
2021-09-04T16:37:24.8381690Z   --source-path <path>, -sourcepath <path>
2021-09-04T16:37:24.8382670Z         Specify where to find input source files
2021-09-04T16:37:24.8383990Z   --system <jdk>|none          Override location of system modules
2021-09-04T16:37:24.8385370Z   --target <release>, -target <release>
2021-09-04T16:37:24.8386570Z         Generate class files suitable for the specified Java SE release. Supported releases: 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18
2021-09-04T16:37:24.8388070Z   --upgrade-module-path <path>
2021-09-04T16:37:24.8389040Z         Override location of upgradeable modules
2021-09-04T16:37:24.8390480Z   -verbose                     Output messages about what the compiler is doing
2021-09-04T16:37:24.8391930Z   --version, -version          Version information
2021-09-04T16:37:24.8393380Z   -Werror                      Terminate compilation if warnings occur
2021-09-04T16:37:24.8395160Z Main [-X, -X] []
2021-09-04T16:37:24.8395860Z rc:0
2021-09-04T16:37:24.8396520Z javac/DIRECT:
2021-09-04T16:37:24.8397830Z   --add-exports <module>/<package>=<other-module>(,<other-module>)*
2021-09-04T16:37:24.8399140Z         Specify a package to be considered as exported from its defining module
2021-09-04T16:37:24.8400780Z         to additional modules, or to all unnamed modules if <other-module> is ALL-UNNAMED.
2021-09-04T16:37:24.8403050Z   --add-reads <module>=<other-module>(,<other-module>)*
2021-09-04T16:37:24.8404200Z         Specify additional modules to be considered as required by a given module.
2021-09-04T16:37:24.8405750Z         <other-module> may be ALL-UNNAMED to require the unnamed module.
2021-09-04T16:37:24.8407430Z   --default-module-for-created-files <module-name>
2021-09-04T16:37:24.8408890Z         Fallback target module for files created by annotation processors, if none specified or inferred.
2021-09-04T16:37:24.8410640Z   -Djava.endorsed.dirs=<dirs>  Override location of endorsed standards path
2021-09-04T16:37:24.8412280Z   -Djava.ext.dirs=<dirs>       Override location of installed extensions
2021-09-04T16:37:24.8413820Z   --help-lint                  Print the supported keys for -Xlint
2021-09-04T16:37:24.8415200Z   --patch-module <module>=<file>(:<file>)*
2021-09-04T16:37:24.8416210Z         Override or augment a module with classes and resources
2021-09-04T16:37:24.8417110Z         in JAR files or directories
2021-09-04T16:37:24.8418510Z   -Xbootclasspath:<path>       Override location of bootstrap class files
2021-09-04T16:37:24.8420080Z   -Xbootclasspath/a:<path>     Append to the bootstrap class path
2021-09-04T16:37:24.8421630Z   -Xbootclasspath/p:<path>     Prepend to the bootstrap class path
2021-09-04T16:37:24.8423170Z   -Xdiags:{compact,verbose}    Select a diagnostic mode
2021-09-04T16:37:24.8424430Z   -Xdoclint
2021-09-04T16:37:24.8425370Z         Enable recommended checks for problems in javadoc comments
2021-09-04T16:37:24.8426730Z   -Xdoclint:(all|none|[-]<group>)[/<access>]
2021-09-04T16:37:24.8427770Z         Enable or disable specific checks for problems in javadoc comments,
2021-09-04T16:37:24.8428920Z         where <group> is one of accessibility, html, missing, reference, or syntax,
2021-09-04T16:37:24.8430170Z         and <access> is one of public, protected, package, or private.
2021-09-04T16:37:24.8431760Z   -Xdoclint/package:[-]<packages>(,[-]<package>)*
2021-09-04T16:37:24.8432880Z         Enable or disable checks in specific packages. Each <package> is either the
2021-09-04T16:37:24.8434310Z         qualified name of a package or a package name prefix followed by .*, which
2021-09-04T16:37:24.8435940Z         expands to all sub-packages of the given package. Each <package> can be prefixed
2021-09-04T16:37:24.8438060Z         with - to disable checks for the specified package or packages.
2021-09-04T16:37:24.8439610Z   -Xlint                       Enable recommended warnings
2021-09-04T16:37:24.8440840Z   -Xlint:<key>(,<key>)*
2021-09-04T16:37:24.8441750Z         Warnings to enable or disable, separated by comma.
2021-09-04T16:37:24.8443150Z         Precede a key by - to disable the specified warning.
2021-09-04T16:37:24.8444570Z         Use --help-lint to see the supported keys.
2021-09-04T16:37:24.8446010Z   -Xmaxerrs <number>           Set the maximum number of errors to print
2021-09-04T16:37:24.8447550Z   -Xmaxwarns <number>          Set the maximum number of warnings to print
2021-09-04T16:37:24.8550380Z   -Xpkginfo:{always,legacy,nonempty}
2021-09-04T16:37:24.8636310Z         Specify handling of package-info files
2021-09-04T16:37:24.8639600Z   -Xplugin:"name args"
2021-09-04T16:37:24.8663470Z         Name and optional arguments for a plug-in to be run
2021-09-04T16:37:24.8665480Z   -Xprefer:{source,newer}
2021-09-04T16:37:24.8683700Z         Specify which file to read when both a source file and class file are found for an implicitly compiled class
2021-09-04T16:37:24.8728280Z   -Xprint
2021-09-04T16:37:24.8729520Z         Print out a textual representation of specified types
2021-09-04T16:37:24.8730900Z   -XprintProcessorInfo
2021-09-04T16:37:24.8731960Z         Print information about which annotations a processor is asked to process
2021-09-04T16:37:24.8733270Z   -XprintRounds
2021-09-04T16:37:24.8734200Z         Print information about rounds of annotation processing
2021-09-04T16:37:24.8736230Z   -Xstdout <filename>          Redirect standard output
2021-09-04T16:37:24.8737760Z These extra options are subject to change without notice.
2021-09-04T16:37:24.8739170Z Main [--help-lint, --help-lint] []
2021-09-04T16:37:24.8739960Z rc:0
2021-09-04T16:37:24.8740890Z javac/DIRECT:
2021-09-04T16:37:24.8742040Z The supported keys for -Xlint are:
2021-09-04T16:37:24.8742920Z     all                  Enable all warnings
2021-09-04T16:37:24.8743980Z     auxiliaryclass       Warn about an auxiliary class that is hidden in a source file, and is used from other files.
2021-09-04T16:37:24.8744800Z     cast                 Warn about use of unnecessary casts.
2021-09-04T16:37:24.8745500Z     classfile            Warn about issues related to classfile contents.
2021-09-04T16:37:24.8746200Z     deprecation          Warn about use of deprecated items.
2021-09-04T16:37:24.8747550Z     dep-ann              Warn about items marked as deprecated in JavaDoc but not using the @Deprecated annotation.
2021-09-04T16:37:24.8748410Z     divzero              Warn about division by constant integer 0.
2021-09-04T16:37:24.8749030Z     empty                Warn about empty statement after if.
2021-09-04T16:37:24.8749680Z     exports              Warn about issues regarding module exports.
2021-09-04T16:37:24.8750470Z     fallthrough          Warn about falling through from one case of a switch statement to the next.
2021-09-04T16:37:24.8751300Z     finally              Warn about finally clauses that do not terminate normally.
2021-09-04T16:37:24.8752780Z     missing-explicit-ctor Warn about missing explicit constructors in public and protected classes in exported packages.
2021-09-04T16:37:24.8753780Z     module               Warn about module system related issues.
2021-09-04T16:37:24.8754410Z     opens                Warn about issues regarding module opens.
2021-09-04T16:37:24.8755110Z     options              Warn about issues relating to use of command line options.
2021-09-04T16:37:24.8755850Z     overloads            Warn about issues regarding method overloads.
2021-09-04T16:37:24.8756560Z     overrides            Warn about issues regarding method overrides.
2021-09-04T16:37:24.8757270Z     path                 Warn about invalid path elements on the command line.
2021-09-04T16:37:24.8758010Z     processing           Warn about issues regarding annotation processing.
2021-09-04T16:37:24.8759040Z     rawtypes             Warn about use of raw types.
2021-09-04T16:37:24.8759730Z     removal              Warn about use of API that has been marked for removal.
2021-09-04T16:37:24.8761030Z     requires-automatic   Warn about use of automatic modules in the requires clauses.
2021-09-04T16:37:24.8762640Z     requires-transitive-automatic Warn about automatic modules in requires transitive.
2021-09-04T16:37:24.8763680Z     serial               Warn about Serializable classes that do not provide a serial version ID. 
2021-09-04T16:37:24.8765050Z                          Also warn about access to non-public members from a serializable element.
2021-09-04T16:37:24.8765870Z     static               Warn about accessing a static member using an instance.
2021-09-04T16:37:24.8766610Z     strictfp             Warn about unnecessary use of the strictfp modifier.
2021-09-04T16:37:24.8767930Z     synchronization      Warn about synchronization attempts on instances of value-based classes.
2021-09-04T16:37:24.8769460Z     text-blocks          Warn about inconsistent white space characters in text block indentation.
2021-09-04T16:37:24.8770810Z     try                  Warn about issues relating to use of try blocks (i.e. try-with-resources).
2021-09-04T16:37:24.8771580Z     unchecked            Warn about unchecked operations.
2021-09-04T16:37:24.8772250Z     varargs              Warn about potentially unsafe vararg methods.
2021-09-04T16:37:24.8772950Z     preview              Warn about use of preview language features.
2021-09-04T16:37:24.8773540Z     none                 Disable all warnings
2021-09-04T16:37:24.8774850Z Main [-version, -version] []
2021-09-04T16:37:24.8775370Z rc:0
2021-09-04T16:37:24.8775690Z javac/DIRECT:
2021-09-04T16:37:24.8776460Z javac 18-internal
2021-09-04T16:37:24.8777290Z Main [-fullversion, -fullversion] []
2021-09-04T16:37:24.8777740Z rc:0
2021-09-04T16:37:24.8778060Z javac/DIRECT:
2021-09-04T16:37:24.8779450Z javac full version "18-internal+0-jaikiran-85748cf4a8efb69cbe69667851a14321804a51b6"
2021-09-04T16:37:24.8780590Z >>>>> Expected string appears more than once: 18
2021-09-04T16:37:24.8781400Z java.lang.Exception: 1 errors occurred
2021-09-04T16:37:24.8782230Z 	at OptionModesTester.runTests(OptionModesTester.java:81)
2021-09-04T16:37:24.8783100Z 	at InfoOptsTest.main(InfoOptsTest.java:41)
2021-09-04T16:37:24.8784350Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2021-09-04T16:37:24.8786310Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
2021-09-04T16:37:24.8788620Z 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2021-09-04T16:37:24.8791250Z 	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
2021-09-04T16:37:24.8792700Z 	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:312)
2021-09-04T16:37:24.8794000Z 	at java.base/java.lang.Thread.run(Thread.java:833)
2021-09-04T16:37:24.8794980Z JavaTest Message: Test threw exception: java.lang.Exception
2021-09-04T16:37:24.8795660Z JavaTest Message: shutting down test
2021-09-04T16:37:24.8797640Z TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.Exception: 1 errors occurred


PR: https://git.openjdk.java.net/jdk/pull/5372

More information about the core-libs-dev mailing list