RFR: 8264449: Enable reproducible builds with SOURCE_DATE_EPOCH [v12]

[John Neffenger]

On Tue, 4 Apr 2023 15:46:20 GMT, John Neffenger <jgneff at openjdk.org> wrote:

>> This pull request allows for reproducible builds of JavaFX on Linux, macOS, and Windows by defining the `SOURCE_DATE_EPOCH` environment variable. For example, the following commands create a reproducible build:
>> 
>> 
>> $ export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)
>> $ bash gradlew sdk jmods javadoc
>> $ strip-nondeterminism -v -T $SOURCE_DATE_EPOCH build/jmods/*.jmod
>> 
>> 
>> The three commands:
>> 
>> 1. set the build timestamp to the date of the latest source code change,
>> 2. build the JavaFX SDK libraries, JMOD archives, and API documentation, and
>> 3. recreate the JMOD files with stable file modification times and ordering.
>> 
>> The third command won't be necessary once Gradle can build the JMOD archives or the `jmod` tool itself has the required support. For more information on the environment variable, see the [`SOURCE_DATE_EPOCH`][1] page. For more information on the command to recreate the JMOD files, see the [`strip-nondeterminism`][2] repository. I'd like to propose that we allow for reproducible builds in JavaFX 17 and consider making them the default in JavaFX 18.
>> 
>> #### Fixes
>> 
>> There are at least four sources of non-determinism in the JavaFX builds:
>> 
>> 1. Build timestamp
>> 
>>     The class `com.sun.javafx.runtime.VersionInfo` in the JavaFX Base module stores the time of the build. Furthermore, for builds that don't run on the Hudson continuous integration tool, the class adds the build time to the system property `javafx.runtime.version`.
>> 
>> 2. Modification times
>> 
>>     The JAR, JMOD, and ZIP archives store the modification time of each file.
>> 
>> 3. File ordering
>> 
>>     The JAR, JMOD, and ZIP archives store their files in the order returned by the file system. The native shared libraries also store their object files in the order returned by the file system. Most file systems, though, do not guarantee the order of a directory's file listing.
>> 
>> 4. Build path
>> 
>>     The class `com.sun.javafx.css.parser.Css2Bin` in the JavaFX Graphics module stores the absolute path of its `.css` input file in the corresponding `.bss` output file, which is then included in the JavaFX Controls module.
>> 
>> This pull request modifies the Gradle and Groovy build files to fix the first three sources of non-determinism. A later pull request can modify the Java files to fix the fourth.
>> 
>> [1]: https://reproducible-builds.org/docs/source-date-epoch/
>> [2]: https://salsa.debian.org/reproducible-builds/strip-nondeterminism
>
> John Neffenger has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 21 commits:
> 
>  - Merge branch 'master' into allow-reproducible-builds
>    
>    Include two commits that fix WebKit build failures on Windows and macOS:
>    
>      8282359: Intermittent WebKit build failure on Windows:
>               C1090: PDB API call failed, error code 23
>      8286089: Intermittent WebKit build failure on macOS in JavaScriptCore
>  - Merge branch 'master' into allow-reproducible-builds
>  - Support JDK 17 GA or later for building JavaFX
>  - Merge branch 'master' into allow-reproducible-builds
>  - Add '--date' argument for deterministic JMOD files
>  - Merge branch 'master' into allow-reproducible-builds
>  - Merge branch 'master' into allow-reproducible-builds
>  - Comment out 'jmod --date' until building on JDK 19
>    
>    Support for the 'jmod --date' option was added to JDK 19 starting
>    with the 19+2 early-access build, and it was backported to JDK 17
>    starting with release 17.0.3. It is not available in JDK 18.
>  - Merge 'master' into allow-reproducible-builds
>  - Make minimal changes for new '--date' option
>  - ... and 11 more: https://git.openjdk.org/jfx/compare/810bd90d...e42a0709

Thanks for finding this, Kevin. For my own reference, the format of the Java version string is explained in the API specification of [Runtime.Version][1], which defines the `$OPT` additional build information as matching the regular expression `([-a-zA-Z0-9.]+)`. No colon characters are permitted.

> We could consider a future RFE to adjust the format of the version string, although I note that the JDK also adds the date code to the version string only for developer builds.

The JDK [removed the timestamp][2] in version 9 for [JDK-8170632][3], so now they look something like `20-internal+36-adhoc.root.build`. More to your point, I now see that the JDK also adds additional information by default, just like JavaFX.

> No it doesn't. What it does mean is that they would need to set the following properties to the same values that were used for the official build:

Right. Sorry, I should have known that after all the "release" builds I do for this pull request!

I would like to replace the current non-standard build timestamp with one that conforms to ISO 8601. This pull request uses the ISO 8601 *extended* format, but the standard also defines a *basic* format that does result in a valid Java version `OPT` field. [The consensus][4] seems to be that both the date and time must use the same format type, either extended or basic, which makes the current JavaFX build timestamp non-standard.

[1]: https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/lang/Runtime.Version.html
[2]: https://github.com/openjdk/jdk/commit/e262ab65
[3]: https://bugs.openjdk.org/browse/JDK-8170632
[4]: https://stackoverflow.com/q/44870006

What do you think about simply replacing this pull request's extended format with the basic format?

See the following sample program for details:


import java.text.SimpleDateFormat;
import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneOffset;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;
import java.util.Date;

public class Timestamp {

    private static final String OPT = "([-a-zA-Z0-9.]+)";
    private static final String OK = " (okay)";
    private static final String NOT_OK = " (NOT okay)";

    public static void main(String[] args) {
        // ISO 8601 extended format
        var buildInstant = Instant.now().truncatedTo(ChronoUnit.SECONDS);
        String extended = buildInstant.toString();

        // Non-standard format used in the current JavaFX release
        var buildDate = Date.from(buildInstant);
        var format = new SimpleDateFormat("yyyy-MM-dd-HHmmss");
        String current = format.format(buildDate);

        // ISO 8601 basic format
        var localTime = LocalDateTime.ofInstant(buildInstant, ZoneOffset.UTC);
        var formatter = DateTimeFormatter.ofPattern("yyyyMMdd'T'HHmmss'Z'");
        String basic = localTime.format(formatter);

        System.out.print("ISO 8601 extended format = " + extended);
        System.out.println(extended.matches(OPT) ? OK : NOT_OK);

        System.out.print("Current JavaFX format = " + current);
        System.out.println(current.matches(OPT) ? OK : NOT_OK);

        System.out.print("ISO 8601 basic format = " + basic);
        System.out.println(basic.matches(OPT) ? OK : NOT_OK);
    }
}


An example of its output is shown below:


$ java Timestamp
ISO 8601 extended format = 2023-04-06T04:20:16Z (NOT okay)
Current JavaFX format = 2023-04-05-212016 (okay)
ISO 8601 basic format = 20230406T042016Z (okay)

-------------

PR Comment: https://git.openjdk.org/jfx/pull/446#issuecomment-1498506001