RFR: JDK-8241616: Timestamps on ct.sym entries lead to non-reproducible builds

[Magnus Ihse Bursie]

On 2020-05-11 17:25, Jan Lahoda wrote:
> Thanks Magnus for the SOURCE_DATE_EPOCH!
>
> I've adjusted the patch to use it:
> http://cr.openjdk.java.net/~jlahoda/8241616/webrev.01/
>
> It also explicitly sets the output of TransitiveDependencies on the 
> command line (before it was passing just the directory into which the 
> file was written). Also should resolve the suspicious timestamp code 
> noted by Alan, by removing the code altogether, as we can use the 
> timestamp provided by the build.
Looks good to me!

>
> The ct.sym will now be reproducible only if the --with-source-date 
> option is used. I wonder why at least "current" is not the default - 
> that would probably make testing reproducible builds much easier.
That was my original intention, but after working with the patch I grew 
wary. Other tools might pick up SOURCE_DATE_EPOCH and without more 
testing on different platforms I would rather have the default something 
that mimics the old behavior.

/Magnus
>
> How does this look?
>
> Thanks,
>     Jan
>
> On 07. 05. 20 17:48, Magnus Ihse Bursie wrote:
>> On 2020-04-30 14:50, Jan Lahoda wrote:
>>> On 30. 04. 20 14:29, Magnus Ihse Bursie wrote:
>>>> On 2020-04-30 12:41, Alan Bateman wrote:
>>>>>
>>>>>
>>>>> On 30/04/2020 08:03, Jan Lahoda wrote:
>>>>>> Hi,
>>>>>>
>>>>>> The building of lib/ct.sym is not reproducible, due to timestamps 
>>>>>> of files inside the file (which is basically a zip file).
>>>>>>
>>>>>> The proposed solution is to use a constant timestamp for the 
>>>>>> files inside the ct.sym file. To simplify the construction, the 
>>>>>> CreateSymbols tool does not produce files on the filesystem 
>>>>>> anymore, but rather constructs the ct.sym directly.
>>>>>>
>>>>>> Proposed webrev: 
>>>>>> http://cr.openjdk.java.net/~jlahoda/8241616/webrev.00/
>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8241616
>>>>> 1587656816359 = 2020-04-23T15:46:56.359Z. Is there anything 
>>>>> significant with this timestamp? Magnus might have suggestions but 
>>>>> maybe it would be saner to pick up the value of DEFAULT_VERSION_DATE.
>>>> There is an officially suggested standard, SOURCE_DATE_EPOCH for 
>>>> reproducible builds. [1] I have long-term vision to include this in 
>>>> the entire JDK build, but has of yet not even started on that. :-(
>>>>
>>>> This might be a good first step to start using it. I can assist in 
>>>> making the needed makefile changes to have that environment 
>>>> variable available.
>>>
>>> I think a standard mechanism would be great. I don't think this 
>>> patch is very urgent, so I am happy to wait some time (rather than 
>>> add a timestamp to symbols and then remove it when there's a 
>>> standard mechanism).
>>
>> JDK-8244592 is now in mainline, and you can start relying on 
>> SOURCE_DATE_EPOCH being present in the environment when building.
>>
>> /Magnus
>>>
>>> Thanks,
>>>     Jan
>>>
>>>>
>>>> /Magnus
>>>>
>>>> [1] https://reproducible-builds.org/specs/source-date-epoch/
>>>>>
>>>>> There is some curious code that generates the timestamp with:
>>>>>    Long.toString(FileTime.from(Instant.now()).toMillis())
>>>>> Could this use Instant.now().toEpochMilli() instead?
>>>>>
>>>>> -Alan
>>>>>
>>>>>
>>>>
>>