Proposal: JDK-8231640 - (prop) Canonical property storage

[Magnus Ihse Bursie]

On 2021-08-25 14:03, Magnus Ihse Bursie wrote:
> Hi Jaikiran,
>
> I'm glad to see this issue finally getting some love and attention! :)
>
> I don't fully support those "inclinations" that say that the old API 
> should not change. I think keeping the old random order of store() 
> would mean a missed chance to do good, otherwise a lot of Java 
> programs will never get reproducible output of property files. We do 
> have a chance of helping the community, in a single stroke, to make 
> lots of application (more) reproducible without any programmer effort. 
> There is a growing community pushing for reproducible builds (see e.g. 
> https://reproducible-builds.org). Java programs tend to have 
> reproducibility issues due to several cases of non-determinism in the 
> Java platform, and I really think we should try to rectify that.
>
> The specification for store() does not say anything about the order 
> properties are written, so I think the implementation is free to 
> change this to a deterministic order. As long as files written by an 
> updated version of store() can be read by old versions (and vice 
> versa) there will be no compatibility issue in Java programs. And 
> since the old (current...) order is non-deterministic, it seems 
> exceedingly unlikely that any external program depends on the order 
> currently generated by store().
>
> The problem is with the time stamp, which the spec states should be 
> present. I understand why changing this might need a new method. 

This got me thinking a bit. If we are willing to change the spec 
slightly, we could use the standard `SOURCE_DATE_EPOCH` environment 
variable. Many tools support this already. The meaning of this variable 
is that, if present, tools that generate timestamps should use that 
value instead. [1] It seems unlikely that a user running Java with 
`SOURCE_DATE_EPOCH` should get upset if the behavior of 
Properties.store() changed. Without this variable, store() would 
generate timestamps as usual of the current time.

/Magnus

[1] https://reproducible-builds.org/docs/source-date-epoch/


> But I think we should try to steer users to this new method. Otherwise 
> it is likely not to be used by those who should use it (the 
> programmers) to the detrimental effect of users who get properties 
> file which change for no good reason. Having an "attractive" name is 
> definitely part of that. The name should scream "use me as a first 
> hand choice for storing properties to a file". I don't really get that 
> feeling from storeCanonical().
>
>> One thing I do remember is the JDK build (through the make files) 
>> would have certain Java code it would call to do some build steps. Is 
>> there a easy way to find all such build related Java files within the 
>> JDK? I would like to see if there are any references/calls to this 
>> method from those build files. I am doing some searches myself, but 
>> knowing where to search will give me more confidence that I haven't 
>> missed out any.
>
> The java buildtool sources are located in the "make" directory, more 
> specifically in "make/jdk/src", "make/langtools/src" and "make/src" 
> (yeah, I know -- a cleanup is way overdue). I did a quick search now 
> but could not find any references to Properties.store().
>
> I'm trying to remember if we create properties during the build... We 
> have several instances where .properties files in the Java source code 
> are converted to hard-coded classes (for performance), and other where 
> .properties files are copied verbatim. Ah, right, they are "cleaned" 
> beforehand. We used to do this by a Java program, but nowadays they 
> are mangled by sed. I think replacing that sed script with a trivial 
> Java program doing storeCanonical() would be on the list of things to 
> do. I can assist with that, when you are starting to get your 
> implementation done.
>
> We might also write something as part of the jlink process that gets 
> embedded in the resulting jimage; not quite sure about that. You 
> should find that code in the normal src/ codebase though.
>
> /Magnus