Help with JDK-8211057

[Maurizio Cimadamore]

Hi Magnus,
it's true that CompileProperties relies on Properties, which rely on 
Hashtable (not HashMap) which is likely NOT to have predictable 
iteration order (although I haven't been able to make it spit things in 
different orders on my Linux box). That said, it seems like the code 
keeps that into account:

Properties p = new Properties();
try {
      p.load(new FileInputStream(propertiesPath));
}

...

List<String> sortedKeys = new ArrayList<String>();
for ( Object key : p.keySet() ) {
       sortedKeys.add((String)key);
}
Collections.sort(sortedKeys); // <-----


That is, there is always a sort step on the keys, which should guarantee 
some predictability of the output?

So I think this is ok.

But, looking at your JBS issue more in detail, I see that the problem 
you have is NOT with CompileProperties but with the PropertiesParser 
tool; this is a more complex tool. Eyeballing at it, I find one 
suspicious piece of code:

http://hg.openjdk.java.net/jdk/jdk/file/354fb27fd38a/make/langtools/tools/propertiesparser/gen/ClassGenerator.java#l151

This is a complex stream call which ends up with a Collectors.groupingBy 
- and that method ends up collecting things into an HashMap.

I suggest changing as follows:

Map<FactoryKind, List<Map.Entry<String, Message>>> groupedEntries =
                 messageFile.messages.entrySet().stream()
                         .collect(Collectors.groupingBy(e -> 
FactoryKind.parseFrom(e.getKey().split("\\.")[1]), LinkedHashMap::new, 
Collectors.toList()));

And see whether your problem is addressed? (seems so on my machine)

Hope this helps

Maurizio



On 24/09/18 10:29, Magnus Ihse Bursie wrote:
> Hi compiler folks!
>
> Could anyone possibly help me with having a look at 
> https://bugs.openjdk.java.net/browse/JDK-8211057? The 
> CompileProperties build tool helper is not generating reproducible 
> output, that is, if you run it multiple times one after another, with 
> no source code changes, it will (sometimes) not generate identical 
> output. This breaks our build tests for reproducibility. :-(
>
> I've fixed a handful of similar build tool issues the last year, and 
> almost all of the cases ended up with someone iterating over a HashMap 
> and just outputting elements in the order they happened to get them. 
> However, this case seem different. I see no HashMap, and there is code 
> that clearly intends to sort the output.
>
> My best guess here is that Properties.keySet() is backed by a HashMap, 
> and that Collections.sort(sortedKeys) does not do what the author of 
> the code intended, but that's just my guess.
>
> Since this is hard to reproduce (it's an highly intermittent failure, 
> typically appearing mostly on Windows machines) I'd really appreciate 
> some help by reasoning about the code.
>
> /Magnus