RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

[Erik Joelsson]

On Thu, 7 Apr 2022 18:17:57 GMT, Andrew Leonard <aleonard at openjdk.org> wrote:

>> make/common/NativeCompilation.gmk line 1159:
>> 
>>> 1157:       ifeq ($(call isTargetOs, linux), true)
>>> 1158:         ifeq ($$($1_COMPILE_WITH_DEBUG_SYMBOLS), true)
>>> 1159:           $1_LINK_OBJS_RELATIVE := true
>> 
>> I realize this PR has already been integrated, but I have some questions about this. The comment talk about "ASM" objects (I assume that means output of assembly files), but the code change is not restricted to assembly files. 
>> 
>> Afaict, this change means that all builds on linux with debug symbols and reproducible builds will use relative paths! This is perhaps the most common compilation scenario, and it would mean that our efforts to keep a CWD neutral command line is basically in vain. :-(
>> 
>> Or can anyone (@erikj79, @andrew-m-leonard) explain to me why this would not be the case?
>
> @magicus 
> So I did think about this, and the alternative I think would be to hard code a list of which object files are the output from assembly files, of which there is about a dozen. Unless we could capture somehow the list of assembled files as we go ...?
> As @erikj79 points out this is just for the "linking". As the problem is the assembly file objects don't have the equivalent of debug prefix mapping, and so when stripped the full file path of the object file ends up in the debuginfo file, and results in the libjvm.so non-determinism due to different debug CRC.

It would certainly be possible to identify the objects created from assembly source files based on file extensions, but it wouldn't help. The command line would still be dependent on the current working directory (correctly guessed!) if just one of the argument files is relative.

The property Magnus is trying to preserve is to be able to copy a command line from the build log and run it in whichever directory you happen to be in, without having to figure out what the CWD was for that particular command. We worked hard to try to figure out ways of preserving that as much as possible, but as you say, if we want true reproducibility, then we have to give it up for link command lines where assembly source files are included.

Now I realize that you aren't even checking if the link unit contains any objects from assembly files. I think we should be able to deduce that by filtering on the extension in the source file list. That would miss assembly input in EXTRA_OBJS, so perhaps we need an explicit parameter to SetupNativeCompilation for cases where this would happen. I think it applies to gtest, where we add all the object files from libjvm.

-------------

PR: https://git.openjdk.java.net/jdk/pull/8124