RFR: 8293116: Incremental JDK build could be sped up [v2]

[Erik Joelsson]

On Thu, 1 Sep 2022 16:36:55 GMT, Jan Lahoda <jlahoda at openjdk.org> wrote:

>> Before make 4.0, there was no `$(file ...)` function in make. That meant that if you needed to write something to a file directly from the makefile, you had to do something like `$(shell echo "foo" > /my/file)`. This works ok as long as the amount of text you need to write is small enough, but when writing long lists of filenames, you will eventually hit OS limits on command line length (which are different on different OSes, most notably much shorter on Windows). Our solution to this was ListPathsSafely, which breaks up long lists of path names into manageable chunks (and also compresses parts of the path using sed).
>> 
>> In make 4.0, the $(file) function was introduced, which makes this so much easier. Since it was also more performant, I implemented a 4.0 version of ListPathsSafely. However, we haven't changed the build to require make >4.0, so we have to keep both implementations around.
>> 
>> The WriteFile macro was only intended writing smaller files, so doesn't have any of the command splitting functionality of ListPathsSafely. 
>> 
>> So to answer your question, no there isn't a good way to achieve what you are asking for today, not without modifying or extending the macros. One could imagine adding a flag to ListPathsSafely that made it use space instead of newline as separator. Another solution could be to have Depend.java accept something like 
>> 
>> "-XDmodifiedFilesFile=/path/to/file"
>> 
>> where the file is a list as ListPathsSafely would create it.
>> 
>> We could also consider requiring make 4.0, but that will need some socializing first.
>
> I can change `Depend` for this, that is easy, but it is not clear how to write the file. I've tried the naive `$$(eval $$(call ListPathsSafely, $$?, $$($1_MODFILES)))` at the end of `$$($1_FILELIST):` but that does not seem to write the modified files into the file - it produces an empty file. Is there a way for me to write the modified files into the file? Thanks.

Oh right, ListPathsSafely takes the data to write by reference, not by value (as in you supply the name of a variable that contains the data, not the data itself). I doubt `?` can be sent into an eval as a variable name, but I may be wrong. Perhaps you could extract the value into another variable first in a separate line in the recipe, something like this:


        $$(eval $1_MODFILES := $$?)
        $$(eval $$(call ListPathsSafely, $1_MODFILES, $$($1_MODFILELIST)))

(note that I redefined $1_MODFILES to be the list of modified files and introduced a new variable $1_MODFILELIST for the filename where we print it)

-------------

PR: https://git.openjdk.org/jdk/pull/10104