RFR: 8293116: Incremental JDK build could be sped up [v2]

[Jan Lahoda]

On Thu, 1 Sep 2022 13:25:19 GMT, Erik Joelsson <erikj at openjdk.org> wrote:

>> Is there some recommendation on how to generate what is needed? Which is:
>> 
>> "-XDmodifiedFiles=<list-of-files> "
>> 
>> 
>> I was looking at the `WriteFile` rule, and it is not clear to be what is the problem there, but I'm not very knowledgeable of the build system.
>> 
>> I was trying to use `ListPathsSafely`, but wasn't able to find a way that would be working.
>> 
>> Thanks!
>
> Before make 4.0, there was no `$(file ...)` function in make. That meant that if you needed to write something to a file directly from the makefile, you had to do something like `$(shell echo "foo" > /my/file)`. This works ok as long as the amount of text you need to write is small enough, but when writing long lists of filenames, you will eventually hit OS limits on command line length (which are different on different OSes, most notably much shorter on Windows). Our solution to this was ListPathsSafely, which breaks up long lists of path names into manageable chunks (and also compresses parts of the path using sed).
> 
> In make 4.0, the $(file) function was introduced, which makes this so much easier. Since it was also more performant, I implemented a 4.0 version of ListPathsSafely. However, we haven't changed the build to require make >4.0, so we have to keep both implementations around.
> 
> The WriteFile macro was only intended writing smaller files, so doesn't have any of the command splitting functionality of ListPathsSafely. 
> 
> So to answer your question, no there isn't a good way to achieve what you are asking for today, not without modifying or extending the macros. One could imagine adding a flag to ListPathsSafely that made it use space instead of newline as separator. Another solution could be to have Depend.java accept something like 
> 
> "-XDmodifiedFilesFile=/path/to/file"
> 
> where the file is a list as ListPathsSafely would create it.
> 
> We could also consider requiring make 4.0, but that will need some socializing first.

I can change `Depend` for this, that is easy, but it is not clear how to write the file. I've tried the naive `$$(eval $$(call ListPathsSafely, $$?, $$($1_MODFILES)))` at the end of `$$($1_FILELIST):` but that does not seem to write the modified files into the file - it produces an empty file. Is there a way for me to write the modified files into the file? Thanks.

-------------

PR: https://git.openjdk.org/jdk/pull/10104