Review Request: 8005540: build-infra: Improve incremental build speed on windows by caching find results

[Kelly O'Hair]

A little hard to review, but in general I really like the idea.
I had experimented with files containing the 'find' output in the past, but never thought about using make variables. Very clever.
The variables and values are in the make process itself, easier to keep up-to-date and trust, faster too.

I'd be tempted to think about using the same mechanism on all platforms eventually, could reduce the getdents system calls
significantly which might not show up as a huge win build time wise, but reducing system calls can be a very good thing.

This kind of change right now is a little risky, but let's get it in. Worse case scenario we just disable it.

Consider it reviewed.

-kto

On Dec 27, 2012, at 8:30 AM, Erik Joelsson wrote:

> A few weeks back, I did some experiments with find on windows/cygwin. File caching is not nearly as efficient as on linux, combined with slow forks this makes repeated runs of find over the same set of files very inefficient. This pattern is used a lot in the new makefiles for finding sources.
> 
> Make variables and filters are very efficient and fast however, so in many cases, we can replace $(shell find ...) expressions with $(filter ) expressions over previous find results. To make this as transparent as possible, I've implemented two macros, one for filling the cache and one for potentially using it, if the correct set of directories are in the cache, and otherwise revert to $(shell find). This works pretty well and reduces the time to run make a second time quite a lot. The trick is to initiate the cache with a good set of directories at the top of each affected makefile.
> 
> In addition to this, some more optimization of removing unneeded $(shell) expressions has been done.
> 
> http://cr.openjdk.java.net/~erikj/8005540/webrev.01/
> 
> /Erik