RFR: JDK-8220383: Incremental build is broken and inefficient

[Erik Joelsson]

Our incremental build performance has degraded a lot recently. On 
Windows it's especially bad (as usual). There are multiple issues. Some 
cases of running way too many find/grep during rule setup. Some files 
get rebuilt on every incremental build. With all these changes, the 
incremental build with no changes of "make product-bundles test-bundles" 
on a Windows machine I was testing went from 4m to 24s. Here is a list 
of the fixes:

We have a rather inefficient pattern where a makefile is responsible for 
two top level targets, the general build of something and then the 
copying of that something into images. The drawback of this pattern is 
that all the rules in the makefile get evaluated twice, and this is 
potentially expensive. My workaround for this is to enclose the 
expensive parts in a conditional that checks what target the makefile 
was called with. This pattern is already in use in at least Bundles.gmk.

In CompileJavaModules.gmk, if IMPORT_MODULES_CLASSES isn't set, wildcard 
will try to look for /$(MODULE), which can sometimes be very costly and 
freeze the system for 10-20s (likely due to going to check on the network).

Images.gmk, on Windows, the CDS archive was always being rebuilt due to 
the rule targeting the wrong file.

NativeCompilation.gmk has two fixes. Reduce the number of $(shell find 
...) calls by doing just one and then ordering the files found. The 
second fix is more intricate. The adding of the IMPORT_LIBRARY to the 
list of TARGETS was missing. This was causing trouble in combination 
with a test lib, libHasNoEntryPoint.c, which does not produce a link.lib 
file. When building incrementally, the makefile cannot find the 
HasNoEntryPoint.lib file and so deletes HasNoEntryPoint.dll to force a 
rebuild. By adding the IMPORT_LIBRARY to the targets, the 
HasNoEntryPoint.lib will at least be touched. I also had to remove the 
safety check that a .lib file was really created since it apparently 
isn't always created.

In ProcessMarkdown.gmk, the $(shell grep ) expression was called for 
every markdown file, even if no rebuilding was needed.

TestFilesCompilation.gmk called SetupNativeCompilation for each test 
.c/cpp file, which would in turn call find on the src dir where that 
file was located. I changed the call to use the EXTRA_FILES parameter 
instead of SRC to avoid the extra calls to find.

In CompileJvm.gmk I added a call to FillCacheFind so we can reuse the 
set of hotspot sources since they are used more than once.

I have run a comparison build of our main platforms with the patch.

There are certainly more improvements possible, but I chose to limit 
myself to the most obviously broken parts here.

Webrev: http://cr.openjdk.java.net/~erikj/8220383/webrev.01/index.html

Bug: https://bugs.openjdk.java.net/browse/JDK-8220383

/Erik