RFR: JDK-8150736: Excessive disk space used by build system

Erik Joelsson erik.joelsson at oracle.com
Fri Sep 30 10:17:38 UTC 2016


Here is a rather large patch which should make life better for most 
people building and developing OpenJDK. The main goal of this patch is 
to reduce unnecessary space used by the build. A side goal also become 
to figure out a better general strategy for handling native debug info 
during the build since that was a large part of the wasted space. These 
are the high level changes:

 1. Removed the dist step in the hotspot build (and the corresponding
    import step in the jdk build) and linking libjvm directly into
    support/modules_libs. This removes 2 extra copies of all the hotspot
    native libraries.
 2. Made many files in the exploded image ($(OUTPUT_DIR)/jdk) symlinks
    into support/modules_*. Note that binaries and libraries cannot be
    symlinks since the image won't run then. Debug symbols and other
    files seem to work fine though.
 3. Introduced separate top level targets for building the jdk, jre (and
    serverjre) images. A developer can run "make product-images-jdk" and
    skip the others. The test targets now depend on just the jdk image
    to avoid building extra images that aren't needed for tests.
 4. Changed SetupNativeCompilation to strip binaries by default unless
    told not to, and removed the separate strip build step. This lets us
    get rid of modules_*-stripped.
 5. Build debug symbols into the OUTPUT_DIR in SetupNativeCompilation
    (instead of OBJECT_DIR and then copy to OUTPUT_DIR). For normal jdk
    and hotspot libraries, this means the debug symbols end up directly
    in support/modules_libs/$MODULE/... and there is only one copy of them.
 6. If zipping debug symbols, this will also leave the unzipped
    debugsymbols in OUTPUT_DIR, where we wouldn't do so before. This
    also means that the exploded image will get symlinks to the unzipped
    debug symbols regardless of if we are asked to zip them or not. This
    in turn means that debugging the exploded image will just work. This
    change is space neutral since we were already keeping a copy of the
    unzipped debugsymbols in the OBJECT_DIR before, they have just been
    moved to a more useful place.
 7. Stop copying debug symbols from test native binaries since we aren't
    stripping them anyway. Better to just leave debug information in the
    test binaries.

The reduction in size of course varies a lot depending on OS and 
configuration, but as an example, on my 64bit Linux workstation 
(debug-symbols=zipped) the size of the build directory has gone from 
5.8GB when building "make images" before to 3.8GB when building "make 
product-images-jdk". The biggest gain is from only building the jdk 
image when only that is needed. A fairer comparison is "make images 
test-image" which has gone from 7.3GB to 5.4GB, much due to not 
duplicating debug information in test binaries. On Solaris the gain is 
generally bigger.

Another nice effect of this is that the exploded image now has debug 
information for all native libraries by default, and that handling of 
dSYM directories on Macosx should now work properly.

The only significant changes detected by comparison builds is that some 
binaries that previously weren't stripped, now are, which they also 
should be.

Bug: https://bugs.openjdk.java.net/browse/JDK-8150736

Webrev: http://cr.openjdk.java.net/~erikj/8150736/webrev.01/

/Erik




More information about the build-dev mailing list