RFR: JDK-8150736: Excessive disk space used by build system
Erik Joelsson
erik.joelsson at oracle.com
Fri Sep 30 10:17:38 UTC 2016
Here is a rather large patch which should make life better for most
people building and developing OpenJDK. The main goal of this patch is
to reduce unnecessary space used by the build. A side goal also become
to figure out a better general strategy for handling native debug info
during the build since that was a large part of the wasted space. These
are the high level changes:
1. Removed the dist step in the hotspot build (and the corresponding
import step in the jdk build) and linking libjvm directly into
support/modules_libs. This removes 2 extra copies of all the hotspot
native libraries.
2. Made many files in the exploded image ($(OUTPUT_DIR)/jdk) symlinks
into support/modules_*. Note that binaries and libraries cannot be
symlinks since the image won't run then. Debug symbols and other
files seem to work fine though.
3. Introduced separate top level targets for building the jdk, jre (and
serverjre) images. A developer can run "make product-images-jdk" and
skip the others. The test targets now depend on just the jdk image
to avoid building extra images that aren't needed for tests.
4. Changed SetupNativeCompilation to strip binaries by default unless
told not to, and removed the separate strip build step. This lets us
get rid of modules_*-stripped.
5. Build debug symbols into the OUTPUT_DIR in SetupNativeCompilation
(instead of OBJECT_DIR and then copy to OUTPUT_DIR). For normal jdk
and hotspot libraries, this means the debug symbols end up directly
in support/modules_libs/$MODULE/... and there is only one copy of them.
6. If zipping debug symbols, this will also leave the unzipped
debugsymbols in OUTPUT_DIR, where we wouldn't do so before. This
also means that the exploded image will get symlinks to the unzipped
debug symbols regardless of if we are asked to zip them or not. This
in turn means that debugging the exploded image will just work. This
change is space neutral since we were already keeping a copy of the
unzipped debugsymbols in the OBJECT_DIR before, they have just been
moved to a more useful place.
7. Stop copying debug symbols from test native binaries since we aren't
stripping them anyway. Better to just leave debug information in the
test binaries.
The reduction in size of course varies a lot depending on OS and
configuration, but as an example, on my 64bit Linux workstation
(debug-symbols=zipped) the size of the build directory has gone from
5.8GB when building "make images" before to 3.8GB when building "make
product-images-jdk". The biggest gain is from only building the jdk
image when only that is needed. A fairer comparison is "make images
test-image" which has gone from 7.3GB to 5.4GB, much due to not
duplicating debug information in test binaries. On Solaris the gain is
generally bigger.
Another nice effect of this is that the exploded image now has debug
information for all native libraries by default, and that handling of
dSYM directories on Macosx should now work properly.
The only significant changes detected by comparison builds is that some
binaries that previously weren't stripped, now are, which they also
should be.
Bug: https://bugs.openjdk.java.net/browse/JDK-8150736
Webrev: http://cr.openjdk.java.net/~erikj/8150736/webrev.01/
/Erik
More information about the build-dev
mailing list