First webrev for smart javac wrapper

[Fredrik Öhrström]

The current build-infra/jdk8 forest can now be configured using --enable-sjavac

This causes the build to do proper incremental builds of Java sources (all the other
sources are already properly handled when building incrementally)
i.e. only the java sources that need to be compiled are actually compiled.

Dependency tracking causes other packages that import from the recompiled
package to be recompiled, but >only< if the public api of the first package was
changed. 

It also attempts to use multiple cores, i.e. start a JavaCompiler per core.
These compilers do currently not share any state at all! This is something
that we should improve upon in the future.

It spawns a background javac server in which all sjavac compilations take place.

Tested on linux32, linux64 and macosx64. 
Building on my workstation, 8 cores, 16GB RAM, SSD, fully primed ccache.

-- Build times ----------
Target 
Start 2012-07-10 23:39:19
End   2012-07-10 23:42:20
00:00:08 corba
00:01:10 hotspot
00:00:07 jaxp
00:00:10 jaxws
00:00:53 jdk
00:00:33 langtools
00:03:01 TOTAL
-------------------------

When building with an empty ccache:
-- Build times ----------
Target 
Start 2012-07-10 23:43:58
End   2012-07-10 23:52:57
00:00:08 corba
00:06:21 hotspot
00:00:08 jaxp
00:00:10 jaxws
00:01:38 jdk
00:00:34 langtools
00:08:59 TOTAL
-------------------------

The jdk build is slower since the sjavac server shuts down because of inactivity during the hotspot build.
The cost of re-optimizing the byte code is apparent. The timeout can of course be adjusted, or prevented.

After you built the jdk, add a public method foo to javax/management/JMX.java and do make again. 
You will see (and some javac warnings, here redacted):

Compiling BUILD_JDK
Compiling javax.management(85) 
Package :javax.management  pubapi has changed!
Compiling com.sun.jmx.interceptor(2) com.sun.jmx.mbeanserver(31) com.sun.jmx.remote.internal(14) com.sun.jmx.remote.protocol.iiop(4) com.sun.jmx.remote.protocol.rmi(2) com.sun.jmx.remote.security(7) com.sun.jmx.remote.util(5) com.sun.tools.jconsole(2) java.lang.management(21) javax.management.loading(9) 
Compiling javax.management.modelmbean(12) javax.management.monitor(10) javax.management.openmbean(25) javax.management.relation(27) javax.management.remote(18) javax.management.remote.rmi(9) 
Compiling javax.management.timer(4) sun.management(50) sun.management.jmxremote(3) sun.tools.jconsole(35) sun.tools.jconsole.inspector(24) 
Compiling ClassLoadingImpl.c
Compiling FileSystemImpl.c
Compiling Flag.c
Compiling GarbageCollectorImpl.c
Compiling GcInfoBuilder.c
Compiling HotSpotDiagnostic.c
Compiling HotspotThread.c
Compiling MemoryImpl.c
Compiling MemoryManagerImpl.c
Compiling MemoryPoolImpl.c
Compiling ThreadImpl.c
Compiling VMManagementImpl.c
Linking libmanagement.so
Running rmic -v1.2 -iiop for javax.management.remote.rmi.RMIConnectionImpl javax.management.remote.rmi.RMIServerImpl
Running rmic -v1.2 -keepgenerated for javax.management.remote.rmi.RMIConnectionImpl javax.management.remote.rmi.RMIServerImpl
Running rmic -iiop -standardPackage for javax.management.remote.rmi.RMIConnectionImpl javax.management.remote.rmi.RMIServerImpl

The pubapi change in JMX.java triggered a recompilation of other packages importing from javax.management,
(it does not track dependencies between java source files, only between java packages.) 
Then the recompiles of files with native methods, regenerate the c header files, and this triggers the recompilation
of the C-sources that include these header files. Clearly in this case, the header files should not be regenerated
since the native api has not changed, but that feature is waiting to be built into the smart javac file manager.
(ie not write non-changed headers to disk)
And finally it regenerates the rmi code related to the management api.

The first webrev is here. http://cr.openjdk.java.net/~ohrstrom/webrev-2012-07-10-sjavac-v1/
Awaiting feedback to create the next webrev!

Thanks!
//Fredrik




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20120711/3d421817/attachment.html