Incremental java compile AKA javac print compile dependencies

Tue May 25 07:43:23 PDT 2010

On 05/24/2010 11:47 PM, Joshua Maurice wrote:
> Thank you. I will ask that group.
>
> However, your suggestion of the class files doesn't quite fit what I 
> want to do. Using the information from class files, (ignoring the 
> constant problem), (without great modification), will only work if the 
> build cascades endlessly downstream. I am specifically trying to make 
> a system where the cascade will terminate when no further recompiles 
> downstream are required. To this end, the information in the class 
> files is woefully insufficient. Ex: C extends B. B extends A. A 
> declares member field x. Class D uses C.x. I'm pretty sure that in D's 
> class file, A and B are not mentioned. Some change to B could 
> introduce a field named x, "hiding" A.x, potentially changing the 
> meaning and compile of D, potentially even resulting in a clean build 
> failure, but this "incremental" build would silently call a success.
>
> Also, the JavaFileMananger idea will not work for the same reason that 
> -verbose as is will not work. I need / want this information on a per 
> java file basis, but javac "caches" referenced class files, only 
> loading them once, so neither will give me what I need.

By analysing the contents of a class file you can determine what 
accessible aspects of any class each class depends on, and that should 
be sufficient to solve the dependency analysis problem.  You do /not/ 
want to look at the verbose output because it will tell you too much 
irrelevant info.  For example, if class A calls method print in class B, 
all that matters is that A calls B.print -- it does not matter how 
B.print is implemented -- whether it uses System.err.println, or 
java.util.logging or CORBA.    Analyzing -verbose output will tell you 
all the implementation dependencies; you only need the "API signature" 
dependencies.

Furthermore, by computing an MD5 checksum for the API signature of each 
classfile, you can determine whether a classfile's API signature has 
been changed by a recompilation.   If the checksum is changed after a 
compilation, you need to recompile dependent classfiles. If the checksum 
has not changed, you don't need to recompile dependent classfiles. So if 
you just edit comments in your file, or if you just change use of 
System.err.println to System.out.println, then you don't need to 
recompile any of the classes that depend on the recompiled class.  And 
that's the holy grail of incremental compilation.

>
> PS: Is there a non-arrogant way to suggest reading the 
> comp.lang.java.programmer discussion, as it contains most of this 
> discussion already, without sounding like a self absorbed jerk who 
> thinks his time is more important than everyone else's? Probably not. 
> I'll ask compiler-dev.

Very perceptive.

>
> On Mon, May 24, 2010 at 9:29 PM, Jonathan Gibbons 
> <jonathan.gibbons at oracle.com <mailto:jonathan.gibbons at oracle.com>> wrote:
>
>     Joshua,
>
>     This question is better asked on compiler-dev at openjdk.java.net
>     <mailto:compiler-dev at openjdk.java.net>.
>
>     There are various ways you can get the information you are asking,
>     in a single invocation of the compiler and without changing the
>     compiler.
>
>     One way would be to invoke the compiler via the JSR 199 API. You
>     can provide your own JavaFileManager which can keep track of files
>     read and written by the compiler. For the class files that are
>     written, you can analyze the class files to determine all the
>     classes that are referenced by that class file. Class files also
>     identify the source files that they come from. That should be
>     enough to get you almost all the information that you need. The
>     only information you'll miss using that technique is inlined
>     constants.
>
>     -- Jon G
>
>
>
>
>     On 05/24/2010 07:23 PM, Joshua Maurice wrote:
>
>         I'm sorry. I'm new to this list, and I'm not sure if this is
>         the appropriate
>         forum. I was looking at the javac source code, and I am now
>         looking for help
>         or guidance. Specifically, I want to create an incrementally
>         correct build
>         system for java, preferably without having to (re)write a Java
>         compiler. I
>         need some additional information to be printed from javac to
>         get full
>         dependency information. Please see:
>         http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4639384
>
>         http://groups.google.com/group/comp.lang.java.programmer/browse_thread/thread/bb6f663d55700951#
>         for a detailed discussion. The short version is that -verbose
>         output almost
>         is what I need. I need to know the full list of class files
>         loaded for each
>         java file being compiled, and the full list of java files in
>         the compile
>         used by each java file being compiled; that is, the full list
>         of java file
>         and class file \compile\ dependencies of each java file in the
>         compile. One
>         can already get this information by calling javac once on the
>         set of
>         interested java files to do the compile, then an additional
>         java -verbose
>         once per java file to get the dependency information.
>
>         However, calling javac so many times is quite slow, and
>         unnecessary. I would
>         like to enhance javac to print out this dependency information
>         with one
>         invocation aka without resorting to calling javac once per
>         java file, and
>         preferably get this change into the actual official released
>         javac.
>
>         PS: Yes this information on its own is insufficient to do an
>         incrementally
>         correct java compile. However, when combined with Ghost
>         Dependencies (please
>         see:
>         http://www.jot.fm/issues/issue_2004_12/article4.pdf
>         ), I think that this would actually work. I have some tests to
>         this effect
>         already with my proposed system which currently calls javac
>         once per java
>         file to get the compile dependency information.
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20100525/b5c6cd45/attachment.html 