RFR: timestamp oddities -sourcepath vs zip/jar

[Martin Buchholz]

Jonathan,

I think we're in complete agreement.

For jsr166 development, we have a work around.  IIRC It used to be that we
needed to do a "ant clean" after we updated our JDK or else some of our
sources would fail to be recompiled, but I fiddled with ant/javac for a
while until the problem went away, but I'm fuzzy on the details (probably
-Xprefer:source)


On Thu, Sep 26, 2013 at 1:42 AM, Jonathan Gibbons <
jonathan.gibbons at oracle.com> wrote:

>  Martin,
>
> Your proposal is phrased somewhat better than the earlier one, but
> realistically, it is too late to do anything in this area for JDK 8.
>
> What I disliked about the earlier proposal was the special casing of
> zip/jar files which was just plain wrong. Here, you are implicitly
> suggesting a better rule, which is based more on the logical paths involved
> (i.e. sourcepath, classpath, bootclasspath) than the nature of the file.
> Effectively, you are saying that in javac's game of rock-paper-scissors,
> sourcepath beats bootclasspath, no matter what the timestamps say.  This
> behavior could be embodied in a new value for the -Xprefer option, leaving
> the existing behaviors unchanged.
>
> -- Jon
>
>
> On 09/25/2013 01:23 PM, Martin Buchholz wrote:
>
> Jonathan,
>
>  The current behavior is logical, but it is not what users intuitively
> expect.  There is a problem for anyone who tries to build JDK boot classes
> outside of the normal JDK build process.  E.g. we do this with jsr166.  We
> want to build against an unmodified JDK (rt.jar) but to generate new class
> files for "our" sources.
>
>  I think that most users in this situation would want the behavior of:
> - files in the sourcepath get compiled to the destdir
> - when looking up a type, if it's in the sourcepath, check if it's also in
> destdir with a newer timestamp; if not, (re)compile it.  If in neither,
> consult rt.jar.
>
>  (that may be hard to do compatibly and without introducing complexity)
>
>
>
> On Tue, Sep 24, 2013 at 7:29 AM, Jonathan Gibbons <
> jonathan.gibbons at oracle.com> wrote:
>
>>  Fredrik,
>>
>> It is a misconfiguration to have classes on the sourcepath, with
>> corresponding classes in the compiled classes directory, AND in rt.jar.  If
>> you're compiling the core JDK classes, you should make sure that rt.jar is
>> not on the classpath or bootclasspath.  For example, you should put sources
>> on the source path and then set the bootclasspath to the compiled classes
>> directory.  This is how the "old build" worked, successfully, for many
>> years.
>>
>> Although the situation you are describing is specific to the JDK build,
>> the general recommendation still applies: don't put a zip file on the class
>> path containing compiled classes for any sources you are compiling.
>>
>> -- Jon
>>
>>
>> On 09/23/2013 10:40 PM, Fredrik Öhrström wrote:
>>
>>  It is a serious bug that command line javac can behave in two different
>> ways when given completely static and predictable source code.
>>  Especially since the change in behavior comes from timestamps inside
>> rt.jar, which a normal user has no control of, nor can easily see.
>>
>>  The intent of the timestamp compare is to know if the class file is out
>> of date in respect to the source file.
>>  Thus if the source has been changed then recompile and write the class.
>> Now the class is up to date.
>> However a class inside a jar file and >especially< inside rt.jar is
>> completely outside of this loop and
>> should only be used when there is no class file in bin, nor any source
>> file in src. Only the timestamp
>> of classes in bin, that can actually be affected by recompilation should
>> matter.
>>
>>  As I said, there are several ways to fix this. Another way would be to
>> exclude rt.jar from the comparison.
>> -Xprefer:source is not good enough because it will recompile everything
>> indiscriminately on the sourcepath
>> even though the class has already been compiled and exists in bin.
>>
>>  Thus preferredFileObject is faulty by design and has managed to stay
>> that way since
>> not may people have exercised -sourcepath, especially not when building
>> classes that
>> also exist in rt.jar.
>>
>>  //Fredrik
>>
>>
>>
>>  2013/9/23 Jonathan Gibbons <jonathan.gibbons at oracle.com>
>>
>>>  On 09/22/2013 06:41 AM, Fredrik Öhrström wrote:
>>>
>>>> The preferredFileObject test in ClassReader.java does cause confusion
>>>> when timestamps inside zip files are too new.
>>>>
>>>> Lets say that you have in src/alfa/A.java
>>>> public class A { void a() { java.lang.Object.foo = 42; } }
>>>> and in src/java/lang/Object.java
>>>> public class Object { public static int foo; }
>>>>
>>>> You can now always compile using the following command:
>>>> javac -d bin src/alfa/A.java src/java/lang/Object.java
>>>>
>>>> But you can only >>sometimes<< compile using the following command:
>>>> javac -d bin -sourcepath src src/alfa/A.java
>>>>
>>>> When it fails, javac has picked up java.lang.Object from rt.jar and
>>>> not from the sourcepath. This happens when the timestamp inside rt.jar
>>>> is newer than the source file.
>>>>
>>>> The net result is that when compiling the OpenJDK with --enable-sjavac
>>>> everything works fine, then you upgrade your jdk and the compilation
>>>> fails in very weird ways. And will fail until you touch all your source
>>>> files.
>>>>
>>>> There are several ways to fix this, but all require some thought.
>>>> However a simple fix, would be to ignore the timestamps of class files
>>>> inside zip/jars because they probably do not mean anything important....
>>>>
>>>> Thus in preferredFileObject, if b is a ZipFileIndexFileObject, then use
>>>> the source!
>>>> protected JavaFileObject preferredFileObject(JavaFileObject a,
>>>>                                            JavaFileObject b) {
>>>>     if (preferSource)
>>>>         return (a.getKind() == JavaFileObject.Kind.SOURCE) ? a : b;
>>>>     else {
>>>>         if (b instanceof
>>>> com.sun.tools.javac.file.ZipFileIndexArchive.ZipFileIndexFileObject) {
>>>>             return a;
>>>>         }
>>>>         long adate = a.getLastModified();
>>>>         long bdate = b.getLastModified();
>>>>         // 6449326: policy for bad lastModifiedTime in ClassReader
>>>>         //assert adate >= 0 && bdate >= 0;
>>>>         return (adate > bdate) ? a : b;
>>>>     }
>>>> }
>>>>
>>>> What do you think?
>>>>
>>>> //Fredrik
>>>>
>>>>
>>>>
>>>>
>>>>
>>>  Fredrik,
>>>
>>> There is no guarantee that items in zip files are presented to javac in
>>> the class you test for, so the instanceof test is highly suspicious and
>>> likely to be a source of inconsistent behavior.  For example, what happens
>>> if someone using using javac via the CompilerAPI (JSR 199) and provides
>>> their own custom file manager?
>>>
>>> If you want to make sure you're compiling source code, use
>>> -Xprefer:source.  Or, keep your timestamps straight.
>>>
>>> -- Jon
>>>
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20130925/d6002cae/attachment-0001.html