Strange covariant/generics downcasting issue with JDK8
Vicente-Arturo Romero-Zaldivar
vicente.romero at oracle.com
Mon Nov 11 13:13:56 PST 2013
Hi Mark,
I have been studying the code and the jar you sent. Thanks a lot for
reducing the test case. The fact that this code works on 7 and fails in
8 it's a bug in 7. Because of this bug, some programmers may think that
there is a regression in javac 8 provoking a compatibility issue with
javac 7. This has also been detected by Liam Cushon who reported 3 cases
related to this issue [1], a fact that you also mentioned in a previous
mail.
So what is happening here?
I will include for clarity the problematic section in your code:
List<String> strings = Arrays.asList("one", "two", "three");
assertThat(strings).describedAs("test")
.isNotEmpty()
.has(new RegexMatch("th.*"))
.doesNotHave(new RegexMatch("moo.*"))
.hasSize(3);
and also let's also include a highly simplified version of RegexMatch:
public static class RegexMatch extends Condition {
...
}
So the thing here is that if RegexMatch is defined this way javac8
detects an error:
../jdk8-covariantfail-master/AppTest.java:50: error: cannot find symbol
.hasSize(3);
^
symbol: method hasSize(int)
location: class AbstractAssert
1 error
but if RegexMatch is defined as:
public static class RegexMatch extends Condition<Object> {
...
}
then, there is no error.
This is very interesting. Let's focus on the case that produces the error.
First thing to note is that there are two calls using an instance of a
raw type: 'new RegexMatch("...")' which inherits from the raw type
'Condition'. Let's go to the one before the call to 'hasSize()' which is
where javac finds the error:
doesNotHave(new RegexMatch("moo.*"))
This method is defined in class org.fest.assertions.api.AbstractAssert,
also very simplified:
class AbstractAssert<S extends AbstractAssert<S, A>, A> implements
Assert<S, A> {
S doesNotHave(Condition<? super A>) {}
} //signatures taken from the class files
so according to the spec what is happening here is that a raw type is
being passed to the 'doesNotHave()' method which is applicable by
subtyping (see the JLS, Java SE 7 Edition, section 15.12.2.2), so an
unchecked conversion is necessary for the method to be applicable, so
it's return type is erased (see the JLS, Java SE 7 Edition, section
15.12.2.6), in this case the return type will be 'AbstractAssert' and
later the method hasSize will be seek at 'AbstractAssert' but it's not
defined there nor in any of it's super classes and that's why javac
complains that it can't find the method.
Probably the open question is: why then it works when no raw types are used?
Well, there is a long chain of calls but if we consider that:
public class Assertions {
public static <T> ListAssert<T> assertThat(java.util.List<T> list){}
}
All calls after assertThat will be applied to ListAssert<String>.
Several methods, has() and doesNotHave() are defined in AbstractAssert,
so considering that ListAssert is defined as, very simplified:
public class ListAssert<T> extends AbstractIterableAssert<ListAssert<T>,
java.util.List<T>, T> {}
and AbstractIterableAssert is:
public abstract class AbstractIterableAssert<S extends
AbstractIterableAssert<S, A, T>, A extends Iterable<T>, T>
AbstractAssert<S, A> {}
and considering the definition of AbstractAssert above, we have that
when raw types are not used the return type of doesNotHave() is:
'ListAssert<String>' and starting from this class, ListAssert, javac
will look for 'hasSize()' finding it in it's parent
'AbstractIterableAssert'. I guess that if 'hasSize()' would has been
located at 'AbstractAssert' instead of at a subclass of it, there
wouldn't be an error in javac's output for this test case.
As an additional information, the patch that produced this variation can
be seen at [2], the related bug can be seen at [3] and the master bug
entry at [4].
Thanks,
Vicente
[1]
http://mail.openjdk.java.net/pipermail/compiler-dev/2013-October/007726.html
[2] http://hg.openjdk.java.net/jdk8/tl/langtools/rev/48ee63caaa93
[3] https://bugs.openjdk.java.net/browse/JDK-7144506
[4] https://bugs.openjdk.java.net/browse/JDK-7115044
On 04/11/13 22:23, Mark Derricutt wrote:
>
> Vicente,
>
> Thanks for the follow up - I was meaning to chase this up myself.
>
> I tried my existing project with the latest TIP builds of JDK8 and the
> problem still occurs, however using your updated test case I still
> can't reproduce it.
>
> I suspect you've not actually tried to run the maven based project I
> provided ( understandable as that's quite heavy weight if you've not
> got an install ), so I've simplified my test case repo down to a
> Makefile and the fest.jar I've been using:
>
> https://github.com/talios/jdk8-covariantfail/archive/master.zip
>
> Under JDK7 this yields:
>
> |bash-4.2$ java -version
> java version "1.7.0_40"
> Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
> Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
> bash-4.2$ make clean classes
> rm -f *.class
> javac -g -cp fest-assert-core-2.0M10.jar AppTest.java
> Note: AppTest.java uses unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.
> |
>
> whilst under OpenJDK8:
>
> |bash-4.2$ JAVA_HOME=/Users/amrk/Applications/j2sdk-image java -version
> openjdk version "1.8.0-jdk8-b108"
> OpenJDK Runtime Environment (build 1.8.0-jdk8-b108-20130923)
> OpenJDK 64-Bit Server VM (build 25.0-b50, mixed mode)
> bash-4.2$
> bash-4.2$ JAVA_HOME=/Users/amrk/Applications/j2sdk-image make clean classes
> rm -f *.class
> javac -g -cp fest-assert-core-2.0M10.jar AppTest.java
> AppTest.java:50: error: cannot find symbol
> .hasSize(3);
> ^
> symbol: method hasSize(int)
> location: class AbstractAssert
> Note: AppTest.java uses unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.
> 1 error
> make: *** [AppTest.class] Error 1
> |
>
> Hopefully, with this simplified build you might be able to spot
> something more usefull?
>
> Cheers,
> Mark
>
> On 2 Nov 2013, at 7:29, Vicente-Arturo Romero-Zaldivar wrote:
>
> Hi Mark,
>
> I have been trying to reproduce the issue you reported. I have
> created a test case that should reproduce the compilation error
> but I couldn't. Please see my last comment in [1].
>
> I have compiled this test with last version of the compiler from
> [2], I get warnings but no errors.
>
> Maybe you can try to compile the test at [1] with your settings
> and confirm that you get the same error(s) as before. If you don't
> get the same errors could you please check that the test matches
> the version of fest-assert you are using?
>
> I have simplified the hierarchy while keeping the same method
> definitions, but I have seen that there are several versions of
> the library around so probably there can possibly be
> some**mismatches with your current version.
>
> Thanks,
> Vicente
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8026329
> [2] http://hg.openjdk.java.net/jdk8/tl/langtools/
>
> On 05/10/13 22:15, Mark Derricutt wrote:
>
> Just following up on this now that Java One is over and
> hopefully more eyes can help me track down just what's going
> on here.
>
> Try as a may I've not yet been able to distil this problem
> down to a single class file to reproduce, but the extracted
> project on github (
> https://github.com/talios/jdk8-covariantfail ) continues to
> fail under JDK8.
>
> The FEST-Assert library uses a fairly complex class hierarchy,
> employing some gnarly "self type" generics in order to provide
> the appropriate type to javac.
>
> When compiling under Java 7 the project works fine, but when
> compiling under JDK8 ( developer preview, and also HEAD/master
> builds from mercurial ) fail to compile the 2nd test function,
> as the inferred return type is getting lost somewhere along
> the way.
>
> If the 2nd test is broken up with intermediate variables then
> the call to hasSize(3) on line 37 compiles fine.
>
> Given this works under Java 7, and fails under Java 8 - this
> is clearly some form of regression/bug somewhere,
> just exactly what is broken, and how to reproduce it is so far
> eluding me, any help in isolating this issue so that a
> relevant bug can be raised, tracked, and fixed would be helpful.
>
> Mark
>
> --
> Mark Derricutt mark at talios.com <mailto:mark at talios.com> —
> twitter https://twitter.com/talios — podcast
> http://www.illegalargument.com — blog
> http://www.theoryinpractice.net — google+ http://gplus.to/talios
>
> On 23/09/2013, at 5:30 PM, Mark Derricutt > wrote:
>
> Hi all,
>
> Since you're all enjoying JavaOne at the moment I was
> thinking it was about time that I starting building
> $work's project under JDK8 and give it some love over my
> normal small projects and ran into an odd issue with
> generics/covariant returns that works fine under 7u40 and
> breaks under 8DP.
>
> I've extracted this into a simple project [1] which I've
> pushed to github, I've added a comment on the code that
> breaks under JDK8.
>
> The code in question is:
>
> List strings…
>
> assertThat(strings).describedAs("test")
> .isNotEmpty()
> .has(regexMatch("th./"))
> .doesNotHave(regexMatch("moo./"))
> .hasSize(3);
>
> When calling assertThat(strings) a class of type
> ListAssertion is returned, and passed along the call chain.
>
> The problem appears to be that when doesNotHave() returns,
> javac no longer sees a ListAssertion but rather a top
> level AbstractAssertion on which doesNotHave is defined:
>
> /** {@inheritDoc} */
> public S has(Condition<? super A> condition) {
> conditions.assertHas(info, actual, condition);
> return myself;
> }
>
> /** {@inheritDoc} */
> public S doesNotHave(Condition<? super A> condition) {
> conditions.assertDoesNotHave(info, actual, condition);
> return myself;
> }
>
> Now, both has() and doesNotHave() appear to have the exact
> same signature, yet commenting out the call to
> doesNotHave() doesn't appear to trigger what appears to be
> a down casting to AbstractAssertion which doesn't include
> a method called hasSize(), the output I see from javac is:
>
> ~/D/covariantfail (master|…) $
> /Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/bin/java
> -version
> java version "1.8.0-ea"
> Java(TM) SE Runtime Environment (build 1.8.0-ea-b106)
> Java HotSpot(TM) 64-Bit Server VM (build 25.0-b48, mixed mode)
>
> ~/D/covariantfail (master|…) $
> /Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/bin/javac
> -version
> javac 1.8.0-ea
>
> /Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/bin/javac
> -d /Users/amrk/Dropbox/covariantfail/target/test-classes
> -classpath
> /Users/amrk/Dropbox/covariantfail/target/test-classes:/Users/amrk/Dropbox/covariantfail/target/classes:/Users/amrk/.m2/repository/org/testng/testng/6.8.5/testng-6.8.5.jar:/Users/amrk/.m2/repository/junit/junit/4.10/junit-4.10.jar:/Users/amrk/.m2/repository/org/hamcrest/hamcrest-core/1.1/hamcrest-core-1.1.jar:/Users/amrk/.m2/repository/org/beanshell/bsh/2.0b4/bsh-2.0b4.jar:/Users/amrk/.m2/repository/com/beust/jcommander/1.27/jcommander-1.27.jar:/Users/amrk/.m2/repository/org/yaml/snakeyaml/1.6/snakeyaml-1.6.jar:/Users/amrk/.m2/repository/org/easytesting/fest-assert-core/2.0M10/fest-assert-core-2.0M10.jar:/Users/amrk/.m2/repository/org/easytesting/fest-util/1.2.5/fest-util-1.2.5.jar:
> -sourcepath
> /Users/amrk/Dropbox/covariantfail/src/test/java:
> /Users/amrk/Dropbox/covariantfail/src/test/java/com/talios/RegexMatch.java
> /Users/amrk/Dropbox/covariantfail/src/test/java/com/talios/AppTest.java
> -g -nowarn -target 1.5 -source 1.5 -encoding UTF-8
> /Users/amrk/Dropbox/covariantfail/src/test/java/com/talios/AppTest.java:37:
> error: cannot find symbol
> .hasSize(3);
> ^
> symbol: method hasSize(int)
> location: class AbstractAssert
> Note:
> /Users/amrk/Dropbox/covariantfail/src/test/java/com/talios/AppTest.java
> uses unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.
> 1 error
>
> Is this a known bug in the JDK at all? I'm quite surprised
> no one else has hit something like this before if its not?
>
> [1] https://github.com/talios/jdk8-covariantfail
> [2]
> https://github.com/talios/jdk8-covariantfail/commit/faa016e57f754230da46e3453b964d497ec065b5#commitcomment-4151896
>
> -- Mark Derricutt ( mark at talios.com
> <mailto:mark at talios.com> mark at talios.com
> <mailto:mark at talios.com> )
> — twitter: https://twitter.com/talios
> — podcast: http://www.illegalargument.com
> http://www.illegalargument.com/
> — blog: http://www.theoryinpractice.net
> http://www.theoryinpractice.net/
> — google+: http://gplus.to/talios
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20131111/c8eeebaf/attachment-0001.html
More information about the compiler-dev
mailing list