Issue with Comparator.comparing

[Maurizio Cimadamore]

On 12/12/2018 23:44, B. Blaser wrote:
> It's interesting to see that your example fails when invoking
> 'Integer::compareTo' but not when invoking 'Comparator::compare':

Actually, the failure happens in this line:

if (c.compare(a[runHi++], a[lo]) < 0) { // Descending

Where, c is a Comparator<? super T>, and the array to be sorted is a T[].

The comparator generated by Comparator.comparing is then failing 
because, if you look at how the method is implemented:

public static <T, U extends Comparable<? super U>> Comparator<T> comparing(
             Function<? super T, ? extends U> keyExtractor)
     {
         Objects.requireNonNull(keyExtractor);
         return (Comparator<T> & Serializable)
             (c1, c2) -> 
keyExtractor.apply(c1).compareTo(keyExtractor.apply(c2));
     }

It is clear that the method implementation relies on the assumption that 
keyExtractor returns homogeneous things.


But note that, even if the signature was made stricter, as in:

public static <T, U extends Comparable<U>> Comparator<T> comparing(
             Function<? super T, ? extends U> keyExtractor) { ... }

The example I posted in my previous email would still compile - because 
U is inferred as a capture variable generated during method reference 
typing. In other words, the compiler *thinks* that the method reference 
Foo::t is really returning things that can be compareTo with each other, 
which is a flawed assumption!

Note that, what you say:

 > Semantically, 'List<Foo<?>>' means potentially heterogeneous, not 
definitely.

Is already an hint that something is going wrong - generic types are 
universally qualified things - whatever we prove for them must hold FOR 
ALL possible instantiation. Which seems clearly *not* to be the case 
here, as it's easy to think of examples where allowing the above list to 
be treated as an homogeneous comparable collection would lead to issues.


Concluding, I really believe that the capture conversion introduced by 
JDK-8029307, which was kind of a forced move (to type check a method 
reference you have to reason about membership, which implies capture of 
the receiver), is messing things up here, because, from a type system 
perspective, it makes it look as if all possible calls to the functional 
interface derived from that method reference is _always_ going to yield 
the _same_ captured type, where in reality different captures will be 
emitted. This is what made me think that this is indeed a type system 
issue, very close (if not the same) to the one described in JDK-8170887.

Maurizio