Introduce a compiler option to store generic type information about a lambda expression using the Signature Attribute

[Stephan Ewen]

Thank you for the reply. It is good to separate the proposal to add generic
signatures to synthetic methods from the motivation.

I understand that the motivation is a bit unorthodox. This is only a
workaround for us - we would gladly go for a different solution,
if possible. The original problem is the missing runtime type information
for instances of generic types. We do depend on that
to a good extend for our use case - a large scale data processing engine,
which needs type information for serialization
(network transfer), off-heap memory operations, etc. Think of it like in a
relational database, that needs to know the schema
of the tables. In our case, that schema is the types that go in and out of
the data transformation functions (map, reduce, join,
filter, ...). The schema may actually include generic type variables, we
handle that.

Reflection is a less than optimal solution for that. It used to work,
because before the lambdas, all definitions of functions
came with their very own (anonymous) class, on which we could reflect. With
lambdas, this is no longer true - this is where
the complication started.

A simple uniform way to access generic types or signatures of function
instances would be the preferred solution, definitely.
Until then, we have to work around.

We are also not surfacing any of that behavior to any user of the system,
but we only use it inside utility classes.
There are many examples where frameworks use non standardized utils, or
make a certain assumption about a feature
implementation and special case for different Java versions, platforms, or
vendors
(the netty I/O library, several compression libraries). It is not nice to
do it that way, but it has successfully enabled
certain use-cases and functionality in Java.



On Mon, Jan 26, 2015 at 12:35 AM, Brian Goetz <brian.goetz at oracle.com>
wrote:

> There doesn't seem to be anything *wrong* with having a signature
> attribute on synthetic methods; the fact that we don't have them now is
> more an artifact of how erasure is implemented in the compiler than a
> deliberate decision to strip these methods of generic type information.  As
> we've seen in Project Valhalla, we've been bitten multiple times by missing
> generic information (which is missing for reasons that are effectively
> accidental.)  So I have no problem with the general idea of being more
> uniform about including generic signature attributes.
>
> That said, the motivation that's been presented for adding them now is a
> pretty awful one.  I get why people want reflection to work over lambda
> instances, but that's not how reflection works -- reflection reflects over
> classes, not instances.  The current translation strategy happens to be one
> that, were this attribute there, would enable reflection to "accidentally
> work" to provide generic information, but this *will* change, at which
> point any reflection-based strategy falls apart (at which point people
> accuse of breaking their should-have-never-worked-in-the-first-place
> code.)
>
>
> On 1/7/2015 10:47 AM, Maurizio Cimadamore wrote:
>
>> Hi Timo,
>> thanks for the patch - if I'm not mistaken, this would be the very first
>> case in which a method marked with ACC_SYNTHETIC also gets a signature
>> attribute. I will consult with the rest of the team and see if there's
>> any objection.
>>
>> Cheers
>> Maurizio
>>
>> On 07/01/15 15:22, Timo Walther wrote:
>>
>>> Hi all,
>>>
>>> the Java Reflection API allows to access the generic signature of
>>> methods through the java.lang.reflect.Method#getGenericReturnType()
>>> method. However, this is not yet supported for Lambda expressions.
>>>
>>> Given the following classes:
>>>
>>> class Tuple2<F0,F1> {
>>>   F0 field0;
>>>   F1 field1;
>>>
>>>   public Tuple2(F0 f0, F1 f1) {
>>>     this.field0 = f0;
>>>     this.field1 = f1;
>>>   }
>>> }
>>>
>>> interface Map<IN, OUT> {
>>>   OUT map(IN in);
>>> }
>>>
>>> Currently, there is no solution that allows to get further type
>>> information about expressions like:
>>>
>>> Map<String, Tuple2<String, Integer>> map = (str) -> new Tuple2<>(str, 1);
>>> System.out.println(getReturnType(map)) // can only print Tuple2 =>
>>> information about the Tuple2's fields are always lost
>>>
>>> Especially data-intensive runtimes (like Apache Flink[0] where I am
>>> working on) need as much type information as possible to be efficient.
>>> Therefore, we searched for a way to also extract type information from
>>> lambda expressions. Adding a generic signature to the generated
>>> "lambda$XXX" static methods (at least by using a compiler option)
>>> would be the perfect solution. It seems that the JVM Specification
>>> does not prohibit such an implementation. An implementation of a later
>>> extraction is described here[1].
>>>
>>> The Eclipse JDT compiler team is introducing a compiler option
>>> "-genericsignature" with version 4.5 M4[2].
>>>
>>> I have implemented a patch prototype. It can be found at:
>>> http://people.apache.org/~twalthr/patches/lambdaSignature.patch
>>>
>>> The patch postpones the type erasure of the lambda function's
>>> parameters after the generic descriptor has been saved in a newly
>>> introduced variable in MethodSymbol (if compiler option
>>> "-lambdasignature" is set). The contents of the variable is read in
>>> ClassWriter and written to method's Signature attribute in the
>>> classfile. Tests are also included.
>>>
>>> No change to the class file format is required. The change is guarded
>>> by a compiler option, so without that addition, nothing should behave
>>> differently.
>>>
>>>
>>> [0] http://flink.apache.org/
>>> [1]
>>> http://stackoverflow.com/questions/21887358/reflection-
>>> type-inference-on-java-8-lambdas
>>>
>>> [2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=449063
>>>
>>>
>>> Regards,
>>> Timo Walther
>>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20150202/a11d432d/attachment-0001.html>