Revisiting field references

[Brian Goetz]

Alan;

Thanks for these thoughts.  Indeed, field references are something that 
keep coming up in the "if we only had" department, and your analysis 
covers many of the important points.  They are especially useful in APIs 
such as the Equivalence one that Kevin and Liam have already written 
about, as well as similar APIs (e.g., Comparator factories).  Let me 
make some connections with other roads not taken, as well as some 
connections with other features that are being considered.

> Getter as Supplier
> Setter as Consumer

It surely seems nice to be able to SAM-convert a field ref to an 
appropriately-shaped SAM type based on target typing.  Another question 
we should ask is: should field refs (and method refs, for that matter) 
have a standalone type?   In the original JSR-335 discussions, another 
"left for the future" item was whether method refs should be "enhanced" 
compared to lambdas.  Such areas include: better equals() implementation 
(two method refs for the same method could be equal), better toString() 
implementation (using the name of the method, rather than an opaque 
descriptor), reflective behavior as described in your Increased 
Transparency section (retrieving the nominal metadata for the method 
class, name, and descriptor), etc.

Two more areas that were discussed, but left for the future, for method 
references were:

  - Alternate target types, such as Method or MethodHandle (yes, Remi, 
we see you);
  - Explicit method references (with explicit parameter types, such as 
`Foo::bar(int,int)`), which could be used in contexts where a SAM target 
type is not available -- such as Method.

Further, the work done on constant folding exposes interesting 
optimization opportunities for APIs that can truck in member references; 
for a field ref such as `Holder::field`, foldable (compile-time 
invocable) APIs could be brought to bear.  I don't want to deep dive on 
this here, other than to point out that there is a significant connection.

> More subtly, source code may
> become harder to understand when the expression this::field may mean
> two very different things, either a read or a write.

Thanks for bringing up this point, as it speaks to the "we _can_, but 
should we?" question.  Target typing is a powerful thing, but as you 
say, it is a little scary for the same locution to represent such 
different behaviors.

> We could have some sort of
> FieldReference object describing the class in which the field lives,
> the name and type of the field, and which object (if any) is bound as
> the receiver.

A few points here:

  - Some of these items are security-sensitive, and others are much less 
so.  The name of the method or field, for example, is pretty safe to 
expose (it is already exposed through stack traces), whereas exposing 
bound receivers or captured arguments seems a pretty clear no-go (I 
doubt anyone passing `foo::bar` to a library method thinks they are 
sharing `foo` too.)  I think the line here is: expose nominal metadata 
(class names, method names, method descriptors), and not live objects 
(class mirrors, method handles, bound arguments, etc), except as 
mediated by access control (e.g., caller provides a Lookup.)

  - It's not either-or; MethodReference / FieldReference could be 
interfaces, and when SAM-converting a method/field reference to a 
suitable SAM type, the resulting object is of type `(SamType & 
MethodReference)`.

  - FieldReference/MethodReference could be the standalone type of field 
refs (and either non-overloaded method references, or method references 
with explicit parameter types.)

> An advantage of supporting this is that it could enable libraries that
> currently accept lambdas to generate more efficient code.

This connects with the constant-folding, as well as offering some 
API-design options, such as:

     static<T> Comparator<T> ofFields(FieldReference... fields)

which would allow for invocations like

     Comparator.ofFields(Foo::a, Foo::b)

Having a factory that takes a varargs of field references is a good 
trigger to try optimizing transparently, as you know you have 
optimizable references in hand.

> I hope it goes without saying that I am not proposing to actually
> implement Comparator.optimize any time soon: it’s just a convenient,
> well-known example of the kind of library that could be gradually
> improved by promoting field references from “sugar for a lambda” to
> reified objects.

A good API is both easy to use and easy to optimize.  Capturing 
higher-level semantics such as transparent field refs vs opaque lambdas 
scores well on both counts, regardless of specific optimization avenues 
you might have in mind.

> Annotation parameters
>
> Last, if we had such a FieldRef descriptor, we might like to be able
> to use them as annotation parameters, making it possible to be more
> formal about annotations like
>
> class Stream {
>    private Lock lock;
>    @GuardedBy(Stream::lock) // next() only called while holding lock
>    public int next() {...}
> }

Method refs in annotations are another one on the "would like to do 
eventually" list; adding ConstantDynamic removed one of the major 
impediments, but there's still a bunch of work in between here and 
there.  To your comment about method refs, the main impediment is the 
lack of explicitly typed method references, but we know how to do that too.

> Probably this would mean having FieldReference implement Constable, so
> that Holder::field could be put in the constant pool, along with other
> annotation parameters.

Yes.  The key is that any symbolic metadata (Class, MethodType) is 
mediated by a Lookup (for the reflective path), and by the implicit 
resolution context (when loading the classfile.)

> This also suggests that a FieldReference object
> should not directly store the bound receiver, since that could not be
> put in the constant pool; instead we would want a FieldReference to
> always be unbound, and then some sort of decorator or wrapper that
> holds a FieldReference tied to a captured receiver.

A bound method ref is not a constant; only unbound / static method refs 
would be.  Same story for fields.

> Open Questions
>
> The first set of questions is: are these all reasonable, useful
> features? Am I missing any pitfalls that they imply?
>
> One looming design question is unfortunately syntax: is Foo::x really
> the best syntax? It's very natural, but it will be ambiguous if Foo
> also has a method named x.

... which will be the case very often.  And in fact, that method is 
probably (but we can't guarantee) an accessor for the field.  And, while 
we can construct precedence rules that make sense for various use cases, 
in reality, sometimes you want Foo::x to be the field (such as in the 
comparator example), and sometimes you want it to be the method (such as 
when you're sharing it with foreign code, because you want the defensive 
copy that the accessor might do.) So even if we bias towards the method 
(which as you say, is a forced move), there still needs to be a way to 
denote the field ref explicitly in case of conflict.

> Finally, if anyone has implementation tips I would be happy to hear
> them. I am pretty new to javac, and while I've thrown together an
> implementation that desugars field references into getter lambdas it’s
> far from a finished feature, and I’m sure what I’ve already done
> wasn't done the best way. Finding all the places that would need to
> change is no small task.

I know that Dan went through a similar exercise a long time ago, 
identifying the places in the spec/compiler that would take stress if we 
wanted to broaden the set of target types for method refs. While that's 
not exactly the same problem, it would be useful to dredge that up, as 
it will likely cover some of the same ground.

For prototyping purposes, I wouldn't try to use the same token; that's 
just making your life harder.  Pick something easily parsable, even if 
its hideous.

More thoughts later.