PROPOSAL: Language Escape Operator

[Bruce Chapman]

Reinier Zwitserloot wrote:
> Can you give me a use-case that used a back-tick? I found the property 
> keyword one to be very un-compelling (in fact, I would use it as proof 
> that we don't need it, and that even if it was available, it wouldn't 
> even be used. If I was designing an AST view language, I wouldn't do 
> that, and if I had to pick one, I'd count the need to type a backtick 
> as a major con). I think we're misunderstanding each other on some 
> topics because of a lack of examples. I understand the point of this 
> operator is for there to be NO actual use-cases yet, but nevertheless, 
> gazing into the future should give some possible examples, no matter 
> how esoteric.
OK, so Property would be fairly easy to parse in the absence of an 
escape operator. I have used the property plugin on a real project and 
typing the backtick wasn't a major negative.

So some more examples - straight off the top of my head

what about operator overloading say

BigDecimal a,b,c;

b = a \+ c;   <=> b = a.add(c);

or String switches where the first syntax error wrt  JLS occurs way 
after the start of the statement.  (I am assuming in the absence of a 
language escape you'd use a parsing error for the standard java language 
to trigger a reparse with various loaded sugars.)


Or here's a favourite PITA for me. There is no simple obvious way to 
pass a generic type as an annotation value. What you can do is have the 
annotation value be a Class<?>[], then just parse the supplied classes 
assuming no raw types, recursive read as many type parameters as you need.

@interface SomeReasonForAType {
   Class<?>[] value();
}

so when you wanted to specify Map<String,List<String>> it would be 
encoded as  {Map.class,String.class, List.class, String.class}

now what if we got an IDE hosted syntactic sugar for that with the 
following mapping

\Map<String,List<String>>   <=> {Map.class,String.class, List.class, 
String.class}

maybe only when used as an annotation value.




Stefan Schulz has explained the back tick problem to me (I only use a 
US-101 keyboard so back tick is no big deal) and I think that is a good 
enough reason to maybe use one of the other available characters. Don't 
get hung up on the individual character, it is easily changed at this 
point. Would you prefer \? . I'd be happy to change it to \, there are 
some semantic advantages to \.
>
> NB: Loved your slides on the topic. I have comments because I care :P
Glad about that. I was a wondering how much they made sense in the 
absence of the narrative, its pretty hard to know when you've done it 
yourself with the aim of Illustrating a talk.

Have you made comment somewhere regarding the slides? or are you 
referring to these comments on coin-dev?

thanks

Bruce
>
>  --Reinier Zwitserloot
>
>
>
> On Mar 27, 2009, at 23:45, Bruce Chapman wrote:
>
>> Comments inline
>>
>> thanks
>>
>> Bruce
>>
>> Reinier Zwitserloot wrote:
>>> The backtick, on a standard US 101, needs to be typed by awkwardly
>>> bending your left pinky, traditionally the least flexible typing
>>> finger. At least, for 10-fingers on the home row kind of typer. As
>>> Stefan pointed out, on international keyboards its akin to using the
>>> umlaut (without any characters under it) as a symbol.
>>>
>>>
>>> However, that's not really my issue with this proposal, this is:
>>>
>>> Who ever types or sees that backtick?
>>>
>>> 1. The source file - but, no. It doesn't even end in .java, there is
>>> no way that source could possibly be mistaken for java code.
>>>
>>> 2. The programmer - but, no. I would never use a new language (even if
>>> its folding sugar on vanilla java) if it requires me to type a
>>> gazillion backticks. I wouldn't do it even if it were a nice, easily
>>> typed symbol like, say, semi-colon. Turning my code into perl cartoon
>>> swearing surely cannot be the point. Even if I never see them (because
>>> the folder folds them away!), I don't want to hit a special 'I want
>>> sugar' modifier key, that breaks the flow just as much as a cmd-key
>>> combo. Also, You hit shift routinely as you are 'just typing', so
>>> what's the problem with hitting CMD+X or whatever key such a folding
>>> IDE comes up with?
>>>
>>> 3. The parser - but, no. A code-folding view on the AST would never be
>>> in a position to parse such a thing.
>>>
>>>
>> The proposal doesn't say how the language escape operator should be
>> used. If you are talking about the example use case then here is the
>> logic for why I think it is a good idea.
>>
>> 1. Syntactic sugar is good because it increases the signal to noise
>> ratio when humans read or write the code.
>>
>> 2. Syntactic sugar in the language has a few problems
>>    - It takes years to get it in there
>>    - Only a few very select and very universal ones get in the language
>>    - If it is done wrong then you have to live with it (no rollback)
>>    - Its existence constrains the way in which new language features
>> are implemented
>>
>> 3. So if you do syntactic sugar as a view in the IDE and say you use a
>> single character language escape operator then
>>    - You get almost all the benefits of having it in the language - at
>> the cost of typing one more character
>>    - you can choose whether you want each one separately
>>    - If you use one, it doesn't affect anyone else looking at the code
>>    - you completely avoid all the negatives.
>>
>> 4. Thats a pretty big nett gain in my book
>>
>> If you don't find any use compelling, you are under no obligation to 
>> use it.
>>>
>>> other problem:
>>>
>>> Many non-java languages already need the backtick, such as scala,
>>> which uses it to mean: The contents of the backticks are an
>>> identifier, even if it looks like a keyword. This allows you to call
>>> Thread.yield() in scala like so : Thread.`yield`(), because 'yield' in
>>> scala is a keyword, and therefore Thread.yield() would not invoke the
>>> method (it is instead a syntax error). Java has only 2 common keyboard
>>> symbols left, the backtick and the hash, and taking one of them is a
>>> big deal. I am not convinced its worth it, mostly because of the above
>>> problems.
>>>
>> Right, and so when choosing the character we need to allow for how Java
>> will implement that feature of specifying names of API elements
>> generated in other languages that are not valid Java identifiers. That
>> work is underway as far as I know.
>>>
>>>
>>> The only benefit I can see is that, IF the code-folding AST view
>>> editor does use the backtick as some kind of escaping key, that any
>>> future java expansions do not all of a sudden make previous syntax
>>> sugar legal actual java. But this really doesn't sound very good
>>> either, because:
>>>
>>> 1. The backtick really really needs to be in the source, so that if I
>>> copy and paste, it sticks around.
>> For the example use case, I would imagine that copying and pasting would
>> copy and paste the desugared form. IF you pasted it into another source
>> file in the same IDE with the sugar loaded, then it would get folded
>> back into the sugared form. IF you posted into a blog entry, the
>> desugared form would get pasted. I just don't see a problem here.
>>> A view on the AST could dynamically
>>> add it to the text buffer when you copy and paste, but this is
>>> introducing rather a lot of voodoo. There's also no way for different
>>> AST node view tools t
>>>
>>> 2. Odds are that a very useful and often used desugaring in a popular
>>> AST view editor (which we don't have yet, but hopefully someone will
>>> take the time to write one someday), will end up in java itself.
>>> Except it can't, by definition: That backtick is in the way.
>>>
>> You are presupposing how a possible use of this proposal might work. If
>> there was a really popular IDE based syntactic sugar that became so
>> ubiquitous that it made sense to put it in the language, then migration
>> could be considered at that point. However, if it was good enough in a
>> fit for purpose sense that this scenario came about, then one could ask
>> what would actually be gained by putting it in the language. I don't
>> know what would happen, it is at this stage very hypothetical. I do
>> however want to make sure that we at least have the opportunity to get
>> to that point. IF all the easily type characters got used for other
>> things, then we have lost the opportunity.
>>> 3. If, other than the backtick, the syntax is the same, or at least so
>>> similar that a key-by-key parsing is going to have issues, as a real
>>> java syntax, than the AST view desugaring syntax *needs to go away*.
>>> That point I made before but you haven't answered it yet:
>> Sorry I missed that. Could you please restate your question in and I'll
>> have a go at answering it.
>>
>>> One of the
>>> points of ast views is that you can just change everything without
>>> breaking backwards compatibility. The only compatibility you're
>>> breaking is training your programmers to use a different syntax -
>>> which they have to do anyway, because they're learning to use a new
>>> java version which shipped with at least 1 non-trivial language change
>>> (after all, something that used to be illegal in java but legal as an
>>> AST view syntax, is now legal java!) - I consider the penalty for not
>>> having a language escape operator very low as is, in other words.
>>>
>>> Take that last point, and combine it with the notion that I thoroughly
>>> do not look forward to hitting ` all the time, and you see why I don't
>>> like this proposal.
>>>
>>>
>>> This is what I would type in an AST view language:
>>>
>>> private     property int foo;
>>>
>>> and this is what it should write to disk
>>>
>>> @TypedDifferently(original="private    property int foo;")
>>> private int foo;
>>>
>>> @GeneratedBy("#foo") public Constructor(int foo) {
>>> this.foo = foo;
>>> }
>>>
>>> @GeneratedBy("#foo") public int getFoo() {
>>> return foo;
>>> }
>>>
>>> @GeneratedBy("#foo") public void setFoo(int foo) { this.foo = foo;}
>>>
>>> @GeneratedBy("#foo") public void addFooListener(.....
>>>
>>> (you get the point)
>>>
>>>
>>> but if I load this file, -with or without annotations in it-, I want
>>> to see:
>>>
>>> private     property int foo;
>>>
>>> -though- I'll accept that it might give me:
>>>
>>> property private int foo;
>>>
>>> If the @OriginallyTyped annotation is no longer there.
>>>
>>> I don't want to type backticks. I don't want to see backticks.
>>>
>>>
>>>
>>>
>>>
>>>
>>> A proposal to make *THIS* work well would be great.
>> I would really like to work on that too. Yes it is hard and a
>> significant amount of work, but I think we are at a point in history
>> where it has become feasible.
>>
>> It would be a shame if those of us interested in pursuing this later
>> found that we really needed a language escape operator to make it all
>> work from a technical and aesthetic view and while we had been
>> progressing, a couple of esoteric language changes had come along and
>> used up all the viable escape characters.  (I think coin has other
>> proposals or comments for using all of them) I just want to "stake the
>> claim" on such a character now before it is too late because I honestly
>> believe this is the most valuable way we could use one of these 
>> characters.
>>> I actually tried
>>> to write up the specs for an AST node editing language, and it's very
>>> very difficult. One of the issues you run into is that you can not
>>> save layout at all - you need to save the code to a java file, in a
>>> different structure, so it becomes almost impossible to preserve the
>>> whitespace, or things like order between keywords, or ordering between
>>> methods, when you read it back in. I thought of magic comments that
>>> keep the original syntax around, but that makes the java code ugly and
>>> not very accessible for people who are not using your particular AST
>>> desugaring engine.
>>>
>>> Something standardized there that links code to the originally typed
>>> stuff (as a comment or string literal) would allow vanilla java IDEs
>>> to not render that comment at all (their own little AST node view :P)
>>> and to notice that a user is editing a generated block, and e.g. show
>>> a little warning, or alternatively offer a view of the originally
>>> typed code, which a vanilla java editor may not understand, though
>>> likely the raw english text in the source would carry some meaning of
>>> intent, probably moreso than the generated code.
>>>
>>>
>>> Such a proposal wouldn't neccessarily fit in the JLS, it's just a
>>> standardized format that is also legal vanilla java (by using magic
>>> comments, probably), which all IDEs agree on as something they'll
>>> eventually detect. If you want special syntax for this, that would be
>>> fine too, but its okay if its verbose. You could introduce a keyword
>>> such as 'ast-node-view', which is so rare that I really doubt anyone's
>>> going to run into an issue with that no longer being a legal
>>> identifier. There's so much you can do - the backtick seems like one
>>> of the worst choices.
>>>
>>>
>> I could comment on that, but it is getting way off topic. Maybe we could
>> collaborate with other interested parties to explore this further in a
>> different  forum.
>>>  --Reinier Zwitserloot
>>>
>> Bruce
>>
>