General community and Process Questions (blog comment followups)

[Joseph D. Darcy]

Hello.

>  Hi all,
>
>  I'm hoping to productively bring your attention to an off-list
>  discussion. Recently the jdk7 announce blog "The Planetarium" posted
>  a short teaser on the state of the upcoming jdk7 language changes
> 
(http://blogs.sun.com/theplanetarium/entry/summarizing_java_language_changes_for).
>  As often seems to happen (and I'll admit to being part of), the
>  comments turned into a negative compliant-fest on current proposal
>  set and process for coin.

Yes, I had seen the Planetarium post and responding to the concerns you
raise below on this list is the right forum.

[snip]

>  So here goes..
>
>  a) The two proposals initially mentioned by the first blog commenter,
>  regex or non-backslash escaping and multi-line string declarations,
>  seem to meet the project coin mission perfectly. They have a large
>  positive usability impact and a large group of casual advocates,
>  additionally, from those unfamiliar with the jdk internals, they seem
>  like "small" changes already present in most other modern languages.
>
>
>  However, (from the outside) it appears that those proposals have been
>  dropped and it's unclear why. I believe that this has lead to some
>  people into dismissing the entire process as capricious and opaque.
>  Would it be possible to get an insiders explanation on why these
>  proposals are off the table? Is it just that no one wrote a formal
>  proposal?
>

[snip]

As a general comment, as I've stated before Project Coin's call for 
proposals phase [1], the overriding responsibility is for the proposer 
(or other advocate for the change) to positively demonstrate why a 
change should get *in*, the responsibility is not to prove that a change 
should stay out.   At a minimum that requires a proposal to stay within 
the size bounds in all of the specification, implementation, and testing 
dimensions. [2]  Additionally, even if a Coin Proposal is in scope 
size-wise, the proposal must also be a good way to evolve Java.

Detailing why a proposal fails in one or more of these ways is very time 
consuming; indeed, for some of the proposals that were sent in, writing 
a rebuttal could take longer than the time spent on the proposal ;-)  
Given the nearly 70 proposals that were sent it, it is not practical to 
provide a detailed analysis of all or even most proposals since it could 
easily consume several solid months of engineering time, engineering 
time that would be more productively directed at implementing good 
proposals.  Even if the analysis were done, at the end of those months, 
I'm confident not all the advocates would be convinced of the flaws in 
their proposals and there would be many more proposals eager to be 
evaluated if that evaluation continued to be offered.

Many proposals sent into Coin were not well written and lacked 
prototypes.  While the string proposal in question was sent in early, 
went through several helpful iterations of refinement on the list, and 
laudably included a prototype implementation, in the end I thought the 
proposal provided limited benefits over the alternatives for the 
complexity it introduced.

>  This leads to
>
>  b) What is the process for finalizing the changes that go into coin
>  and then get folded into the jdk. I understand that people who
>  participate here have some influence, but who has final say? Does the
>  whole thing get wrapped up in a JSR and get voted? Is it just a
>  judgement call of the internal Sun architects? Understanding the
>  process would really help it feel more 'open' and I think get people
>  psyched about contributing.

At the moment, informed by the discussions on the mailing list and in 
consultation with people inside and outside of Sun, I have the final say 
to determine the list of small language changes for JDK 7.  As has been 
previously stated, our intention is to file a JSR for the language 
changes after the feature list has been determined.  Having been a spec 
lead previously, JSRs are not democracies either, although consensus 
driven decision making is preferred and encouraged.

There are numerous reasons for this decision making arrangement, some of 
which I will discuss below.

The blog entries I wrote last year and before [3] [4] about experiences 
evolving the Java language were intended to allow interested people to 
participate much more fully in the language evolution process by 
providing context about what work is needed and what some of the 
concerns are.  The Coin list has been a good start to building the skill 
level for broader participation.  However, any feature added to the Java 
programming language must be supported *FOREVER* and only Sun has been 
willing to do that. That is why we reserve the right to have the final 
word over language decisions.

In the context of Project Coin, just sending in a rough idea or request 
for a language change essentially has no value since there are already 
many, many more ideas for reasonable languages changes than can be 
accommodated in JDK 7 or any subsequent release.  The hard part, and the 
most valuable part, is the work analyzing the impact and benefit of a 
change, which ideally includes a prototype.  Project Coin invited 
community participation in that work.

"Design by committee" is often derided as an inappropriate way to manage 
technical projects.  Simple polling about technical issues is design by 
committee where the committee can be arbitrarily large and any pretense 
of expertise in the area is removed.  Therefore, polling would be a poor 
way to drive specific technical decisions about the Java Programming 
Language. One of the benefits of working in a technical field is that 
technical problems often have right answers, regardless of how many 
people agree or disagree with them.

This is not intended to be a slight against Java programmers who 
contributed suggestions informed by their daily experiences with the 
language to the Coin alias, to Sun's bug database, or elsewhere.  
Rather, it is a recognition that, just as being a player and being a 
coach are district roles requiring distinct skills, using the language 
and evolving it and district tasks with related but separate skills 
sets. Polling can provide a good indication about what pain points 
people are experiencing; that is an important input, but only one kind 
of input, to how the language should change.

Most Java programmers do not need to be language experts.  This is very 
much a feature and *not* a bug of the Java ecosystem.  Not having to be 
a language expert means Java programmers have time to be experts about 
other things :-)

One of Project Coin's goals is to build up a larger body of expertise 
outside of Sun that can analyze, quantify, measure, and evaluate 
developments in the Java language..  A good way to get influence is to 
do some work.  I've been a bit disappointed in several aspects of the 
coin contributions.  For example, back in May there was an open issue 
concerning the grammar changes needed for underscores in numbers [5].  
No one sent in a message on this matter until I sent in a grammar in 
July [6].   Neither the coin list nor readers on my blog caught the 
embarrassing but small mistake I made in the grammar [7].  In terms of 
difficulty of language evolution, this grammar change is easy.  Having 
taken an undergraduate class in compilers or automata theory is 
sufficient background to work on this, but no one else contributed a 
grammar for this despite presumed interest in the feature and the large 
number of coin subscribers.

I am not aware of any independent assessment of the utility of sets of 
coin features together or other systematic analysis of the proposals.

Collectively the Coin community to date has neither managed to get all 
the small details of proposals worked out nor provided an alternative 
larger view of the potential goals for the effort.  Sun has been working 
on the Java language for many years, is in a position to get input and 
feedback from numerous parties (including Coin!), understands the need 
for thorough analysis and mature evaluation, and has provided ongoing 
support.  Therefore ultimate decision making on Java language changes 
will stay with Sun.

All the code needed to develop a language change is now available from 
OpenJDK.  People are now able to "scratch their own itch" in 
experimenting with language changes and for those not wanting to work 
directly under the auspices of the OpenJDK umbrella project, there are 
alternatives (https://kijaro.dev.java.net/).  I encourage those who want 
to see more language changes in Java to complain less and code (and 
propose and test and analyze) more; that is the best way to be 
convincing that a change should go into Java.  As a counterpoint, I 
counsel against the too-common habit of showing up with no money and no 
technical expertise and expecting others to drop their own priorities to 
undertake a massive engineering project on another's behalf. [8]

Regards,

-Joe

[1] "Criteria for desirable small language changes,"
http://blogs.sun.com/darcy/entry/criteria_for_desirable_small_language

[2] "Guidance on measuring the size of a language change"
http://blogs.sun.com/darcy/entry/guidance_measure_language_change_size

[3] "Project Coin: Small Language Change Proposal Form Available,"
http://blogs.sun.com/darcy/entry/project_coin

[4] "So you want to change the Java Programming Language...",
http://blogs.sun.com/darcy/entry/so_you_want_to_change

[5] http://mail.openjdk.java.net/pipermail/coin-dev/2009-May/001882.html

[6] http://mail.openjdk.java.net/pipermail/coin-dev/2009-July/002080.html

[7] http://mail.openjdk.java.net/pipermail/coin-dev/2009-July/002110.html

[8] http://mail.openjdk.java.net/pipermail/jdk7-dev/2009-June/000666.html
http://mail.openjdk.java.net/pipermail/jdk7-dev/2009-June/000691.html
http://mail.openjdk.java.net/pipermail/jdk7-dev/2009-June/000706.html