Runtime command-line deprecation (Was Re: RFR: 8066821(S) Enhance command line processing to manage deprecating and obsoleting -XX command line arguments

Fri Jul 17 13:29:27 UTC 2015

One tiny correction below...

On 7/16/15 5:52 PM, David Holmes wrote:
> Hi Derek,
>
> One general comment inline below that I meant to make to Karen's email 
> earlier ...
>
> On 17/07/2015 9:07 AM, Derek White wrote:
>> Hi Karen, Coleen,
>>
>> I'll respond to one issue which you both brought up here, and respond to
>> the rest inline.
>>
>> I think you both were skeptical of the need to "deprecate & handle" an
>> argument (where the argument isn't an alias for another argument),
>> although Karen suggested that might be good for external option removal,
>> which has a really long lead time.
>>
>>  From the options I've looked at, the decision to ignore vs. handle a
>> deprecated option depends on what the option does and what users expect.
>> For simple tuning or diagnostic options, there is little harm if we
>> don't handle "-XX:+ UseOldInlining". The user would be hard-pressed to
>> tell the difference. But if we wanted to deprecate "-XX:+UseG1GC" (for
>> an unlikely example), this would make a big difference if we simply
>> ignored the option. In this case we should still handle the option after
>> emitting a deprecation warning (for at least one release).
>>
>> Should we ever go from "deprecate & handle" in one release to full
>> removal in the next? Or should we always have a release that "deprecates
>> & ignores" an option before removing it? Ignoring an old option is often
>> a convenience for customers, but on the other hand it's kind of lying to
>> the customers. Maybe they're asking for "XX:-MSG" and we say "sure boss"
>> and pour on the monosodium glutamate :-)
>>
>> On the other hand, really handling a deprecated option has a cost too,
>> so code complexity (and reliability), or time and space overheads might
>> call for simply ignoring a deprecated option instead of always handling
>> it in one release and ignoring it in the next.
>>
>> Coming up with some guidelines for this is a good idea, but I don't
>> think hard rules will work.
>>
>> Specific responses below.
>>
>> FYI  - New webrev is going through last round of merge testing.
>>
>>   - Derek
>>
>> On 7/6/15 5:00 PM, Coleen Phillimore wrote:
>>>
>>> Hi,
>>>
>>> I'm happy for more clarity in the process of removing command line
>>> arguments.  I have some questions and comments below.
>>>
>>> On 6/26/15 4:09 PM, Karen Kinnear wrote:
>>>> Derek,
>>>>
>>>> I am really glad you are looking into this. I expanded this to the
>>>> runtime folks in case many of them have not yet seen this.
>>>> Mary just forwarded this and I believe you haven't checked it in yet,
>>>> so perhaps still time to discuss and make sure
>>>> we all are together on the deprecation policy.
>>>>
>>>> The runtime team was discussing at staff this week how we handle
>>>> deprecating jvm command-line options as
>>>> we are looking to do more clean up in the future.
>>>>
>>>> So our internal change control process classifies our exported
>>>> interfaces into three categories:
>>>> External: command-line flags e.g. -verbose:gc, -Xmx, ...
>>>>      This includes any commercial flags
>>>> Private: -XX flags documented (e.g. performance tuning or
>>>> troubleshooting)
>>>>      I would assume this would include -XX product, experimental
>>>> flags and manageable flags
>>>> Internal: undocumented -XX flags
>>>>      I would assume this would include -XX develop and diagnostic 
>>>> flags
>>>>
>>>> (please correct me if my assumptions are wrong folks)
>>>
>>> This is a good categorization.  Although I think there's some grey
>>> area between External and Private, where some XX options are so
>>> commonly used they should be considered External.   Some of the GC
>>> options may fall into this category like UseParNewGC.
>> Yes, I agree that there is unexpected flag promotion. It would be great
>> to get a common understanding of where the lines are between External,
>> Private, and Internal. The flag types like commercial, product,
>> experimental, develop, diagnostic are important, although I'd argue that
>> experimental flags are more likely "internal" not private. I think we
>> also need a liberal definition of "Documented" to include not just
>> official documentation but blog postings and even serious mentions in
>> places like StackOverflow. If the only reference that Google can find to
>> an -XX flag is in the openJDK source code, then chances are it is
>> "Internal".
>
> The classifications that Karen listed come from a document that 
> pre-dates open-sourcing of Java by many years. In an OpenSource world 
> all flags are "documented" - and there are a few webpages that serve 
> as tables of VM options for everyone to see. Not to mention all the 
> publicly visible bug reports. So a new definition of "documented" is 
> certainly needed.
>
> I also think we need definitions based more on the type of flag: 
> product, develop, diagnostic, experimental, commercial - as a first 
> level classification. With the concerns of external/internal/private 
> applying as a secondary classification when needed. Obviously not all 
> combinations make sense.
>
> I agree that in the worst-case there could well be a three phase process:
>  - deprecate and handle
>  - deprecate and ignore
>  - remove (give error on use)
>
> but that should only be needed for external, product flags - those 
> with the broadest use-base and exposure. Most will need the second two 
> stages only. And for some we should be able to just rip them out 
> whenever eg experimental-internal/private.
>
> Aside: in the JDK deprecation has mostly been phase one only, 
> unfortunately, with a handful of phase two. Alas no stage three.

There's at least one stage 3 in the JDK:

java.lang.Thread.suspend() throws UnsupportedOperationException
since JDK8-B96.

https://bugs.openjdk.java.net/browse/JDK-7059085

http://hg.openjdk.java.net/jdk9/dev/jdk/annotate/669be1677ab7/src/share/classes/java/lang/Thread.java#l848

Dan

>
> Cheers,
> David
> ------
>
>>>> The way I understand that we handle private -XX options today is a
>>>> 2-step removal: (obsolete_jvm_flags - where the
>>>> release number is in a table and could be undefined)
>>>>
>>>> Using your helpful taxonomy
>>>> fromhttps://bugs.openjdk.java.net/browse/JDK-806682:
>>>>
>>>> Today: private -XX options use 2-step removal (obsolete_jvm_flags)
>>>>      release 1: Deprecate & Obsolete - warn about the option but do
>>>> nothing with it (we can remove all code that supports it)
>>>>      release 2: Dead - unrecognized
>>>>    - the point of the 2-step is to give customers time to modify any
>>>> scripts they use
>>>>
>>>> I believe we have very rarely removed External flags - since
>>>> customers, licensees, etc. may expect them.
>>>>
>>>> Mini-proposal:
>>>> 1) I would recommend that we add a future set of changes to add
>>>> consistent handling for the External flags -
>>>> so that they would follow a three-step removal:
>>>>      release 1: Deprecate & Handle - warn and keep supporting
>>>>      release 2: Deprecate & Obsolete - warn and do nothing
>>>>      release 3: Dead - unrecognized
>>>>
>>>> 2) For the Internal flags - I think it would be kindest to customers
>>>> and not a huge amount of additional work if
>>>> we were to follow the Private model of using a 2 step.
>>>>
>>>> 3) leave the Private flags with the current 2-step removal
>>>
>>> Yes, this reflects our current model.
>>>> 4) add support for aliasing - for any of them
>>>>      So that if you are doing aliasing, you would follow the model
>>>> you are adding
>>>>       release 1: Deprecated & Handled - i.e. warn and still support
>>>> (and set the information for the new alias)
>>>>       release 2: Dead - unrecognized
>>>>
>>>> 5) Could we possibly take the two flags that followed a different
>>>> model already, i.e. moving to
>>>> Deprecated & Handled and handle those as mistakes rather than part of
>>>> a new general model?
>>>
>>> So this is my question which is around the code review in question.
>>> Note, Derek, can you resend the RFR to hotspot-dev at openjdk.java.net so
>>> it gets to all of us (even the compiler team may want to share in the
>>> mechanism).
>>>
>>> Why are UseParNewGC and MaxGCMinorPauseMillis deprecated and handled
>>> and not deprecated and obsolete?  I don't actually see why have the
>>> distinction here?
>>>
>>> Did these flags follow the deprecation procedure below?  Why not just
>>> make them obsolete, why have this other mechanism?  I may be asking
>>> the same question as Karen.
>>>
>>> I think the aliased flags could have a deprecate and handle model
>>> (where handle == alias) in which case you only need one table for the
>>> aliased flags, and you need to keep them in globals.hpp so they're
>>> parsed properly.
>>
>> The real reason I wrote the code that way is because I was supporting
>> the current policy (classic "mechanism not policy" checkin). I didn't
>> (and don't) have the history on how we got there. But after thinking
>> about it (see top post), I think there will be some cases where we'll
>> have handle options instead of ignoring them and vice verse. And the
>> final decision may come down to engineering, or customer, or partner
>> concerns, or management whimsy, or whatever the current fad is. We're
>> talking about code changes over multiple years here :-)
>>>> Or do you think we will see more cases other than aliasing in which
>>>> we would want to
>>>>      release 1: Deprecate & Handle and then release 2: Dead
>>>>      rather than release 1: Deprecate & Obsolete and then 2: Dead
>>>>      or rather than a 3 step like the External option proposal above?
>>>
>>> I think for the aliased flags you want the first option here
>>> (deprecate&handle), right?  But that's only because you need to keep
>>> the option in globals.hpp so it parses correctly, otherwise the second
>>> option (deprecate&obsolete) would be the preference?
>>>
>>> The options in the GC that are being deprecated and handled have had
>>> warnings about them for a while, so making them obsolete doesn't feel
>>> too soon.
>>
>> Yes, but just went through the approval process.
>>>
>>> Also, I agree with Kim's comment below that your comment lines are too
>>> long.  My fonts are too big to have windows this wide.
>>>
>>> thanks,
>>> Coleen
>>>> thanks,
>>>> Karen
>>>>
>>>> p.s. Note that all of the deprecation needs to
>>>> 1) work with licensee engineering to ensure we give licensee's a
>>>> head's up and get feedback
>>>> 2) file a change control request
>>>>
>>>> - we try to do these both as bulk requests to reduce the processing
>>>> overhead.
>>>>
>>>> p.p.s. Details
>>>>
>>>> 1.  Do the warnings print today or are they silent? Just want to make
>>>> sure we are conscious of how
>>>> those are handled if any of this moves to the new unified logging
>>>> mechanism for which the new default
>>>> for "warning" level is to print.
>> The deprecation messages go through warning(), which is controlled by
>> PrintWarnings (defaults to true). This is how the obsolete flag warnings
>> are printed. The new deprecation mechanism replaces a mishmash of calls
>> to jio_fprintf() and warning(), with a variety of different ways of
>> saying the same thing.
>>>> 2. "will likely be removed in a future release"? If we have already
>>>> set the release it will be removed - is this a bit
>>>> vague or did I not read closely enough?
>> That text came from some of the deprecated GC options. If removal has
>> been scheduled, we could say something more definite, or even the exact
>> release. We don't print the exact "death" release for obsolete options
>> currently though.
>>>>
>>>> On Feb 3, 2015, at 6:30 PM, Derek White wrote:
>>>>
>>>>> Request for review (again):
>>>>>
>>>>> - Updated with Kim's suggestion.
>>>>> - Stopped double-printing warnings in some cases.
>>>>> - Initial caps on warning() messages.
>>>>>
>>>>> Webrev:http://cr.openjdk.java.net/~drwhite/8066821/webrev.04/
>>>>> CR:https://bugs.openjdk.java.net/browse/JDK-8066821
>>>>> Tests:     jtreg, jprt
>>>>>
>>>>> Thanks for looking!
>>>>>
>>>>> - Derek
>>>>>
>>>>> On 1/28/15 3:47 PM, Kim Barrett wrote:
>>>>>
>> ...