Runtime command-line deprecation (Was Re: RFR: 8066821(S) Enhance command line processing to manage deprecating and obsoleting -XX command line arguments

[David Holmes]

Hi Derek,

One general comment inline below that I meant to make to Karen's email 
earlier ...

On 17/07/2015 9:07 AM, Derek White wrote:
> Hi Karen, Coleen,
>
> I'll respond to one issue which you both brought up here, and respond to
> the rest inline.
>
> I think you both were skeptical of the need to "deprecate & handle" an
> argument (where the argument isn't an alias for another argument),
> although Karen suggested that might be good for external option removal,
> which has a really long lead time.
>
>  From the options I've looked at, the decision to ignore vs. handle a
> deprecated option depends on what the option does and what users expect.
> For simple tuning or diagnostic options, there is little harm if we
> don't handle "-XX:+ UseOldInlining". The user would be hard-pressed to
> tell the difference. But if we wanted to deprecate "-XX:+UseG1GC" (for
> an unlikely example), this would make a big difference if we simply
> ignored the option. In this case we should still handle the option after
> emitting a deprecation warning (for at least one release).
>
> Should we ever go from "deprecate & handle" in one release to full
> removal in the next? Or should we always have a release that "deprecates
> & ignores" an option before removing it? Ignoring an old option is often
> a convenience for customers, but on the other hand it's kind of lying to
> the customers. Maybe they're asking for "XX:-MSG" and we say "sure boss"
> and pour on the monosodium glutamate :-)
>
> On the other hand, really handling a deprecated option has a cost too,
> so code complexity (and reliability), or time and space overheads might
> call for simply ignoring a deprecated option instead of always handling
> it in one release and ignoring it in the next.
>
> Coming up with some guidelines for this is a good idea, but I don't
> think hard rules will work.
>
> Specific responses below.
>
> FYI  - New webrev is going through last round of merge testing.
>
>   - Derek
>
> On 7/6/15 5:00 PM, Coleen Phillimore wrote:
>>
>> Hi,
>>
>> I'm happy for more clarity in the process of removing command line
>> arguments.  I have some questions and comments below.
>>
>> On 6/26/15 4:09 PM, Karen Kinnear wrote:
>>> Derek,
>>>
>>> I am really glad you are looking into this. I expanded this to the
>>> runtime folks in case many of them have not yet seen this.
>>> Mary just forwarded this and I believe you haven't checked it in yet,
>>> so perhaps still time to discuss and make sure
>>> we all are together on the deprecation policy.
>>>
>>> The runtime team was discussing at staff this week how we handle
>>> deprecating jvm command-line options as
>>> we are looking to do more clean up in the future.
>>>
>>> So our internal change control process classifies our exported
>>> interfaces into three categories:
>>> External: command-line flags e.g. -verbose:gc, -Xmx, ...
>>>      This includes any commercial flags
>>> Private: -XX flags documented (e.g. performance tuning or
>>> troubleshooting)
>>>      I would assume this would include -XX product, experimental
>>> flags and manageable flags
>>> Internal: undocumented -XX flags
>>>      I would assume this would include -XX develop and diagnostic flags
>>>
>>> (please correct me if my assumptions are wrong folks)
>>
>> This is a good categorization.  Although I think there's some grey
>> area between External and Private, where some XX options are so
>> commonly used they should be considered External.   Some of the GC
>> options may fall into this category like UseParNewGC.
> Yes, I agree that there is unexpected flag promotion. It would be great
> to get a common understanding of where the lines are between External,
> Private, and Internal. The flag types like commercial, product,
> experimental, develop, diagnostic are important, although I'd argue that
> experimental flags are more likely "internal" not private. I think we
> also need a liberal definition of "Documented" to include not just
> official documentation but blog postings and even serious mentions in
> places like StackOverflow. If the only reference that Google can find to
> an -XX flag is in the openJDK source code, then chances are it is
> "Internal".

The classifications that Karen listed come from a document that 
pre-dates open-sourcing of Java by many years. In an OpenSource world 
all flags are "documented" - and there are a few webpages that serve as 
tables of VM options for everyone to see. Not to mention all the 
publicly visible bug reports. So a new definition of "documented" is 
certainly needed.

I also think we need definitions based more on the type of flag: 
product, develop, diagnostic, experimental, commercial - as a first 
level classification. With the concerns of external/internal/private 
applying as a secondary classification when needed. Obviously not all 
combinations make sense.

I agree that in the worst-case there could well be a three phase process:
  - deprecate and handle
  - deprecate and ignore
  - remove (give error on use)

but that should only be needed for external, product flags - those with 
the broadest use-base and exposure. Most will need the second two stages 
only. And for some we should be able to just rip them out whenever eg 
experimental-internal/private.

Aside: in the JDK deprecation has mostly been phase one only, 
unfortunately, with a handful of phase two. Alas no stage three.

Cheers,
David
------

>>> The way I understand that we handle private -XX options today is a
>>> 2-step removal: (obsolete_jvm_flags - where the
>>> release number is in a table and could be undefined)
>>>
>>> Using your helpful taxonomy
>>> fromhttps://bugs.openjdk.java.net/browse/JDK-806682:
>>>
>>> Today: private -XX options use 2-step removal (obsolete_jvm_flags)
>>>      release 1: Deprecate & Obsolete - warn about the option but do
>>> nothing with it (we can remove all code that supports it)
>>>      release 2: Dead - unrecognized
>>>    - the point of the 2-step is to give customers time to modify any
>>> scripts they use
>>>
>>> I believe we have very rarely removed External flags - since
>>> customers, licensees, etc. may expect them.
>>>
>>> Mini-proposal:
>>> 1) I would recommend that we add a future set of changes to add
>>> consistent handling for the External flags -
>>> so that they would follow a three-step removal:
>>>      release 1: Deprecate & Handle - warn and keep supporting
>>>      release 2: Deprecate & Obsolete - warn and do nothing
>>>      release 3: Dead - unrecognized
>>>
>>> 2) For the Internal flags - I think it would be kindest to customers
>>> and not a huge amount of additional work if
>>> we were to follow the Private model of using a 2 step.
>>>
>>> 3) leave the Private flags with the current 2-step removal
>>
>> Yes, this reflects our current model.
>>> 4) add support for aliasing - for any of them
>>>      So that if you are doing aliasing, you would follow the model
>>> you are adding
>>>       release 1: Deprecated & Handled - i.e. warn and still support
>>> (and set the information for the new alias)
>>>       release 2: Dead - unrecognized
>>>
>>> 5) Could we possibly take the two flags that followed a different
>>> model already, i.e. moving to
>>> Deprecated & Handled and handle those as mistakes rather than part of
>>> a new general model?
>>
>> So this is my question which is around the code review in question.
>> Note, Derek, can you resend the RFR to hotspot-dev at openjdk.java.net so
>> it gets to all of us (even the compiler team may want to share in the
>> mechanism).
>>
>> Why are UseParNewGC and MaxGCMinorPauseMillis deprecated and handled
>> and not deprecated and obsolete?  I don't actually see why have the
>> distinction here?
>>
>> Did these flags follow the deprecation procedure below?  Why not just
>> make them obsolete, why have this other mechanism?  I may be asking
>> the same question as Karen.
>>
>> I think the aliased flags could have a deprecate and handle model
>> (where handle == alias) in which case you only need one table for the
>> aliased flags, and you need to keep them in globals.hpp so they're
>> parsed properly.
>
> The real reason I wrote the code that way is because I was supporting
> the current policy (classic "mechanism not policy" checkin). I didn't
> (and don't) have the history on how we got there. But after thinking
> about it (see top post), I think there will be some cases where we'll
> have handle options instead of ignoring them and vice verse. And the
> final decision may come down to engineering, or customer, or partner
> concerns, or management whimsy, or whatever the current fad is. We're
> talking about code changes over multiple years here :-)
>>> Or do you think we will see more cases other than aliasing in which
>>> we would want to
>>>      release 1: Deprecate & Handle and then release 2: Dead
>>>      rather than release 1: Deprecate & Obsolete and then 2: Dead
>>>      or rather than a 3 step like the External option proposal above?
>>
>> I think for the aliased flags you want the first option here
>> (deprecate&handle), right?  But that's only because you need to keep
>> the option in globals.hpp so it parses correctly, otherwise the second
>> option (deprecate&obsolete) would be the preference?
>>
>> The options in the GC that are being deprecated and handled have had
>> warnings about them for a while, so making them obsolete doesn't feel
>> too soon.
>
> Yes, but just went through the approval process.
>>
>> Also, I agree with Kim's comment below that your comment lines are too
>> long.  My fonts are too big to have windows this wide.
>>
>> thanks,
>> Coleen
>>> thanks,
>>> Karen
>>>
>>> p.s. Note that all of the deprecation needs to
>>> 1) work with licensee engineering to ensure we give licensee's a
>>> head's up and get feedback
>>> 2) file a change control request
>>>
>>> - we try to do these both as bulk requests to reduce the processing
>>> overhead.
>>>
>>> p.p.s. Details
>>>
>>> 1.  Do the warnings print today or are they silent? Just want to make
>>> sure we are conscious of how
>>> those are handled if any of this moves to the new unified logging
>>> mechanism for which the new default
>>> for "warning" level is to print.
> The deprecation messages go through warning(), which is controlled by
> PrintWarnings (defaults to true). This is how the obsolete flag warnings
> are printed. The new deprecation mechanism replaces a mishmash of calls
> to jio_fprintf() and warning(), with a variety of different ways of
> saying the same thing.
>>> 2. "will likely be removed in a future release"? If we have already
>>> set the release it will be removed - is this a bit
>>> vague or did I not read closely enough?
> That text came from some of the deprecated GC options. If removal has
> been scheduled, we could say something more definite, or even the exact
> release. We don't print the exact "death" release for obsolete options
> currently though.
>>>
>>> On Feb 3, 2015, at 6:30 PM, Derek White wrote:
>>>
>>>> Request for review (again):
>>>>
>>>> - Updated with Kim's suggestion.
>>>> - Stopped double-printing warnings in some cases.
>>>> - Initial caps on warning() messages.
>>>>
>>>> Webrev:http://cr.openjdk.java.net/~drwhite/8066821/webrev.04/
>>>> CR:https://bugs.openjdk.java.net/browse/JDK-8066821
>>>> Tests:     jtreg, jprt
>>>>
>>>> Thanks for looking!
>>>>
>>>> - Derek
>>>>
>>>> On 1/28/15 3:47 PM, Kim Barrett wrote:
>>>>
> ...