From ihse at openjdk.java.net  Mon Feb  1 11:14:59 2021
From: ihse at openjdk.java.net (Magnus Ihse Bursie)
Date: Mon, 1 Feb 2021 11:14:59 GMT
Subject: [jdk16] RFR: 8258378: Final nroff manpage update for JDK 16
Message-ID: <2nAapGYDGTih_WWtZngNa0s8ZBc3ZKOSgz1MEZMHGYg=.a80e96ee-dd5a-42bd-80d7-1f83e72b9686@github.com>

Before RC phase we need to ensure we have the final set of manpage updates published in OpenJDK.

-------------

Commit messages:
 - 8258378: Final nroff manpage update for JDK 16

Changes: https://git.openjdk.java.net/jdk16/pull/142/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk16&pr=142&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8258378
  Stats: 238 lines in 27 files changed: 98 ins; 111 del; 29 mod
  Patch: https://git.openjdk.java.net/jdk16/pull/142.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk16 pull/142/head:pull/142

PR: https://git.openjdk.java.net/jdk16/pull/142

From github.com+7806504+liach at openjdk.java.net  Mon Feb  1 19:18:44 2021
From: github.com+7806504+liach at openjdk.java.net (liach)
Date: Mon, 1 Feb 2021 19:18:44 GMT
Subject: RFR: 8241356: Use a more reliable way to encode Symbol flags
In-Reply-To: <aYRbIKEZmDlWfryC0sW1BSVtzJyIR7BRZS2Vs0XfGaY=.de31b4a7-d2df-489a-9816-08deab0cddef@github.com>
References: <aYRbIKEZmDlWfryC0sW1BSVtzJyIR7BRZS2Vs0XfGaY=.de31b4a7-d2df-489a-9816-08deab0cddef@github.com>
Message-ID: <FSyY4LXJeGt-T6BQ2nSjnpJt5PUP-QS55TrbO8FGdAw=.46ed160b-cd1f-4e6b-a982-06c46bbd8ecc@github.com>

On Fri, 29 Jan 2021 17:31:20 GMT, Jan Lahoda <jlahoda at openjdk.org> wrote:

> [This is a GitHub copy of: https://mail.openjdk.java.net/pipermail/compiler-dev/2020-March/014389.html ]
> 
> Currently, (com.sun.tools.javac.code.)Symbol/s have a long field "flags_field", which holds various one-bit information ("Flags") about the given Symbol.
> 
> We currently have around 64 such Flags, which means we are out of bits in the long field, and adding new flags is not easy.
> 
> We could change the "flags_field" to be a Set over enums, but this would increase the memory footprint notably, and would also slow-down access to flags. Currently, flags operations in javac are very fast and very common, so this is probably too much burden. There are also flags to which we need to access as bit masks, e.g. due to writing to classfile.
> 
> My proposal here is to use an intermediate solution, until we find a better solution, or until a better solution is possible (like due to Valhalla). The idea is as follows:
> -the current long-based Flags are split into 4 groups:
> --"flat" Flags, long based, work exactly as before.
> --enum-based TypeSymbolFlags, MethodSymbolFlags and VarSymbolFlags, which are only applicable to TypeSymbols, MethodSymbols and VarSymbols, respectively, and are checked using methods like Symbol.isFlagSet and set/clear using methods Symbol.setFlag and Symbol.clearFlag. So these flags are mostly encapsulated, even though physically they are currently stored in the flags_field as well.
> 
> There are ~~37~~ 40 "flat" flags and ~~16~~ 17 TypeSymbolFlags (methods and vars have less flags), 57 in total. This gives us at least 7 new flags before we run out of long bits in flags_field again - but even if we do, there are several easy mitigation strategies we could use, like:
> -create a new int/long field on TypeSymbols for the TypeSymbolFlags (probably preferable)
> -split TypeSymbolFlags into Class/Package/MethodSymbolFlags
> 
> The positives of this solution include:
> -safe(r) access to the flags - the access to the extra/symbol-kind-specific flags is mostly encapsulated, and hence mostly safe
> -improves the abstractions at least for some flags
> -reasonably complex patch (attempts to encapsulate all the flags were troublesome in previous experiments)
> -the performances appears to be acceptable.
> 
> The negative that I see is that the code includes several incompatible types of flags now. These cannot be easily confused by accident (as the type system will complain), but still we need to be aware of them when writing code.
> 
> Some more alternatives:
> -add a new field to Symbol, like long extraFlags, and continue with bit masks as we did so far using this new field. The drawback is that it is more error prone (it would be difficult to ensure only the new/extra flags would be stored and read from extraFlags).
> -have a new field, and "flat" and non-"flat" enum-based encapsulated Flags, but don't separate Type/Method/Var flags. Probably something to consider, even though I am a bit reluctant to add new fields without trying to avoid it .
> 
> Any feedback is welcome!

src/jdk.compiler/share/classes/com/sun/tools/javac/code/Flags.java line 544:

> 542: //        RECORD(Flags.RECORD),
> 543: //        RECOVERABLE(Flags.RECOVERABLE),
> 544: //>>>>>>> master

This commented out merge conflict stub should be removed.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2316

From jlahoda at openjdk.java.net  Tue Feb  2 16:28:01 2021
From: jlahoda at openjdk.java.net (Jan Lahoda)
Date: Tue, 2 Feb 2021 16:28:01 GMT
Subject: RFR: 8241356: Use a more reliable way to encode Symbol flags [v2]
In-Reply-To: <aYRbIKEZmDlWfryC0sW1BSVtzJyIR7BRZS2Vs0XfGaY=.de31b4a7-d2df-489a-9816-08deab0cddef@github.com>
References: <aYRbIKEZmDlWfryC0sW1BSVtzJyIR7BRZS2Vs0XfGaY=.de31b4a7-d2df-489a-9816-08deab0cddef@github.com>
Message-ID: <BeN0ieY-9mauOUxv6aUlC3VU7lvOy7TA4_n4soZwZ9Y=.806f69e0-635c-4f72-8f75-89c73668572d@github.com>

> [This is a GitHub copy of: https://mail.openjdk.java.net/pipermail/compiler-dev/2020-March/014389.html ]
> 
> Currently, (com.sun.tools.javac.code.)Symbol/s have a long field "flags_field", which holds various one-bit information ("Flags") about the given Symbol.
> 
> We currently have around 64 such Flags, which means we are out of bits in the long field, and adding new flags is not easy.
> 
> We could change the "flags_field" to be a Set over enums, but this would increase the memory footprint notably, and would also slow-down access to flags. Currently, flags operations in javac are very fast and very common, so this is probably too much burden. There are also flags to which we need to access as bit masks, e.g. due to writing to classfile.
> 
> My proposal here is to use an intermediate solution, until we find a better solution, or until a better solution is possible (like due to Valhalla). The idea is as follows:
> -the current long-based Flags are split into 4 groups:
> --"flat" Flags, long based, work exactly as before.
> --enum-based TypeSymbolFlags, MethodSymbolFlags and VarSymbolFlags, which are only applicable to TypeSymbols, MethodSymbols and VarSymbols, respectively, and are checked using methods like Symbol.isFlagSet and set/clear using methods Symbol.setFlag and Symbol.clearFlag. So these flags are mostly encapsulated, even though physically they are currently stored in the flags_field as well.
> 
> There are ~~37~~ 40 "flat" flags and ~~16~~ 17 TypeSymbolFlags (methods and vars have less flags), 57 in total. This gives us at least 7 new flags before we run out of long bits in flags_field again - but even if we do, there are several easy mitigation strategies we could use, like:
> -create a new int/long field on TypeSymbols for the TypeSymbolFlags (probably preferable)
> -split TypeSymbolFlags into Class/Package/MethodSymbolFlags
> 
> The positives of this solution include:
> -safe(r) access to the flags - the access to the extra/symbol-kind-specific flags is mostly encapsulated, and hence mostly safe
> -improves the abstractions at least for some flags
> -reasonably complex patch (attempts to encapsulate all the flags were troublesome in previous experiments)
> -the performances appears to be acceptable.
> 
> The negative that I see is that the code includes several incompatible types of flags now. These cannot be easily confused by accident (as the type system will complain), but still we need to be aware of them when writing code.
> 
> Some more alternatives:
> -add a new field to Symbol, like long extraFlags, and continue with bit masks as we did so far using this new field. The drawback is that it is more error prone (it would be difficult to ensure only the new/extra flags would be stored and read from extraFlags).
> -have a new field, and "flat" and non-"flat" enum-based encapsulated Flags, but don't separate Type/Method/Var flags. Probably something to consider, even though I am a bit reluctant to add new fields without trying to avoid it .
> 
> Any feedback is welcome!

Jan Lahoda has updated the pull request incrementally with one additional commit since the last revision:

  Removing merge tags are noted on the review - thanks!

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2316/files
  - new: https://git.openjdk.java.net/jdk/pull/2316/files/a9c3ff6e..27e37860

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2316&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2316&range=00-01

  Stats: 7 lines in 1 file changed: 0 ins; 7 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2316.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2316/head:pull/2316

PR: https://git.openjdk.java.net/jdk/pull/2316

From mcimadamore at openjdk.java.net  Wed Feb  3 10:54:42 2021
From: mcimadamore at openjdk.java.net (Maurizio Cimadamore)
Date: Wed, 3 Feb 2021 10:54:42 GMT
Subject: RFR: 8241356: Use a more reliable way to encode Symbol flags [v2]
In-Reply-To: <BeN0ieY-9mauOUxv6aUlC3VU7lvOy7TA4_n4soZwZ9Y=.806f69e0-635c-4f72-8f75-89c73668572d@github.com>
References: <aYRbIKEZmDlWfryC0sW1BSVtzJyIR7BRZS2Vs0XfGaY=.de31b4a7-d2df-489a-9816-08deab0cddef@github.com>
 <BeN0ieY-9mauOUxv6aUlC3VU7lvOy7TA4_n4soZwZ9Y=.806f69e0-635c-4f72-8f75-89c73668572d@github.com>
Message-ID: <UrON-JG2MMegyFMEN0RnsQXQiBqeWn_xmyioBQCMOaQ=.11c34cc3-e675-437c-b7bc-757fd58563ca@github.com>

On Tue, 2 Feb 2021 16:28:01 GMT, Jan Lahoda <jlahoda at openjdk.org> wrote:

>> [This is a GitHub copy of: https://mail.openjdk.java.net/pipermail/compiler-dev/2020-March/014389.html ]
>> 
>> Currently, (com.sun.tools.javac.code.)Symbol/s have a long field "flags_field", which holds various one-bit information ("Flags") about the given Symbol.
>> 
>> We currently have around 64 such Flags, which means we are out of bits in the long field, and adding new flags is not easy.
>> 
>> We could change the "flags_field" to be a Set over enums, but this would increase the memory footprint notably, and would also slow-down access to flags. Currently, flags operations in javac are very fast and very common, so this is probably too much burden. There are also flags to which we need to access as bit masks, e.g. due to writing to classfile.
>> 
>> My proposal here is to use an intermediate solution, until we find a better solution, or until a better solution is possible (like due to Valhalla). The idea is as follows:
>> -the current long-based Flags are split into 4 groups:
>> --"flat" Flags, long based, work exactly as before.
>> --enum-based TypeSymbolFlags, MethodSymbolFlags and VarSymbolFlags, which are only applicable to TypeSymbols, MethodSymbols and VarSymbols, respectively, and are checked using methods like Symbol.isFlagSet and set/clear using methods Symbol.setFlag and Symbol.clearFlag. So these flags are mostly encapsulated, even though physically they are currently stored in the flags_field as well.
>> 
>> There are ~~37~~ 40 "flat" flags and ~~16~~ 17 TypeSymbolFlags (methods and vars have less flags), 57 in total. This gives us at least 7 new flags before we run out of long bits in flags_field again - but even if we do, there are several easy mitigation strategies we could use, like:
>> -create a new int/long field on TypeSymbols for the TypeSymbolFlags (probably preferable)
>> -split TypeSymbolFlags into Class/Package/MethodSymbolFlags
>> 
>> The positives of this solution include:
>> -safe(r) access to the flags - the access to the extra/symbol-kind-specific flags is mostly encapsulated, and hence mostly safe
>> -improves the abstractions at least for some flags
>> -reasonably complex patch (attempts to encapsulate all the flags were troublesome in previous experiments)
>> -the performances appears to be acceptable.
>> 
>> The negative that I see is that the code includes several incompatible types of flags now. These cannot be easily confused by accident (as the type system will complain), but still we need to be aware of them when writing code.
>> 
>> Some more alternatives:
>> -add a new field to Symbol, like long extraFlags, and continue with bit masks as we did so far using this new field. The drawback is that it is more error prone (it would be difficult to ensure only the new/extra flags would be stored and read from extraFlags).
>> -have a new field, and "flat" and non-"flat" enum-based encapsulated Flags, but don't separate Type/Method/Var flags. Probably something to consider, even though I am a bit reluctant to add new fields without trying to avoid it .
>> 
>> Any feedback is welcome!
>
> Jan Lahoda has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Removing merge tags are noted on the review - thanks!

Good experiment! I don't mind too much the fact that we have sets of incompatible flags - but while looking at the new Flags.java I noted several flags that looked *kind-specific* which seem to be defined in the *global* bucket. If (as I suspect) this is no accident, then I guess I'm a bit worried that the proposed patch would land us in a state where _some_ var flags are in the VarSymbolFlags enum, but not _all_ of them.

src/jdk.compiler/share/classes/com/sun/tools/javac/code/Flags.java line 167:

> 165:     /** Flag for synthesized default constructors of anonymous classes.
> 166:      */
> 167:     public static final int ANONCONSTR   = 1<<27;

Question: why this, and other flags that are specific to only one symbol kind have not been translated into the enum? Other examples are `ANONCONSTR_BASED`, `GENERATEDCONSTR`, `COMPACT_RECORD_CONSTRUCTOR`, `THROWS` and probably more.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2316

From mcimadamore at openjdk.java.net  Wed Feb  3 11:02:43 2021
From: mcimadamore at openjdk.java.net (Maurizio Cimadamore)
Date: Wed, 3 Feb 2021 11:02:43 GMT
Subject: RFR: 8241356: Use a more reliable way to encode Symbol flags [v2]
In-Reply-To: <UrON-JG2MMegyFMEN0RnsQXQiBqeWn_xmyioBQCMOaQ=.11c34cc3-e675-437c-b7bc-757fd58563ca@github.com>
References: <aYRbIKEZmDlWfryC0sW1BSVtzJyIR7BRZS2Vs0XfGaY=.de31b4a7-d2df-489a-9816-08deab0cddef@github.com>
 <BeN0ieY-9mauOUxv6aUlC3VU7lvOy7TA4_n4soZwZ9Y=.806f69e0-635c-4f72-8f75-89c73668572d@github.com>
 <UrON-JG2MMegyFMEN0RnsQXQiBqeWn_xmyioBQCMOaQ=.11c34cc3-e675-437c-b7bc-757fd58563ca@github.com>
Message-ID: <2hjUP_18of5ES1130eAY94Chyy0fMqsQQwgXCpPWQuQ=.a0952a36-c55d-486c-b1dd-01dd657b9ef0@github.com>

On Wed, 3 Feb 2021 10:51:28 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Jan Lahoda has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Removing merge tags are noted on the review - thanks!
>
> Good experiment! I don't mind too much the fact that we have sets of incompatible flags - but while looking at the new Flags.java I noted several flags that looked *kind-specific* which seem to be defined in the *global* bucket. If (as I suspect) this is no accident, then I guess I'm a bit worried that the proposed patch would land us in a state where _some_ var flags are in the VarSymbolFlags enum, but not _all_ of them.

I'm also worried about where does this leave symbol construction - for instance, in TypeEnter I now see these lines (unchanged in this patch):

params.add(new VarSymbol(
                        GENERATED_MEMBER | PARAMETER | RECORD | (field == lastField && lastIsVarargs ? Flags.VARARGS : 0),
                        field.name, field.sym.type, csym));
 ```

Here, it seems, we are forced to just use a flat flags mask in the constructor - in other words, the new API is only for testing, and there is an asymmetry between construction and testing.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2316

From jlahoda at openjdk.java.net  Wed Feb  3 13:31:44 2021
From: jlahoda at openjdk.java.net (Jan Lahoda)
Date: Wed, 3 Feb 2021 13:31:44 GMT
Subject: RFR: 8241356: Use a more reliable way to encode Symbol flags [v2]
In-Reply-To: <UrON-JG2MMegyFMEN0RnsQXQiBqeWn_xmyioBQCMOaQ=.11c34cc3-e675-437c-b7bc-757fd58563ca@github.com>
References: <aYRbIKEZmDlWfryC0sW1BSVtzJyIR7BRZS2Vs0XfGaY=.de31b4a7-d2df-489a-9816-08deab0cddef@github.com>
 <BeN0ieY-9mauOUxv6aUlC3VU7lvOy7TA4_n4soZwZ9Y=.806f69e0-635c-4f72-8f75-89c73668572d@github.com>
 <UrON-JG2MMegyFMEN0RnsQXQiBqeWn_xmyioBQCMOaQ=.11c34cc3-e675-437c-b7bc-757fd58563ca@github.com>
Message-ID: <GPiDNAwNKvMY00tj5H7cN_7Qfx2h5m_JWVn3D7JmroM=.ea9f3d64-7a6b-423d-b47a-a141b390758d@github.com>

On Wed, 3 Feb 2021 10:43:52 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Jan Lahoda has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Removing merge tags are noted on the review - thanks!
>
> src/jdk.compiler/share/classes/com/sun/tools/javac/code/Flags.java line 167:
> 
>> 165:     /** Flag for synthesized default constructors of anonymous classes.
>> 166:      */
>> 167:     public static final int ANONCONSTR   = 1<<27;
> 
> Question: why this, and other flags that are specific to only one symbol kind have not been translated into the enum? Other examples are `ANONCONSTR_BASED`, `GENERATEDCONSTR`, `COMPACT_RECORD_CONSTRUCTOR`, `THROWS` and probably more.

Some of these (IIRC e.g. GENERATEDCONSTR, ANNONCONSTR) are also set on trees, and read and written as bitmasks, so it is more difficult to separate them to e.g. methods only. A few others could possibly be move with a little more code shuffling.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2316

From jlahoda at openjdk.java.net  Wed Feb  3 13:31:41 2021
From: jlahoda at openjdk.java.net (Jan Lahoda)
Date: Wed, 3 Feb 2021 13:31:41 GMT
Subject: RFR: 8241356: Use a more reliable way to encode Symbol flags [v2]
In-Reply-To: <2hjUP_18of5ES1130eAY94Chyy0fMqsQQwgXCpPWQuQ=.a0952a36-c55d-486c-b1dd-01dd657b9ef0@github.com>
References: <aYRbIKEZmDlWfryC0sW1BSVtzJyIR7BRZS2Vs0XfGaY=.de31b4a7-d2df-489a-9816-08deab0cddef@github.com>
 <BeN0ieY-9mauOUxv6aUlC3VU7lvOy7TA4_n4soZwZ9Y=.806f69e0-635c-4f72-8f75-89c73668572d@github.com>
 <UrON-JG2MMegyFMEN0RnsQXQiBqeWn_xmyioBQCMOaQ=.11c34cc3-e675-437c-b7bc-757fd58563ca@github.com>
 <2hjUP_18of5ES1130eAY94Chyy0fMqsQQwgXCpPWQuQ=.a0952a36-c55d-486c-b1dd-01dd657b9ef0@github.com>
Message-ID: <8FjBush8jmHv6T-EkbJbVt4yX8m0c4pUhUOBRJbtfAs=.4ddd72c5-4248-4119-b273-87ee31a1ca32@github.com>

On Wed, 3 Feb 2021 10:59:31 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> I'm also worried about where does this leave symbol construction - for instance, in TypeEnter I now see these lines (unchanged in this patch):
> 
> ```
> params.add(new VarSymbol(
>                         GENERATED_MEMBER | PARAMETER | RECORD | (field == lastField && lastIsVarargs ? Flags.VARARGS : 0),
>                         field.name, field.sym.type, csym));
> ```
> 
> Here, it seems, we are forced to just use a flat flags mask in the constructor - in other words, the new API is only for testing, and there is an asymmetry between construction and testing.

The flat flags work exactly as before (i.e. set as bits and checked as bit masks), and that are these bits set here. Only the symbol-kind specific flags are set and checked using methods, see e.g. the constructor for BindingSymbol.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2316

From mcimadamore at openjdk.java.net  Wed Feb  3 18:07:45 2021
From: mcimadamore at openjdk.java.net (Maurizio Cimadamore)
Date: Wed, 3 Feb 2021 18:07:45 GMT
Subject: RFR: 8241356: Use a more reliable way to encode Symbol flags [v2]
In-Reply-To: <GPiDNAwNKvMY00tj5H7cN_7Qfx2h5m_JWVn3D7JmroM=.ea9f3d64-7a6b-423d-b47a-a141b390758d@github.com>
References: <aYRbIKEZmDlWfryC0sW1BSVtzJyIR7BRZS2Vs0XfGaY=.de31b4a7-d2df-489a-9816-08deab0cddef@github.com>
 <BeN0ieY-9mauOUxv6aUlC3VU7lvOy7TA4_n4soZwZ9Y=.806f69e0-635c-4f72-8f75-89c73668572d@github.com>
 <UrON-JG2MMegyFMEN0RnsQXQiBqeWn_xmyioBQCMOaQ=.11c34cc3-e675-437c-b7bc-757fd58563ca@github.com>
 <GPiDNAwNKvMY00tj5H7cN_7Qfx2h5m_JWVn3D7JmroM=.ea9f3d64-7a6b-423d-b47a-a141b390758d@github.com>
Message-ID: <neVWMpNL-T8_rEA4gyN2U9dJdUYMz9L_20T-icWNvMY=.3e743231-3e7e-4f08-8aab-9d863f8dfb66@github.com>

On Wed, 3 Feb 2021 13:26:25 GMT, Jan Lahoda <jlahoda at openjdk.org> wrote:

>> src/jdk.compiler/share/classes/com/sun/tools/javac/code/Flags.java line 167:
>> 
>>> 165:     /** Flag for synthesized default constructors of anonymous classes.
>>> 166:      */
>>> 167:     public static final int ANONCONSTR   = 1<<27;
>> 
>> Question: why this, and other flags that are specific to only one symbol kind have not been translated into the enum? Other examples are `ANONCONSTR_BASED`, `GENERATEDCONSTR`, `COMPACT_RECORD_CONSTRUCTOR`, `THROWS` and probably more.
>
> Some of these (IIRC e.g. GENERATEDCONSTR, ANNONCONSTR) are also set on trees, and read and written as bitmasks, so it is more difficult to separate them to e.g. methods only. A few others could possibly be move with a little more code shuffling.

I see what you mean - as I said, my worry is that, while this patch addresses the paucity of flags in an effective fashion, the resulting code base end up being worse than it was to begin with, as now a client will have to hunt flags in different enums. If the organization was consistent, I'd have no problem with giving each kind of flag its own enum - but given that the organization is more ad-hoc (e.g. driven by which use cases could be morphed into using enums), I'm unsure.

I mean, what's the advantage of the proposed solution vs. organizing the set of long flags in Flags by categories - so that there's no enum, but so that e.g. a variable symbol flag can reuse the same value as some other method symbol flag?

I think it would deliver same saving, w/o the need to change the flag API. Granted, we also not get any protection for when somebody tries to "merge" flags that are not mergeable - but I'm equally not sure as to whether the proposed solution offers a much more principled alternative?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2316

From jlahoda at openjdk.java.net  Wed Feb  3 18:41:40 2021
From: jlahoda at openjdk.java.net (Jan Lahoda)
Date: Wed, 3 Feb 2021 18:41:40 GMT
Subject: RFR: 8241356: Use a more reliable way to encode Symbol flags [v2]
In-Reply-To: <neVWMpNL-T8_rEA4gyN2U9dJdUYMz9L_20T-icWNvMY=.3e743231-3e7e-4f08-8aab-9d863f8dfb66@github.com>
References: <aYRbIKEZmDlWfryC0sW1BSVtzJyIR7BRZS2Vs0XfGaY=.de31b4a7-d2df-489a-9816-08deab0cddef@github.com>
 <BeN0ieY-9mauOUxv6aUlC3VU7lvOy7TA4_n4soZwZ9Y=.806f69e0-635c-4f72-8f75-89c73668572d@github.com>
 <UrON-JG2MMegyFMEN0RnsQXQiBqeWn_xmyioBQCMOaQ=.11c34cc3-e675-437c-b7bc-757fd58563ca@github.com>
 <GPiDNAwNKvMY00tj5H7cN_7Qfx2h5m_JWVn3D7JmroM=.ea9f3d64-7a6b-423d-b47a-a141b390758d@github.com>
 <neVWMpNL-T8_rEA4gyN2U9dJdUYMz9L_20T-icWNvMY=.3e743231-3e7e-4f08-8aab-9d863f8dfb66@github.com>
Message-ID: <gtf1bw-8XKvok8PA18DWx8wMmvDp_5nIHvIckavjt6s=.4cbd9e02-c188-43d2-a928-9d9d4d617513@github.com>

On Wed, 3 Feb 2021 18:05:12 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> I see what you mean - as I said, my worry is that, while this patch addresses the paucity of flags in an effective fashion, the resulting code base end up being worse than it was to begin with, as now a client will have to hunt flags in different enums. If the organization was consistent, I'd have no problem with giving each kind of flag its own enum - but given that the organization is more ad-hoc (e.g. driven by which use cases could be morphed into using enums), I'm unsure.
> 
> I mean, what's the advantage of the proposed solution vs. organizing the set of long flags in Flags by categories - so that there's no enum, but so that e.g. a variable symbol flag can reuse the same value as some other method symbol flag?
> 
> I think it would deliver same saving, w/o the need to change the flag API. Granted, we also not get any protection for when somebody tries to "merge" flags that are not mergeable - but I'm equally not sure as to whether the proposed solution offers a much more principled alternative?

I don't think this is a more principled alternative, the main goal here was to reuse the flag bits as safely as possible. Reusing the same bits across various Symbols manually is basically what we do now, but, for me, it is often difficult to prove that a bit is used only for (e.g.) VarSymbols and I can reuse it for MethodSymbols. With this patch, most of the (symbol-specific) checks are done statically, some are done at runtime (via casts), but we can't easily set a method-only flags on a VariableSymbol by accident.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2316

From jwilhelm at openjdk.java.net  Thu Feb  4 01:24:02 2021
From: jwilhelm at openjdk.java.net (Jesper Wilhelmsson)
Date: Thu, 4 Feb 2021 01:24:02 GMT
Subject: RFR: Merge jdk16
Message-ID: <L2_q8bEWe8cQRclrCLPoHGIKlEGdSGlG4MwTiMo9Oe4=.b41c5dbc-ca26-4e13-afdd-acaf1e8a4089@github.com>

Forwardport JDK 16 -> JDK 17

-------------

Commit messages:
 - Merge
 - 8259794: Remove EA from JDK 16 version string starting with Initial RC promotion on Feb 04, 2021(B35)
 - 8260704: ParallelGC: oldgen expansion needs release-store for _end
 - 8260927: StringBuilder::insert is incorrect without Compact Strings
 - 8258378: Final nroff manpage update for JDK 16
 - 8257215: JFR: Events dropped when streaming over a chunk rotation
 - 8260473: [vector] ZGC: VectorReshape test produces incorrect results with ZGC enabled
 - 8260632: Build failures after JDK-8253353
 - 8260339: JVM crashes when executing PhaseIdealLoop::match_fill_loop
 - 8260608: add a regression test for 8260370
 - ... and 2 more: https://git.openjdk.java.net/jdk/compare/f025bc1d...dad835ee

The webrevs contain the adjustments done while merging with regards to each parent branch:
 - master: https://webrevs.openjdk.java.net/?repo=jdk&pr=2392&range=00.0
 - jdk16: https://webrevs.openjdk.java.net/?repo=jdk&pr=2392&range=00.1

Changes: https://git.openjdk.java.net/jdk/pull/2392/files
  Stats: 2645 lines in 56 files changed: 2497 ins; 69 del; 79 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2392.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2392/head:pull/2392

PR: https://git.openjdk.java.net/jdk/pull/2392

From jwilhelm at openjdk.java.net  Thu Feb  4 02:09:40 2021
From: jwilhelm at openjdk.java.net (Jesper Wilhelmsson)
Date: Thu, 4 Feb 2021 02:09:40 GMT
Subject: Integrated: Merge jdk16
In-Reply-To: <L2_q8bEWe8cQRclrCLPoHGIKlEGdSGlG4MwTiMo9Oe4=.b41c5dbc-ca26-4e13-afdd-acaf1e8a4089@github.com>
References: <L2_q8bEWe8cQRclrCLPoHGIKlEGdSGlG4MwTiMo9Oe4=.b41c5dbc-ca26-4e13-afdd-acaf1e8a4089@github.com>
Message-ID: <9bIQlW7tZAt5i8quzTGD6Z6vHAG4-Q8-_saIecOJ4dM=.b1dacc8a-2678-417c-958b-650ff659723f@github.com>

On Thu, 4 Feb 2021 01:17:48 GMT, Jesper Wilhelmsson <jwilhelm at openjdk.org> wrote:

> Forwardport JDK 16 -> JDK 17

This pull request has now been integrated.

Changeset: 9b7a8f19
Author:    Jesper Wilhelmsson <jwilhelm at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/9b7a8f19
Stats:     2645 lines in 56 files changed: 2497 ins; 69 del; 79 mod

Merge

-------------

PR: https://git.openjdk.java.net/jdk/pull/2392

From ihse at openjdk.java.net  Thu Feb  4 10:17:59 2021
From: ihse at openjdk.java.net (Magnus Ihse Bursie)
Date: Thu, 4 Feb 2021 10:17:59 GMT
Subject: RFR: 8261149: Initial nroff manpage update for JDK 17
Message-ID: <HvPsFX2azuvcaU5K-lTwixZ8Q8fYkTkJakELuAFVnVc=.56d0445a-74e3-48c6-824b-043a008610aa@github.com>

We need to regenerate the exported nroff man pages to include the proper version string. This will also bring in some recent textual updates to the man pages.

-------------

Commit messages:
 - 8261149: Initial nroff manpage update for JDK 17

Changes: https://git.openjdk.java.net/jdk/pull/2402/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2402&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8261149
  Stats: 49 lines in 28 files changed: 0 ins; 21 del; 28 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2402.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2402/head:pull/2402

PR: https://git.openjdk.java.net/jdk/pull/2402

From jlahoda at openjdk.java.net  Wed Feb 10 14:00:52 2021
From: jlahoda at openjdk.java.net (Jan Lahoda)
Date: Wed, 10 Feb 2021 14:00:52 GMT
Subject: RFR: 8247403: JShell: No custom input (e.g. from GUI) possible with
 JavaShellToolBuilder
Message-ID: <LG3Ko-Xn6aySWkkgG2YXLUtVLhpiFcJ7prYtQm-Fzns=.b2c17f87-faef-46e6-ac0d-6f08187dde69@github.com>

Basically, JShell's ConsoleIOContext always uses the JLine's default terminal, which uses System.in, or equivalent, for input. But, if the JShell tool has been created using JavaShellToolBuilder, a different input might have been provided. The proposed solution is to use a different JLine's terminal in this case, based on the provided input.

-------------

Commit messages:
 - 8247403: JShell: No custom input (e.g. from GUI) possible with JavaShellToolBuilder

Changes: https://git.openjdk.java.net/jdk/pull/2509/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2509&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8247403
  Stats: 81 lines in 2 files changed: 74 ins; 0 del; 7 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2509.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2509/head:pull/2509

PR: https://git.openjdk.java.net/jdk/pull/2509

From jjg at openjdk.java.net  Fri Feb 12 00:16:42 2021
From: jjg at openjdk.java.net (Jonathan Gibbons)
Date: Fri, 12 Feb 2021 00:16:42 GMT
Subject: RFR: 8241356: Use a more reliable way to encode Symbol flags [v2]
In-Reply-To: <BeN0ieY-9mauOUxv6aUlC3VU7lvOy7TA4_n4soZwZ9Y=.806f69e0-635c-4f72-8f75-89c73668572d@github.com>
References: <aYRbIKEZmDlWfryC0sW1BSVtzJyIR7BRZS2Vs0XfGaY=.de31b4a7-d2df-489a-9816-08deab0cddef@github.com>
 <BeN0ieY-9mauOUxv6aUlC3VU7lvOy7TA4_n4soZwZ9Y=.806f69e0-635c-4f72-8f75-89c73668572d@github.com>
Message-ID: <tJe6uIJyv9P7NfMGwupM69VXjzsQSOcvuo5n2tppX94=.7ef20990-bc5d-4d18-b0a4-9c13de8538af@github.com>

On Tue, 2 Feb 2021 16:28:01 GMT, Jan Lahoda <jlahoda at openjdk.org> wrote:

>> [This is a GitHub copy of: https://mail.openjdk.java.net/pipermail/compiler-dev/2020-March/014389.html ]
>> 
>> Currently, (com.sun.tools.javac.code.)Symbol/s have a long field "flags_field", which holds various one-bit information ("Flags") about the given Symbol.
>> 
>> We currently have around 64 such Flags, which means we are out of bits in the long field, and adding new flags is not easy.
>> 
>> We could change the "flags_field" to be a Set over enums, but this would increase the memory footprint notably, and would also slow-down access to flags. Currently, flags operations in javac are very fast and very common, so this is probably too much burden. There are also flags to which we need to access as bit masks, e.g. due to writing to classfile.
>> 
>> My proposal here is to use an intermediate solution, until we find a better solution, or until a better solution is possible (like due to Valhalla). The idea is as follows:
>> -the current long-based Flags are split into 4 groups:
>> --"flat" Flags, long based, work exactly as before.
>> --enum-based TypeSymbolFlags, MethodSymbolFlags and VarSymbolFlags, which are only applicable to TypeSymbols, MethodSymbols and VarSymbols, respectively, and are checked using methods like Symbol.isFlagSet and set/clear using methods Symbol.setFlag and Symbol.clearFlag. So these flags are mostly encapsulated, even though physically they are currently stored in the flags_field as well.
>> 
>> There are ~~37~~ 40 "flat" flags and ~~16~~ 17 TypeSymbolFlags (methods and vars have less flags), 57 in total. This gives us at least 7 new flags before we run out of long bits in flags_field again - but even if we do, there are several easy mitigation strategies we could use, like:
>> -create a new int/long field on TypeSymbols for the TypeSymbolFlags (probably preferable)
>> -split TypeSymbolFlags into Class/Package/MethodSymbolFlags
>> 
>> The positives of this solution include:
>> -safe(r) access to the flags - the access to the extra/symbol-kind-specific flags is mostly encapsulated, and hence mostly safe
>> -improves the abstractions at least for some flags
>> -reasonably complex patch (attempts to encapsulate all the flags were troublesome in previous experiments)
>> -the performances appears to be acceptable.
>> 
>> The negative that I see is that the code includes several incompatible types of flags now. These cannot be easily confused by accident (as the type system will complain), but still we need to be aware of them when writing code.
>> 
>> Some more alternatives:
>> -add a new field to Symbol, like long extraFlags, and continue with bit masks as we did so far using this new field. The drawback is that it is more error prone (it would be difficult to ensure only the new/extra flags would be stored and read from extraFlags).
>> -have a new field, and "flat" and non-"flat" enum-based encapsulated Flags, but don't separate Type/Method/Var flags. Probably something to consider, even though I am a bit reluctant to add new fields without trying to avoid it .
>> 
>> Any feedback is welcome!
>
> Jan Lahoda has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Removing merge tags are noted on the review - thanks!

I've reviewed the one javadoc change, and made a comment there. Although I approve the change to that one file, I will leave the review/approval of the javac changes to the javac experts.

src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/util/Extern.java line 594:

> 592:         } else {
> 593:             ModuleSymbol msym = (ModuleSymbol) me;
> 594:             return msym.isFlagSet(TypeSymbolFlags.AUTOMATIC_MODULE);

The change is OK, but the code being changed is not. There should not be references to any `javac.code.*` types outside of the `WorkArounds` class.  Ideally, this function (`isAutomaticModule`) should be on the `Elements` class.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2316

From jjg at openjdk.java.net  Fri Feb 12 00:52:41 2021
From: jjg at openjdk.java.net (Jonathan Gibbons)
Date: Fri, 12 Feb 2021 00:52:41 GMT
Subject: RFR: 8241356: Use a more reliable way to encode Symbol flags [v2]
In-Reply-To: <tJe6uIJyv9P7NfMGwupM69VXjzsQSOcvuo5n2tppX94=.7ef20990-bc5d-4d18-b0a4-9c13de8538af@github.com>
References: <aYRbIKEZmDlWfryC0sW1BSVtzJyIR7BRZS2Vs0XfGaY=.de31b4a7-d2df-489a-9816-08deab0cddef@github.com>
 <BeN0ieY-9mauOUxv6aUlC3VU7lvOy7TA4_n4soZwZ9Y=.806f69e0-635c-4f72-8f75-89c73668572d@github.com>
 <tJe6uIJyv9P7NfMGwupM69VXjzsQSOcvuo5n2tppX94=.7ef20990-bc5d-4d18-b0a4-9c13de8538af@github.com>
Message-ID: <YRKJrpVwLauQfuH67QuiSOKFU6IVdmAZN6P-h_d3FQo=.51e07234-4c3d-45c7-978c-27c98ce3e509@github.com>

On Fri, 12 Feb 2021 00:12:07 GMT, Jonathan Gibbons <jjg at openjdk.org> wrote:

>> Jan Lahoda has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Removing merge tags are noted on the review - thanks!
>
> src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/util/Extern.java line 594:
> 
>> 592:         } else {
>> 593:             ModuleSymbol msym = (ModuleSymbol) me;
>> 594:             return msym.isFlagSet(TypeSymbolFlags.AUTOMATIC_MODULE);
> 
> The change is OK, but the code being changed is not. There should not be references to any `javac.code.*` types outside of the `WorkArounds` class.  Ideally, this function (`isAutomaticModule`) should be on the `Elements` class.

FYI, PR 2537 will move this code to the `Workarounds` class.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2316

From itakiguchi at openjdk.java.net  Thu Feb 18 04:05:53 2021
From: itakiguchi at openjdk.java.net (Ichiroh Takiguchi)
Date: Thu, 18 Feb 2021 04:05:53 GMT
Subject: RFR: 8261920: [AIX] jshell command throws java.io.IOError on non
 English locales
Message-ID: <ZSx49ohjP_YqU76gSw_kc3l-OmgYDlTPTmlMDSw1r5M=.3b552fdd-ba60-43e2-8e53-5ef4dc506fcb@github.com>

jshell uses JLine library.
JLine checks word "columns" in "stty -a" command output to find out terminal size.
The word "columns" was translated on AIX's Japanese locale (other locales also), so above error was happened.

OpenJDK for AIX developer could not find this issue because he/she requires AIX Toolbox's coreutils rpm package as build tools.
Another stty command is in /opt/freeware/bin/stty, it's in coreutils rpm package.

Standard users' system may not have coreutils rpm package or they may not set /opt/freeware/bin in PATH environment variable.
stty command should be executed with C locale to get English message.
Additionally, JLine library requires -F option. AIX's stty command does not support -F option,
but stty command which is in coreutils rpm package supports -F option.
On standard jshell usage, -F option may not be used. But when the system has coreutils rpm package, jshell should use this one.

-------------

Commit messages:
 - AIX stty command should be executed on C locale to get English message

Changes: https://git.openjdk.java.net/jdk/pull/2622/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2622&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8261920
  Stats: 8 lines in 2 files changed: 8 ins; 0 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2622.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2622/head:pull/2622

PR: https://git.openjdk.java.net/jdk/pull/2622

From stuefe at openjdk.java.net  Mon Feb 22 10:55:46 2021
From: stuefe at openjdk.java.net (Thomas Stuefe)
Date: Mon, 22 Feb 2021 10:55:46 GMT
Subject: RFR: 8261920: [AIX] jshell command throws java.io.IOError on non
 English locales
In-Reply-To: <ZSx49ohjP_YqU76gSw_kc3l-OmgYDlTPTmlMDSw1r5M=.3b552fdd-ba60-43e2-8e53-5ef4dc506fcb@github.com>
References: <ZSx49ohjP_YqU76gSw_kc3l-OmgYDlTPTmlMDSw1r5M=.3b552fdd-ba60-43e2-8e53-5ef4dc506fcb@github.com>
Message-ID: <i-O4SOeeYRl50VOmiboEiE2RdfFgAZXGvio3Nn7qmSE=.ed4a5502-1628-4d6b-b119-3b783a87a8f4@github.com>

On Thu, 18 Feb 2021 03:56:25 GMT, Ichiroh Takiguchi <itakiguchi at openjdk.org> wrote:

> jshell uses JLine library.
> JLine checks word "columns" in "stty -a" command output to find out terminal size.
> The word "columns" was translated on AIX's Japanese locale (other locales also), so above error was happened.
> 
> OpenJDK for AIX developer could not find this issue because he/she requires AIX Toolbox's coreutils rpm package as build tools.
> Another stty command is in /opt/freeware/bin/stty, it's in coreutils rpm package.
> 
> Standard users' system may not have coreutils rpm package or they may not set /opt/freeware/bin in PATH environment variable.
> stty command should be executed with C locale to get English message.
> Additionally, JLine library requires -F option. AIX's stty command does not support -F option,
> but stty command which is in coreutils rpm package supports -F option.
> On standard jshell usage, -F option may not be used. But when the system has coreutils rpm package, jshell should use this one.

This seems reasonable. Which tests did you run?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2622

From itakiguchi at openjdk.java.net  Mon Feb 22 11:08:40 2021
From: itakiguchi at openjdk.java.net (Ichiroh Takiguchi)
Date: Mon, 22 Feb 2021 11:08:40 GMT
Subject: RFR: 8261920: [AIX] jshell command throws java.io.IOError on non
 English locales
In-Reply-To: <i-O4SOeeYRl50VOmiboEiE2RdfFgAZXGvio3Nn7qmSE=.ed4a5502-1628-4d6b-b119-3b783a87a8f4@github.com>
References: <ZSx49ohjP_YqU76gSw_kc3l-OmgYDlTPTmlMDSw1r5M=.3b552fdd-ba60-43e2-8e53-5ef4dc506fcb@github.com>
 <i-O4SOeeYRl50VOmiboEiE2RdfFgAZXGvio3Nn7qmSE=.ed4a5502-1628-4d6b-b119-3b783a87a8f4@github.com>
Message-ID: <flulcSVMM-yLp8pQKajsyfuEkL1cY2S_ZmuoWo-5Gfk=.92ceae76-97f8-4c93-b282-df96ea8d2acf@github.com>

On Mon, 22 Feb 2021 10:52:28 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> jshell uses JLine library.
>> JLine checks word "columns" in "stty -a" command output to find out terminal size.
>> The word "columns" was translated on AIX's Japanese locale (other locales also), so above error was happened.
>> 
>> OpenJDK for AIX developer could not find this issue because he/she requires AIX Toolbox's coreutils rpm package as build tools.
>> Another stty command is in /opt/freeware/bin/stty, it's in coreutils rpm package.
>> 
>> Standard users' system may not have coreutils rpm package or they may not set /opt/freeware/bin in PATH environment variable.
>> stty command should be executed with C locale to get English message.
>> Additionally, JLine library requires -F option. AIX's stty command does not support -F option,
>> but stty command which is in coreutils rpm package supports -F option.
>> On standard jshell usage, -F option may not be used. But when the system has coreutils rpm package, jshell should use this one.
>
> This seems reasonable. Which tests did you run?

No testcase is available for this issue, because it's related Locale setting and translated message on AIX platform.
I just execute jshell command on X11's dtterm and aixterm with Ja_JP locale.
(\uXXXX is actually, Japanese character)
$ ./jshell
| JShell\u3078\u3088\u3046\u3053\u305d -- \u30d0\u30fc\u30b8\u30e7\u30f315.0.2
| \u6982\u8981\u306b\u3064\u3044\u3066\u306f\u3001\u6b21\u3092\u5165\u529b\u3057\u3066\u304f\u3060\u3055\u3044: /help intro
Exception in thread "main" java.io.IOError: java.io.IOException: Unable to parse columns
        at jdk.internal.le/jdk.internal.org.jline.terminal.impl.AbstractPosixTerminal.getSize(AbstractPosixTerminal.java:62)
        at jdk.internal.le/jdk.internal.org.jline.terminal.Terminal.getBufferSize(Terminal.java:216)
        at jdk.internal.le/jdk.internal.org.jline.reader.impl.LineReaderImpl.doDisplay(LineReaderImpl.java:741)
        at jdk.internal.le/jdk.internal.org.jline.reader.impl.LineReaderImpl.<init>(LineReaderImpl.java:298)
        at jdk.jshell/jdk.internal.jshell.tool.ConsoleIOContext$2.<init>(ConsoleIOContext.java:133)
        at jdk.jshell/jdk.internal.jshell.tool.ConsoleIOContext.<init>(ConsoleIOContext.java:133)
        at jdk.jshell/jdk.internal.jshell.tool.JShellTool.start(JShellTool.java:978)
        at jdk.jshell/jdk.internal.jshell.tool.JShellToolBuilder.start(JShellToolBuilder.java:254)
        at jdk.jshell/jdk.internal.jshell.tool.JShellToolProvider.main(JShellToolProvider.java:120)
Caused by: java.io.IOException: Unable to parse columns
        at jdk.internal.le/jdk.internal.org.jline.terminal.impl.ExecPty.doGetInt(ExecPty.java:278)
        at jdk.internal.le/jdk.internal.org.jline.terminal.impl.ExecPty.doGetSize(ExecPty.java:263)
        at jdk.internal.le/jdk.internal.org.jline.terminal.impl.ExecPty.getSize(ExecPty.java:170)
        at jdk.internal.le/jdk.internal.org.jline.terminal.impl.AbstractPosixTerminal.getSize(AbstractPosixTerminal.java:60)
        ... 8 more
$

-------------

PR: https://git.openjdk.java.net/jdk/pull/2622

From stuefe at openjdk.java.net  Wed Feb 24 06:21:40 2021
From: stuefe at openjdk.java.net (Thomas Stuefe)
Date: Wed, 24 Feb 2021 06:21:40 GMT
Subject: RFR: 8261920: [AIX] jshell command throws java.io.IOError on non
 English locales
In-Reply-To: <flulcSVMM-yLp8pQKajsyfuEkL1cY2S_ZmuoWo-5Gfk=.92ceae76-97f8-4c93-b282-df96ea8d2acf@github.com>
References: <ZSx49ohjP_YqU76gSw_kc3l-OmgYDlTPTmlMDSw1r5M=.3b552fdd-ba60-43e2-8e53-5ef4dc506fcb@github.com>
 <i-O4SOeeYRl50VOmiboEiE2RdfFgAZXGvio3Nn7qmSE=.ed4a5502-1628-4d6b-b119-3b783a87a8f4@github.com>
 <flulcSVMM-yLp8pQKajsyfuEkL1cY2S_ZmuoWo-5Gfk=.92ceae76-97f8-4c93-b282-df96ea8d2acf@github.com>
Message-ID: <B-dE84jT4lxRqhK2NZgEg7X_Gt_0Jm44EhaveVfIMkM=.e4f28b84-536b-4cb0-b400-04ca76ee12b2@github.com>

On Mon, 22 Feb 2021 11:06:10 GMT, Ichiroh Takiguchi <itakiguchi at openjdk.org> wrote:

>> This seems reasonable. Which tests did you run?
>
> No testcase is available for this issue, because it's related Locale setting and translated message on AIX platform.
> I just execute jshell command on X11's dtterm and aixterm with Ja_JP locale.
> (\uXXXX is actually, Japanese character)
> $ ./jshell
> | JShell\u3078\u3088\u3046\u3053\u305d -- \u30d0\u30fc\u30b8\u30e7\u30f315.0.2
> | \u6982\u8981\u306b\u3064\u3044\u3066\u306f\u3001\u6b21\u3092\u5165\u529b\u3057\u3066\u304f\u3060\u3055\u3044: /help intro
> Exception in thread "main" java.io.IOError: java.io.IOException: Unable to parse columns
>         at jdk.internal.le/jdk.internal.org.jline.terminal.impl.AbstractPosixTerminal.getSize(AbstractPosixTerminal.java:62)
>         at jdk.internal.le/jdk.internal.org.jline.terminal.Terminal.getBufferSize(Terminal.java:216)
>         at jdk.internal.le/jdk.internal.org.jline.reader.impl.LineReaderImpl.doDisplay(LineReaderImpl.java:741)
>         at jdk.internal.le/jdk.internal.org.jline.reader.impl.LineReaderImpl.<init>(LineReaderImpl.java:298)
>         at jdk.jshell/jdk.internal.jshell.tool.ConsoleIOContext$2.<init>(ConsoleIOContext.java:133)
>         at jdk.jshell/jdk.internal.jshell.tool.ConsoleIOContext.<init>(ConsoleIOContext.java:133)
>         at jdk.jshell/jdk.internal.jshell.tool.JShellTool.start(JShellTool.java:978)
>         at jdk.jshell/jdk.internal.jshell.tool.JShellToolBuilder.start(JShellToolBuilder.java:254)
>         at jdk.jshell/jdk.internal.jshell.tool.JShellToolProvider.main(JShellToolProvider.java:120)
> Caused by: java.io.IOException: Unable to parse columns
>         at jdk.internal.le/jdk.internal.org.jline.terminal.impl.ExecPty.doGetInt(ExecPty.java:278)
>         at jdk.internal.le/jdk.internal.org.jline.terminal.impl.ExecPty.doGetSize(ExecPty.java:263)
>         at jdk.internal.le/jdk.internal.org.jline.terminal.impl.ExecPty.getSize(ExecPty.java:170)
>         at jdk.internal.le/jdk.internal.org.jline.terminal.impl.AbstractPosixTerminal.getSize(AbstractPosixTerminal.java:60)
>         ... 8 more
> $

Im sorry, what I meant is how you make sure no regressions are happening. From what I can see you modify a general purpose function which executes Unix commands to always run with C Locale on AIX. Which unix commands are effected by this, and which JDK functionality? From looking at the source I find it difficult to see whether this does not introduce regressions, sorry. Did you run jtreg tests; if yes which tiers?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2622

From itakiguchi at openjdk.java.net  Wed Feb 24 12:54:41 2021
From: itakiguchi at openjdk.java.net (Ichiroh Takiguchi)
Date: Wed, 24 Feb 2021 12:54:41 GMT
Subject: RFR: 8261920: [AIX] jshell command throws java.io.IOError on non
 English locales
In-Reply-To: <B-dE84jT4lxRqhK2NZgEg7X_Gt_0Jm44EhaveVfIMkM=.e4f28b84-536b-4cb0-b400-04ca76ee12b2@github.com>
References: <ZSx49ohjP_YqU76gSw_kc3l-OmgYDlTPTmlMDSw1r5M=.3b552fdd-ba60-43e2-8e53-5ef4dc506fcb@github.com>
 <i-O4SOeeYRl50VOmiboEiE2RdfFgAZXGvio3Nn7qmSE=.ed4a5502-1628-4d6b-b119-3b783a87a8f4@github.com>
 <flulcSVMM-yLp8pQKajsyfuEkL1cY2S_ZmuoWo-5Gfk=.92ceae76-97f8-4c93-b282-df96ea8d2acf@github.com>
 <B-dE84jT4lxRqhK2NZgEg7X_Gt_0Jm44EhaveVfIMkM=.e4f28b84-536b-4cb0-b400-04ca76ee12b2@github.com>
Message-ID: <FBwbLjTvgyEcK8Ie6uOqh0rWHcZ_dynrnLtTjXkPhGw=.d4d9abf6-1b66-48cb-a8ee-e1e8e9c93ae2@github.com>

On Wed, 24 Feb 2021 06:18:50 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> No testcase is available for this issue, because it's related Locale setting and translated message on AIX platform.
>> I just execute jshell command on X11's dtterm and aixterm with Ja_JP locale.
>> (\uXXXX is actually, Japanese character)
>> $ ./jshell
>> | JShell\u3078\u3088\u3046\u3053\u305d -- \u30d0\u30fc\u30b8\u30e7\u30f315.0.2
>> | \u6982\u8981\u306b\u3064\u3044\u3066\u306f\u3001\u6b21\u3092\u5165\u529b\u3057\u3066\u304f\u3060\u3055\u3044: /help intro
>> Exception in thread "main" java.io.IOError: java.io.IOException: Unable to parse columns
>>         at jdk.internal.le/jdk.internal.org.jline.terminal.impl.AbstractPosixTerminal.getSize(AbstractPosixTerminal.java:62)
>>         at jdk.internal.le/jdk.internal.org.jline.terminal.Terminal.getBufferSize(Terminal.java:216)
>>         at jdk.internal.le/jdk.internal.org.jline.reader.impl.LineReaderImpl.doDisplay(LineReaderImpl.java:741)
>>         at jdk.internal.le/jdk.internal.org.jline.reader.impl.LineReaderImpl.<init>(LineReaderImpl.java:298)
>>         at jdk.jshell/jdk.internal.jshell.tool.ConsoleIOContext$2.<init>(ConsoleIOContext.java:133)
>>         at jdk.jshell/jdk.internal.jshell.tool.ConsoleIOContext.<init>(ConsoleIOContext.java:133)
>>         at jdk.jshell/jdk.internal.jshell.tool.JShellTool.start(JShellTool.java:978)
>>         at jdk.jshell/jdk.internal.jshell.tool.JShellToolBuilder.start(JShellToolBuilder.java:254)
>>         at jdk.jshell/jdk.internal.jshell.tool.JShellToolProvider.main(JShellToolProvider.java:120)
>> Caused by: java.io.IOException: Unable to parse columns
>>         at jdk.internal.le/jdk.internal.org.jline.terminal.impl.ExecPty.doGetInt(ExecPty.java:278)
>>         at jdk.internal.le/jdk.internal.org.jline.terminal.impl.ExecPty.doGetSize(ExecPty.java:263)
>>         at jdk.internal.le/jdk.internal.org.jline.terminal.impl.ExecPty.getSize(ExecPty.java:170)
>>         at jdk.internal.le/jdk.internal.org.jline.terminal.impl.AbstractPosixTerminal.getSize(AbstractPosixTerminal.java:60)
>>         ... 8 more
>> $
>
> Im sorry, what I meant is how you make sure no regressions are happening. From what I can see you modify a general purpose function which executes Unix commands to always run with C Locale on AIX. Which unix commands are effected by this, and which JDK functionality? From looking at the source I find it difficult to see whether this does not introduce regressions, sorry. Did you run jtreg tests; if yes which tiers?

I'm sorry I'm confused.
I rerun tier1 and tier2 on AIX platform.
No regression was there.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2622

From stuefe at openjdk.java.net  Wed Feb 24 13:08:40 2021
From: stuefe at openjdk.java.net (Thomas Stuefe)
Date: Wed, 24 Feb 2021 13:08:40 GMT
Subject: RFR: 8261920: [AIX] jshell command throws java.io.IOError on non
 English locales
In-Reply-To: <ZSx49ohjP_YqU76gSw_kc3l-OmgYDlTPTmlMDSw1r5M=.3b552fdd-ba60-43e2-8e53-5ef4dc506fcb@github.com>
References: <ZSx49ohjP_YqU76gSw_kc3l-OmgYDlTPTmlMDSw1r5M=.3b552fdd-ba60-43e2-8e53-5ef4dc506fcb@github.com>
Message-ID: <ZG-Ip7hVK3hKTB74j9jRO-8HYCRbAOHPBwKu74eeTog=.bb4d0874-e274-44d9-894c-9fb2633fadaa@github.com>

On Thu, 18 Feb 2021 03:56:25 GMT, Ichiroh Takiguchi <itakiguchi at openjdk.org> wrote:

> jshell uses JLine library.
> JLine checks word "columns" in "stty -a" command output to find out terminal size.
> The word "columns" was translated on AIX's Japanese locale (other locales also), so above error was happened.
> 
> OpenJDK for AIX developer could not find this issue because he/she requires AIX Toolbox's coreutils rpm package as build tools.
> Another stty command is in /opt/freeware/bin/stty, it's in coreutils rpm package.
> 
> Standard users' system may not have coreutils rpm package or they may not set /opt/freeware/bin in PATH environment variable.
> stty command should be executed with C locale to get English message.
> Additionally, JLine library requires -F option. AIX's stty command does not support -F option,
> but stty command which is in coreutils rpm package supports -F option.
> On standard jshell usage, -F option may not be used. But when the system has coreutils rpm package, jshell should use this one.

Thanks! I'm fine with this change then.

-------------

Marked as reviewed by stuefe (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2622

From itakiguchi at openjdk.java.net  Wed Feb 24 13:16:41 2021
From: itakiguchi at openjdk.java.net (Ichiroh Takiguchi)
Date: Wed, 24 Feb 2021 13:16:41 GMT
Subject: RFR: 8261920: [AIX] jshell command throws java.io.IOError on non
 English locales
In-Reply-To: <ZG-Ip7hVK3hKTB74j9jRO-8HYCRbAOHPBwKu74eeTog=.bb4d0874-e274-44d9-894c-9fb2633fadaa@github.com>
References: <ZSx49ohjP_YqU76gSw_kc3l-OmgYDlTPTmlMDSw1r5M=.3b552fdd-ba60-43e2-8e53-5ef4dc506fcb@github.com>
 <ZG-Ip7hVK3hKTB74j9jRO-8HYCRbAOHPBwKu74eeTog=.bb4d0874-e274-44d9-894c-9fb2633fadaa@github.com>
Message-ID: <vs4sVtnlTalGpaKlqoJ0dHhZalQuib_uPLA6LPhQXz0=.500af044-27d1-40fb-bad7-25111acdd3bd@github.com>

On Wed, 24 Feb 2021 13:06:10 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> jshell uses JLine library.
>> JLine checks word "columns" in "stty -a" command output to find out terminal size.
>> The word "columns" was translated on AIX's Japanese locale (other locales also), so above error was happened.
>> 
>> OpenJDK for AIX developer could not find this issue because he/she requires AIX Toolbox's coreutils rpm package as build tools.
>> Another stty command is in /opt/freeware/bin/stty, it's in coreutils rpm package.
>> 
>> Standard users' system may not have coreutils rpm package or they may not set /opt/freeware/bin in PATH environment variable.
>> stty command should be executed with C locale to get English message.
>> Additionally, JLine library requires -F option. AIX's stty command does not support -F option,
>> but stty command which is in coreutils rpm package supports -F option.
>> On standard jshell usage, -F option may not be used. But when the system has coreutils rpm package, jshell should use this one.
>
> Thanks! I'm fine with this change then.

@tstuefe - Thanks.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2622

From itakiguchi at openjdk.java.net  Wed Feb 24 13:16:42 2021
From: itakiguchi at openjdk.java.net (Ichiroh Takiguchi)
Date: Wed, 24 Feb 2021 13:16:42 GMT
Subject: Integrated: 8261920: [AIX] jshell command throws java.io.IOError on
 non English locales
In-Reply-To: <ZSx49ohjP_YqU76gSw_kc3l-OmgYDlTPTmlMDSw1r5M=.3b552fdd-ba60-43e2-8e53-5ef4dc506fcb@github.com>
References: <ZSx49ohjP_YqU76gSw_kc3l-OmgYDlTPTmlMDSw1r5M=.3b552fdd-ba60-43e2-8e53-5ef4dc506fcb@github.com>
Message-ID: <38aSnq4V9JV71lpLtASvUqRJlwk9I3QBoxqBy7raZXc=.f060f9c2-566e-4b67-a987-c199d60b69f5@github.com>

On Thu, 18 Feb 2021 03:56:25 GMT, Ichiroh Takiguchi <itakiguchi at openjdk.org> wrote:

> jshell uses JLine library.
> JLine checks word "columns" in "stty -a" command output to find out terminal size.
> The word "columns" was translated on AIX's Japanese locale (other locales also), so above error was happened.
> 
> OpenJDK for AIX developer could not find this issue because he/she requires AIX Toolbox's coreutils rpm package as build tools.
> Another stty command is in /opt/freeware/bin/stty, it's in coreutils rpm package.
> 
> Standard users' system may not have coreutils rpm package or they may not set /opt/freeware/bin in PATH environment variable.
> stty command should be executed with C locale to get English message.
> Additionally, JLine library requires -F option. AIX's stty command does not support -F option,
> but stty command which is in coreutils rpm package supports -F option.
> On standard jshell usage, -F option may not be used. But when the system has coreutils rpm package, jshell should use this one.

This pull request has now been integrated.

Changeset: 2c99bad0
Author:    Ichiroh Takiguchi <itakiguchi at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/2c99bad0
Stats:     8 lines in 2 files changed: 8 ins; 0 del; 0 mod

8261920: [AIX] jshell command throws java.io.IOError on non English locales

Reviewed-by: stuefe

-------------

PR: https://git.openjdk.java.net/jdk/pull/2622