From peter.levart at gmail.com  Tue May  5 16:37:55 2020
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 5 May 2020 18:37:55 +0200
Subject: <i18n dev> [15] RFR: 8244152: Remove unnecessary hash map
 resize in LocaleProviderAdapters
In-Reply-To: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
Message-ID: <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>

Hi Naoto,

On 4/30/20 12:18 AM, naoto.sato at oracle.com wrote:
> Hello,
>
> Please review this small fix to the following issue:
>
> https://bugs.openjdk.java.net/browse/JDK-8244152
>
> The proposed changeset is located at:
>
> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>
> The hash map used there didn't have initial capacity, even though the 
> exact numbers are known.


Well, it has to be calculated 1st (countTokens), but I guess this pays 
off when HashSet (the backing HashMap) does not have to be rehashed then.

The expression you use:

 ??? Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)

...has a minimum value of 16. Why is that? 16 is just HashMap's default 
initialCapacity if not specified explicitly. But if you only want to 
store say 1 entry in the map, you can specify 2 as initialCapacity and 
HashMap will happily work for such case without resizing.


So you could just use:

 ??? (int)(tokens.countTokens() / 0.75f) + 1

And even this expression is sometimes overshooting the minimal required 
value by 1 (when # of tokens is "exact" multiple of 0.75f, say 6). I 
think the following could be used to optimally pre-size the HashMap with 
default load factor 0.75:

 ??? (tokens.countTokens() * 4 + 2) / 3


Regards, Peter

>
> Naoto

From naoto.sato at oracle.com  Tue May  5 17:25:13 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Tue, 5 May 2020 10:25:13 -0700
Subject: <i18n dev> [15] RFR: 8244152: Remove unnecessary hash map
 resize in LocaleProviderAdapters
In-Reply-To: <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
Message-ID: <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>

Hi Peter,

You are correct. Thanks. I'll remove that initial value of 16.

Naoto

On 5/5/20 9:37 AM, Peter Levart wrote:
> Hi Naoto,
> 
> On 4/30/20 12:18 AM, naoto.sato at oracle.com wrote:
>> Hello,
>>
>> Please review this small fix to the following issue:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8244152
>>
>> The proposed changeset is located at:
>>
>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>>
>> The hash map used there didn't have initial capacity, even though the 
>> exact numbers are known.
> 
> 
> Well, it has to be calculated 1st (countTokens), but I guess this pays 
> off when HashSet (the backing HashMap) does not have to be rehashed then.
> 
> The expression you use:
> 
>  ??? Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
> 
> ...has a minimum value of 16. Why is that? 16 is just HashMap's default 
> initialCapacity if not specified explicitly. But if you only want to 
> store say 1 entry in the map, you can specify 2 as initialCapacity and 
> HashMap will happily work for such case without resizing.
> 
> 
> So you could just use:
> 
>  ??? (int)(tokens.countTokens() / 0.75f) + 1
> 
> And even this expression is sometimes overshooting the minimal required 
> value by 1 (when # of tokens is "exact" multiple of 0.75f, say 6). I 
> think the following could be used to optimally pre-size the HashMap with 
> default load factor 0.75:
> 
>  ??? (tokens.countTokens() * 4 + 2) / 3
> 
> 
> Regards, Peter
> 
>>
>> Naoto

From naoto.sato at oracle.com  Tue May  5 18:01:55 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Tue, 5 May 2020 11:01:55 -0700
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
 LocaleProviderAdapters
In-Reply-To: <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
Message-ID: <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>

And here is the fix. Please review.

http://cr.openjdk.java.net/~naoto/8244459/webrev.00/

Naoto

On 5/5/20 10:25 AM, naoto.sato at oracle.com wrote:
> Hi Peter,
> 
> You are correct. Thanks. I'll remove that initial value of 16.
> 
> Naoto
> 
> On 5/5/20 9:37 AM, Peter Levart wrote:
>> Hi Naoto,
>>
>> On 4/30/20 12:18 AM, naoto.sato at oracle.com wrote:
>>> Hello,
>>>
>>> Please review this small fix to the following issue:
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8244152
>>>
>>> The proposed changeset is located at:
>>>
>>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>>>
>>> The hash map used there didn't have initial capacity, even though the 
>>> exact numbers are known.
>>
>>
>> Well, it has to be calculated 1st (countTokens), but I guess this pays 
>> off when HashSet (the backing HashMap) does not have to be rehashed then.
>>
>> The expression you use:
>>
>> ???? Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
>>
>> ...has a minimum value of 16. Why is that? 16 is just HashMap's 
>> default initialCapacity if not specified explicitly. But if you only 
>> want to store say 1 entry in the map, you can specify 2 as 
>> initialCapacity and HashMap will happily work for such case without 
>> resizing.
>>
>>
>> So you could just use:
>>
>> ???? (int)(tokens.countTokens() / 0.75f) + 1
>>
>> And even this expression is sometimes overshooting the minimal 
>> required value by 1 (when # of tokens is "exact" multiple of 0.75f, 
>> say 6). I think the following could be used to optimally pre-size the 
>> HashMap with default load factor 0.75:
>>
>> ???? (tokens.countTokens() * 4 + 2) / 3
>>
>>
>> Regards, Peter
>>
>>>
>>> Naoto

From martinrb at google.com  Tue May  5 18:26:58 2020
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 5 May 2020 11:26:58 -0700
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
	LocaleProviderAdapters
In-Reply-To: <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
 <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
Message-ID: <CA+kOe08MbbYsXb9zYJpM3oPoNYJmx3CNsrzz9-Se5RmrGubFmQ@mail.gmail.com>

See related:
https://guava.dev/releases/23.0/api/docs/com/google/common/collect/Maps.html#newHashMapWithExpectedSize-int-

On Tue, May 5, 2020 at 11:03 AM <naoto.sato at oracle.com> wrote:

> And here is the fix. Please review.
>
> http://cr.openjdk.java.net/~naoto/8244459/webrev.00/
>
> Naoto
>
> On 5/5/20 10:25 AM, naoto.sato at oracle.com wrote:
> > Hi Peter,
> >
> > You are correct. Thanks. I'll remove that initial value of 16.
> >
> > Naoto
> >
> > On 5/5/20 9:37 AM, Peter Levart wrote:
> >> Hi Naoto,
> >>
> >> On 4/30/20 12:18 AM, naoto.sato at oracle.com wrote:
> >>> Hello,
> >>>
> >>> Please review this small fix to the following issue:
> >>>
> >>> https://bugs.openjdk.java.net/browse/JDK-8244152
> >>>
> >>> The proposed changeset is located at:
> >>>
> >>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
> >>>
> >>> The hash map used there didn't have initial capacity, even though the
> >>> exact numbers are known.
> >>
> >>
> >> Well, it has to be calculated 1st (countTokens), but I guess this pays
> >> off when HashSet (the backing HashMap) does not have to be rehashed
> then.
> >>
> >> The expression you use:
> >>
> >>      Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
> >>
> >> ...has a minimum value of 16. Why is that? 16 is just HashMap's
> >> default initialCapacity if not specified explicitly. But if you only
> >> want to store say 1 entry in the map, you can specify 2 as
> >> initialCapacity and HashMap will happily work for such case without
> >> resizing.
> >>
> >>
> >> So you could just use:
> >>
> >>      (int)(tokens.countTokens() / 0.75f) + 1
> >>
> >> And even this expression is sometimes overshooting the minimal
> >> required value by 1 (when # of tokens is "exact" multiple of 0.75f,
> >> say 6). I think the following could be used to optimally pre-size the
> >> HashMap with default load factor 0.75:
> >>
> >>      (tokens.countTokens() * 4 + 2) / 3
> >>
> >>
> >> Regards, Peter
> >>
> >>>
> >>> Naoto
>

From huizhe.wang at oracle.com  Tue May  5 18:29:13 2020
From: huizhe.wang at oracle.com (Joe Wang)
Date: Tue, 5 May 2020 11:29:13 -0700
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
 LocaleProviderAdapters
In-Reply-To: <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
 <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
Message-ID: <0ad39c21-a5a4-6c38-b2ba-30d278b865a6@oracle.com>

Hi Naoto,

It seems you've missed the parentheses in :
Set<String> tagset = new HashSet<>(tokens.countTokens() * 4 + 2 / 3);
vs
Set<String> tagset = new HashSet<>((tokens.countTokens() * 4 + 2) / 3);

-Joe

On 5/5/2020 11:01 AM, naoto.sato at oracle.com wrote:
> And here is the fix. Please review.
>
> http://cr.openjdk.java.net/~naoto/8244459/webrev.00/
>
> Naoto
>
> On 5/5/20 10:25 AM, naoto.sato at oracle.com wrote:
>> Hi Peter,
>>
>> You are correct. Thanks. I'll remove that initial value of 16.
>>
>> Naoto
>>
>> On 5/5/20 9:37 AM, Peter Levart wrote:
>>> Hi Naoto,
>>>
>>> On 4/30/20 12:18 AM, naoto.sato at oracle.com wrote:
>>>> Hello,
>>>>
>>>> Please review this small fix to the following issue:
>>>>
>>>> https://bugs.openjdk.java.net/browse/JDK-8244152
>>>>
>>>> The proposed changeset is located at:
>>>>
>>>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>>>>
>>>> The hash map used there didn't have initial capacity, even though 
>>>> the exact numbers are known.
>>>
>>>
>>> Well, it has to be calculated 1st (countTokens), but I guess this 
>>> pays off when HashSet (the backing HashMap) does not have to be 
>>> rehashed then.
>>>
>>> The expression you use:
>>>
>>> ???? Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
>>>
>>> ...has a minimum value of 16. Why is that? 16 is just HashMap's 
>>> default initialCapacity if not specified explicitly. But if you only 
>>> want to store say 1 entry in the map, you can specify 2 as 
>>> initialCapacity and HashMap will happily work for such case without 
>>> resizing.
>>>
>>>
>>> So you could just use:
>>>
>>> ???? (int)(tokens.countTokens() / 0.75f) + 1
>>>
>>> And even this expression is sometimes overshooting the minimal 
>>> required value by 1 (when # of tokens is "exact" multiple of 0.75f, 
>>> say 6). I think the following could be used to optimally pre-size 
>>> the HashMap with default load factor 0.75:
>>>
>>> ???? (tokens.countTokens() * 4 + 2) / 3
>>>
>>>
>>> Regards, Peter
>>>
>>>>
>>>> Naoto


From mark at macchiato.com  Tue May  5 18:42:56 2020
From: mark at macchiato.com (=?UTF-8?B?TWFyayBEYXZpcyDimJXvuI8=?=)
Date: Tue, 5 May 2020 11:42:56 -0700
Subject: <i18n dev> [15] RFR: 8244152: Remove unnecessary hash map
 resize in LocaleProviderAdapters
In-Reply-To: <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
Message-ID: <CAJ2xs_EnPv4fK_BMDKb3QXeg3TpxNvFsUx8Rp=f+u0raEjvoUQ@mail.gmail.com>

> (int)(tokens.countTokens() / 0.75f) + 1
> (tokens.countTokens() * 4 + 2) / 3

if you want A * X / Y, rounded up, you can use (A * X - 1 ) / Y + 1
eg where X/Y = 4/3,

0 => 0
1 => 2
2 => 3
3 => 4
4 => 6
...



Mark


On Tue, May 5, 2020 at 9:48 AM Peter Levart <peter.levart at gmail.com> wrote:

> Hi Naoto,
>
> On 4/30/20 12:18 AM, naoto.sato at oracle.com wrote:
> > Hello,
> >
> > Please review this small fix to the following issue:
> >
> > https://bugs.openjdk.java.net/browse/JDK-8244152
> >
> > The proposed changeset is located at:
> >
> > https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
> >
> > The hash map used there didn't have initial capacity, even though the
> > exact numbers are known.
>
>
> Well, it has to be calculated 1st (countTokens), but I guess this pays
> off when HashSet (the backing HashMap) does not have to be rehashed then.
>
> The expression you use:
>
>      Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
>
> ...has a minimum value of 16. Why is that? 16 is just HashMap's default
> initialCapacity if not specified explicitly. But if you only want to
> store say 1 entry in the map, you can specify 2 as initialCapacity and
> HashMap will happily work for such case without resizing.
>
>
> So you could just use:
>
>      (int)(tokens.countTokens() / 0.75f) + 1
>
> And even this expression is sometimes overshooting the minimal required
> value by 1 (when # of tokens is "exact" multiple of 0.75f, say 6). I
> think the following could be used to optimally pre-size the HashMap with
> default load factor 0.75:
>
>      (tokens.countTokens() * 4 + 2) / 3
>
>
> Regards, Peter
>
> >
> > Naoto
>

From peter.levart at gmail.com  Tue May  5 19:12:37 2020
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 5 May 2020 21:12:37 +0200
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
 LocaleProviderAdapters
In-Reply-To: <CA+kOe08MbbYsXb9zYJpM3oPoNYJmx3CNsrzz9-Se5RmrGubFmQ@mail.gmail.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
 <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
 <CA+kOe08MbbYsXb9zYJpM3oPoNYJmx3CNsrzz9-Se5RmrGubFmQ@mail.gmail.com>
Message-ID: <ab4ad543-013b-1261-fa1c-3fc87c21402a@gmail.com>

Hi Martin,

On 5/5/20 8:26 PM, Martin Buchholz wrote:
> See related:
> https://guava.dev/releases/23.0/api/docs/com/google/common/collect/Maps.html#newHashMapWithExpectedSize-int-


This is basically the same calculation (or at least gives same result) 
as Naoto did (without the max part):

Naoto: (int)(expectedSize / 0.75f) + 1

Guava: (int) ((float) expectedSize / 0.75F + 1.0F)

but in case expectedSize is a multiple of 3, it gives the result which 
is 1 more than needed. If what is needed is also a power of 2, then 
twice the needed space is allocated in the HashMap backing table.


Regards, Peter


>
> On Tue, May 5, 2020 at 11:03 AM <naoto.sato at oracle.com 
> <mailto:naoto.sato at oracle.com>> wrote:
>
>     And here is the fix. Please review.
>
>     http://cr.openjdk.java.net/~naoto/8244459/webrev.00/
>
>     Naoto
>
>     On 5/5/20 10:25 AM, naoto.sato at oracle.com
>     <mailto:naoto.sato at oracle.com> wrote:
>     > Hi Peter,
>     >
>     > You are correct. Thanks. I'll remove that initial value of 16.
>     >
>     > Naoto
>     >
>     > On 5/5/20 9:37 AM, Peter Levart wrote:
>     >> Hi Naoto,
>     >>
>     >> On 4/30/20 12:18 AM, naoto.sato at oracle.com
>     <mailto:naoto.sato at oracle.com> wrote:
>     >>> Hello,
>     >>>
>     >>> Please review this small fix to the following issue:
>     >>>
>     >>> https://bugs.openjdk.java.net/browse/JDK-8244152
>     >>>
>     >>> The proposed changeset is located at:
>     >>>
>     >>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>     >>>
>     >>> The hash map used there didn't have initial capacity, even
>     though the
>     >>> exact numbers are known.
>     >>
>     >>
>     >> Well, it has to be calculated 1st (countTokens), but I guess
>     this pays
>     >> off when HashSet (the backing HashMap) does not have to be
>     rehashed then.
>     >>
>     >> The expression you use:
>     >>
>     >> ???? Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
>     >>
>     >> ...has a minimum value of 16. Why is that? 16 is just HashMap's
>     >> default initialCapacity if not specified explicitly. But if you
>     only
>     >> want to store say 1 entry in the map, you can specify 2 as
>     >> initialCapacity and HashMap will happily work for such case
>     without
>     >> resizing.
>     >>
>     >>
>     >> So you could just use:
>     >>
>     >> ???? (int)(tokens.countTokens() / 0.75f) + 1
>     >>
>     >> And even this expression is sometimes overshooting the minimal
>     >> required value by 1 (when # of tokens is "exact" multiple of
>     0.75f,
>     >> say 6). I think the following could be used to optimally
>     pre-size the
>     >> HashMap with default load factor 0.75:
>     >>
>     >> ???? (tokens.countTokens() * 4 + 2) / 3
>     >>
>     >>
>     >> Regards, Peter
>     >>
>     >>>
>     >>> Naoto
>

From martinrb at google.com  Tue May  5 19:41:31 2020
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 5 May 2020 12:41:31 -0700
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
	LocaleProviderAdapters
In-Reply-To: <ab4ad543-013b-1261-fa1c-3fc87c21402a@gmail.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
 <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
 <CA+kOe08MbbYsXb9zYJpM3oPoNYJmx3CNsrzz9-Se5RmrGubFmQ@mail.gmail.com>
 <ab4ad543-013b-1261-fa1c-3fc87c21402a@gmail.com>
Message-ID: <CA+kOe0-YmLOWONqPMbGxKDsA9HMi9SbWZYaCZYD3VjgR4iBMBA@mail.gmail.com>

Hi Peter,

Are you saying guava has a tiny bug?

On Tue, May 5, 2020 at 12:12 PM Peter Levart <peter.levart at gmail.com> wrote:

> Hi Martin,
> On 5/5/20 8:26 PM, Martin Buchholz wrote:
>
> See related:
>
> https://guava.dev/releases/23.0/api/docs/com/google/common/collect/Maps.html#newHashMapWithExpectedSize-int-
>
>
> This is basically the same calculation (or at least gives same result) as
> Naoto did (without the max part):
>
> Naoto: (int)(expectedSize / 0.75f) + 1
>
> Guava: (int) ((float) expectedSize / 0.75F + 1.0F)
>
> but in case expectedSize is a multiple of 3, it gives the result which is
> 1 more than needed. If what is needed is also a power of 2, then twice the
> needed space is allocated in the HashMap backing table.
>
>
> Regards, Peter
>
>
>
> On Tue, May 5, 2020 at 11:03 AM <naoto.sato at oracle.com> wrote:
>
>> And here is the fix. Please review.
>>
>> http://cr.openjdk.java.net/~naoto/8244459/webrev.00/
>>
>> Naoto
>>
>> On 5/5/20 10:25 AM, naoto.sato at oracle.com wrote:
>> > Hi Peter,
>> >
>> > You are correct. Thanks. I'll remove that initial value of 16.
>> >
>> > Naoto
>> >
>> > On 5/5/20 9:37 AM, Peter Levart wrote:
>> >> Hi Naoto,
>> >>
>> >> On 4/30/20 12:18 AM, naoto.sato at oracle.com wrote:
>> >>> Hello,
>> >>>
>> >>> Please review this small fix to the following issue:
>> >>>
>> >>> https://bugs.openjdk.java.net/browse/JDK-8244152
>> >>>
>> >>> The proposed changeset is located at:
>> >>>
>> >>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>> >>>
>> >>> The hash map used there didn't have initial capacity, even though the
>> >>> exact numbers are known.
>> >>
>> >>
>> >> Well, it has to be calculated 1st (countTokens), but I guess this pays
>> >> off when HashSet (the backing HashMap) does not have to be rehashed
>> then.
>> >>
>> >> The expression you use:
>> >>
>> >>      Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
>> >>
>> >> ...has a minimum value of 16. Why is that? 16 is just HashMap's
>> >> default initialCapacity if not specified explicitly. But if you only
>> >> want to store say 1 entry in the map, you can specify 2 as
>> >> initialCapacity and HashMap will happily work for such case without
>> >> resizing.
>> >>
>> >>
>> >> So you could just use:
>> >>
>> >>      (int)(tokens.countTokens() / 0.75f) + 1
>> >>
>> >> And even this expression is sometimes overshooting the minimal
>> >> required value by 1 (when # of tokens is "exact" multiple of 0.75f,
>> >> say 6). I think the following could be used to optimally pre-size the
>> >> HashMap with default load factor 0.75:
>> >>
>> >>      (tokens.countTokens() * 4 + 2) / 3
>> >>
>> >>
>> >> Regards, Peter
>> >>
>> >>>
>> >>> Naoto
>>
>

From peter.levart at gmail.com  Tue May  5 20:01:56 2020
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 5 May 2020 22:01:56 +0200
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
 LocaleProviderAdapters
In-Reply-To: <CA+kOe0-YmLOWONqPMbGxKDsA9HMi9SbWZYaCZYD3VjgR4iBMBA@mail.gmail.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
 <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
 <CA+kOe08MbbYsXb9zYJpM3oPoNYJmx3CNsrzz9-Se5RmrGubFmQ@mail.gmail.com>
 <ab4ad543-013b-1261-fa1c-3fc87c21402a@gmail.com>
 <CA+kOe0-YmLOWONqPMbGxKDsA9HMi9SbWZYaCZYD3VjgR4iBMBA@mail.gmail.com>
Message-ID: <4e994519-c6bd-a6e9-2ef8-1e8be4ef6008@gmail.com>


On 5/5/20 9:41 PM, Martin Buchholz wrote:
> Hi Peter,
>
> Are you saying guava has a tiny bug?


If it was just 1 too much when expected size is a multiple of 3 then 
that would not be a bug, just sub-optimal calculation. And the same 
calculation is performed also in JDK when a copy constructor is called 
for example.


But I investigated further and what I found could be considered a bug. 
Sometimes, the following expression:


(int) ((float) expectedSize / 0.75f + 1.0f)


...calculates a value that is not enough (due to floating point 
arithmetic and conversion to int) to store the expectedSize elements 
into the HashMap without re-hashing.


What HashMap does with initialCapacity parameter is to round it up to 
nearest power of 2:

 ??? static int tableSizeFor(int cap) {
 ??????? int n = -1 >>> Integer.numberOfLeadingZeros(cap - 1);
 ??????? return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ? MAXIMUM_CAPACITY 
: n + 1;
 ??? }

then it uses this as the initial backing table size. From that table 
size it calculates the threshold value:

 ??? static int threshold(int cap) {
 ??????? float ft = (float) cap * 0.75f;
 ??????? return (cap < MAXIMUM_CAPACITY && ft < (float) MAXIMUM_CAPACITY ?
 ??????????????? (int) ft : Integer.MAX_VALUE);
 ??? }

... and uses it as the max. number of elements that a HashMap can hold 
before it is re-hashed. So I did the following test (comparing the 
effectiveness of above formula with alternative (expectedSize*4+2)/3 
formula):


public class HMTest {
 ??? static final int MAXIMUM_CAPACITY = 1 << 30;

 ??? static int tableSizeFor(int cap) {
 ??????? int n = -1 >>> Integer.numberOfLeadingZeros(cap - 1);
 ??????? return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ? MAXIMUM_CAPACITY 
: n + 1;
 ??? }

 ??? static int threshold(int cap) {
 ??????? float ft = (float) cap * 0.75f;
 ??????? return (cap < MAXIMUM_CAPACITY && ft < (float) MAXIMUM_CAPACITY ?
 ??????????????? (int) ft : Integer.MAX_VALUE);
 ??? }

 ??? public static void main(String[] args) {
 ??????? for (int expectedSize = 0; expectedSize < (Integer.MAX_VALUE - 
2) / 4; expectedSize++) {
 ??????????? int cap1 = (int) ((float) expectedSize / 0.75f + 1.0f);
 ??????????? int cap2 = (expectedSize * 4 + 2) / 3;
 ??????????? int ts1 = tableSizeFor(cap1);
 ??????????? int ts2 = tableSizeFor(cap2);
 ??????????? int th1 = threshold(ts1);
 ??????????? int th2 = threshold(ts2);

 ??????????? if (th1 < expectedSize || th2 < expectedSize) {
 ??????????????? System.out.printf("%d: (%d, %d, %d)%s (%d, %d, %d)%s\n",
 ??????????????????????? expectedSize,
 ??????????????????????? cap1, ts1, th1, (th1 < expectedSize) ? "!" : " ",
 ??????????????????????? cap2, ts2, th2, (th2 < expectedSize) ? "!" : " "
 ??????????????? );
 ??????????? }
 ??????? }
 ??? }
}


And what this prints is the following:


25165825: (33554432, 33554432, 25165824)! (33554434, 67108864, 50331648)
50331649: (67108864, 67108864, 50331648)! (67108866, 134217728, 100663296)
50331650: (67108864, 67108864, 50331648)! (67108867, 134217728, 100663296)
100663297: (134217728, 134217728, 100663296)! (134217730, 268435456, 
201326592)
100663298: (134217728, 134217728, 100663296)! (134217731, 268435456, 
201326592)
100663299: (134217728, 134217728, 100663296)! (134217732, 268435456, 
201326592)
100663300: (134217728, 134217728, 100663296)! (134217734, 268435456, 
201326592)
201326593: (268435456, 268435456, 201326592)! (268435458, 536870912, 
402653184)
201326594: (268435456, 268435456, 201326592)! (268435459, 536870912, 
402653184)
201326595: (268435456, 268435456, 201326592)! (268435460, 536870912, 
402653184)
201326596: (268435456, 268435456, 201326592)! (268435462, 536870912, 
402653184)
201326597: (268435456, 268435456, 201326592)! (268435463, 536870912, 
402653184)
201326598: (268435456, 268435456, 201326592)! (268435464, 536870912, 
402653184)
201326599: (268435456, 268435456, 201326592)! (268435466, 536870912, 
402653184)
201326600: (268435456, 268435456, 201326592)! (268435467, 536870912, 
402653184)
402653185: (536870912, 536870912, 402653184)! (536870914, 1073741824, 
2147483647)
402653186: (536870912, 536870912, 402653184)! (536870915, 1073741824, 
2147483647)
402653187: (536870912, 536870912, 402653184)! (536870916, 1073741824, 
2147483647)
402653188: (536870912, 536870912, 402653184)! (536870918, 1073741824, 
2147483647)
402653189: (536870912, 536870912, 402653184)! (536870919, 1073741824, 
2147483647)
402653190: (536870912, 536870912, 402653184)! (536870920, 1073741824, 
2147483647)
402653191: (536870912, 536870912, 402653184)! (536870922, 1073741824, 
2147483647)
402653192: (536870912, 536870912, 402653184)! (536870923, 1073741824, 
2147483647)
402653193: (536870912, 536870912, 402653184)! (536870924, 1073741824, 
2147483647)
402653194: (536870912, 536870912, 402653184)! (536870926, 1073741824, 
2147483647)
402653195: (536870912, 536870912, 402653184)! (536870927, 1073741824, 
2147483647)
402653196: (536870912, 536870912, 402653184)! (536870928, 1073741824, 
2147483647)
402653197: (536870912, 536870912, 402653184)! (536870930, 1073741824, 
2147483647)
402653198: (536870912, 536870912, 402653184)! (536870931, 1073741824, 
2147483647)
402653199: (536870912, 536870912, 402653184)! (536870932, 1073741824, 
2147483647)
402653200: (536870912, 536870912, 402653184)! (536870934, 1073741824, 
2147483647)


So as you see, for expectedSize < (Integer.MAX_VALUE - 2) / 4 (where the 
alternative formula does not experience overflow and is enough for 
Naoto's case) all miscalculations are due to the JDK/Guava formula which 
in those cases calculates a value that is less than alternative 
formula's value and too small to adequately pre-size the HashMap table.


Voila, we have some bugs to fix or I am doing something wrong here.


Regards, Peter


>
> On Tue, May 5, 2020 at 12:12 PM Peter Levart <peter.levart at gmail.com 
> <mailto:peter.levart at gmail.com>> wrote:
>
>     Hi Martin,
>
>     On 5/5/20 8:26 PM, Martin Buchholz wrote:
>>     See related:
>>     https://guava.dev/releases/23.0/api/docs/com/google/common/collect/Maps.html#newHashMapWithExpectedSize-int-
>
>
>     This is basically the same calculation (or at least gives same
>     result) as Naoto did (without the max part):
>
>     Naoto: (int)(expectedSize / 0.75f) + 1
>
>     Guava: (int) ((float) expectedSize / 0.75F + 1.0F)
>
>     but in case expectedSize is a multiple of 3, it gives the result
>     which is 1 more than needed. If what is needed is also a power of
>     2, then twice the needed space is allocated in the HashMap backing
>     table.
>
>
>     Regards, Peter
>
>
>>
>>     On Tue, May 5, 2020 at 11:03 AM <naoto.sato at oracle.com
>>     <mailto:naoto.sato at oracle.com>> wrote:
>>
>>         And here is the fix. Please review.
>>
>>         http://cr.openjdk.java.net/~naoto/8244459/webrev.00/
>>
>>         Naoto
>>
>>         On 5/5/20 10:25 AM, naoto.sato at oracle.com
>>         <mailto:naoto.sato at oracle.com> wrote:
>>         > Hi Peter,
>>         >
>>         > You are correct. Thanks. I'll remove that initial value of 16.
>>         >
>>         > Naoto
>>         >
>>         > On 5/5/20 9:37 AM, Peter Levart wrote:
>>         >> Hi Naoto,
>>         >>
>>         >> On 4/30/20 12:18 AM, naoto.sato at oracle.com
>>         <mailto:naoto.sato at oracle.com> wrote:
>>         >>> Hello,
>>         >>>
>>         >>> Please review this small fix to the following issue:
>>         >>>
>>         >>> https://bugs.openjdk.java.net/browse/JDK-8244152
>>         >>>
>>         >>> The proposed changeset is located at:
>>         >>>
>>         >>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>>         >>>
>>         >>> The hash map used there didn't have initial capacity,
>>         even though the
>>         >>> exact numbers are known.
>>         >>
>>         >>
>>         >> Well, it has to be calculated 1st (countTokens), but I
>>         guess this pays
>>         >> off when HashSet (the backing HashMap) does not have to be
>>         rehashed then.
>>         >>
>>         >> The expression you use:
>>         >>
>>         >> ???? Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
>>         >>
>>         >> ...has a minimum value of 16. Why is that? 16 is just
>>         HashMap's
>>         >> default initialCapacity if not specified explicitly. But
>>         if you only
>>         >> want to store say 1 entry in the map, you can specify 2 as
>>         >> initialCapacity and HashMap will happily work for such
>>         case without
>>         >> resizing.
>>         >>
>>         >>
>>         >> So you could just use:
>>         >>
>>         >> ???? (int)(tokens.countTokens() / 0.75f) + 1
>>         >>
>>         >> And even this expression is sometimes overshooting the
>>         minimal
>>         >> required value by 1 (when # of tokens is "exact" multiple
>>         of 0.75f,
>>         >> say 6). I think the following could be used to optimally
>>         pre-size the
>>         >> HashMap with default load factor 0.75:
>>         >>
>>         >> ???? (tokens.countTokens() * 4 + 2) / 3
>>         >>
>>         >>
>>         >> Regards, Peter
>>         >>
>>         >>>
>>         >>> Naoto
>>

From peter.levart at gmail.com  Tue May  5 20:26:52 2020
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 5 May 2020 22:26:52 +0200
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
 LocaleProviderAdapters
In-Reply-To: <4e994519-c6bd-a6e9-2ef8-1e8be4ef6008@gmail.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
 <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
 <CA+kOe08MbbYsXb9zYJpM3oPoNYJmx3CNsrzz9-Se5RmrGubFmQ@mail.gmail.com>
 <ab4ad543-013b-1261-fa1c-3fc87c21402a@gmail.com>
 <CA+kOe0-YmLOWONqPMbGxKDsA9HMi9SbWZYaCZYD3VjgR4iBMBA@mail.gmail.com>
 <4e994519-c6bd-a6e9-2ef8-1e8be4ef6008@gmail.com>
Message-ID: <fecd7cee-31a9-7a1a-4cae-70822501f716@gmail.com>

There's more...


Guava (and JDK in copy constructor) formula:

 ??? (int) ((float) expectedSize / 0.75f + 1.0f)


is not the same as Naoto's formula:

 ??? (int) (expectedSize / 0.75f) + 1


Notice that Naoto does addition of 1 in integer arithmetic after 
conversion to int, while Guava/JDK does in floating point before 
conversion to int. If I put Naoto's formula into my test program, no 
undercalculations are detected.


So while Naoto's formula is sub-optimal for expectedSizes that are 
multiples of 3, the Guava/JDK also has a bug.


Regards, Peter


On 5/5/20 10:01 PM, Peter Levart wrote:
>
>
> On 5/5/20 9:41 PM, Martin Buchholz wrote:
>> Hi Peter,
>>
>> Are you saying guava has a tiny bug?
>
>
> If it was just 1 too much when expected size is a multiple of 3 then 
> that would not be a bug, just sub-optimal calculation. And the same 
> calculation is performed also in JDK when a copy constructor is called 
> for example.
>
>
> But I investigated further and what I found could be considered a bug. 
> Sometimes, the following expression:
>
>
> (int) ((float) expectedSize / 0.75f + 1.0f)
>
>
> ...calculates a value that is not enough (due to floating point 
> arithmetic and conversion to int) to store the expectedSize elements 
> into the HashMap without re-hashing.
>
>
> What HashMap does with initialCapacity parameter is to round it up to 
> nearest power of 2:
>
> ??? static int tableSizeFor(int cap) {
> ??????? int n = -1 >>> Integer.numberOfLeadingZeros(cap - 1);
> ??????? return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ? 
> MAXIMUM_CAPACITY : n + 1;
> ??? }
>
> then it uses this as the initial backing table size. From that table 
> size it calculates the threshold value:
>
> ??? static int threshold(int cap) {
> ??????? float ft = (float) cap * 0.75f;
> ??????? return (cap < MAXIMUM_CAPACITY && ft < (float) MAXIMUM_CAPACITY ?
> ??????????????? (int) ft : Integer.MAX_VALUE);
> ??? }
>
> ... and uses it as the max. number of elements that a HashMap can hold 
> before it is re-hashed. So I did the following test (comparing the 
> effectiveness of above formula with alternative (expectedSize*4+2)/3 
> formula):
>
>
> public class HMTest {
> ??? static final int MAXIMUM_CAPACITY = 1 << 30;
>
> ??? static int tableSizeFor(int cap) {
> ??????? int n = -1 >>> Integer.numberOfLeadingZeros(cap - 1);
> ??????? return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ? 
> MAXIMUM_CAPACITY : n + 1;
> ??? }
>
> ??? static int threshold(int cap) {
> ??????? float ft = (float) cap * 0.75f;
> ??????? return (cap < MAXIMUM_CAPACITY && ft < (float) MAXIMUM_CAPACITY ?
> ??????????????? (int) ft : Integer.MAX_VALUE);
> ??? }
>
> ??? public static void main(String[] args) {
> ??????? for (int expectedSize = 0; expectedSize < (Integer.MAX_VALUE - 
> 2) / 4; expectedSize++) {
> ??????????? int cap1 = (int) ((float) expectedSize / 0.75f + 1.0f);
> ??????????? int cap2 = (expectedSize * 4 + 2) / 3;
> ??????????? int ts1 = tableSizeFor(cap1);
> ??????????? int ts2 = tableSizeFor(cap2);
> ??????????? int th1 = threshold(ts1);
> ??????????? int th2 = threshold(ts2);
>
> ??????????? if (th1 < expectedSize || th2 < expectedSize) {
> ??????????????? System.out.printf("%d: (%d, %d, %d)%s (%d, %d, %d)%s\n",
> ??????????????????????? expectedSize,
> ??????????????????????? cap1, ts1, th1, (th1 < expectedSize) ? "!" : " ",
> ??????????????????????? cap2, ts2, th2, (th2 < expectedSize) ? "!" : " "
> ??????????????? );
> ??????????? }
> ??????? }
> ??? }
> }
>
>
> And what this prints is the following:
>
>
> 25165825: (33554432, 33554432, 25165824)! (33554434, 67108864, 50331648)
> 50331649: (67108864, 67108864, 50331648)! (67108866, 134217728, 
> 100663296)
> 50331650: (67108864, 67108864, 50331648)! (67108867, 134217728, 
> 100663296)
> 100663297: (134217728, 134217728, 100663296)! (134217730, 268435456, 
> 201326592)
> 100663298: (134217728, 134217728, 100663296)! (134217731, 268435456, 
> 201326592)
> 100663299: (134217728, 134217728, 100663296)! (134217732, 268435456, 
> 201326592)
> 100663300: (134217728, 134217728, 100663296)! (134217734, 268435456, 
> 201326592)
> 201326593: (268435456, 268435456, 201326592)! (268435458, 536870912, 
> 402653184)
> 201326594: (268435456, 268435456, 201326592)! (268435459, 536870912, 
> 402653184)
> 201326595: (268435456, 268435456, 201326592)! (268435460, 536870912, 
> 402653184)
> 201326596: (268435456, 268435456, 201326592)! (268435462, 536870912, 
> 402653184)
> 201326597: (268435456, 268435456, 201326592)! (268435463, 536870912, 
> 402653184)
> 201326598: (268435456, 268435456, 201326592)! (268435464, 536870912, 
> 402653184)
> 201326599: (268435456, 268435456, 201326592)! (268435466, 536870912, 
> 402653184)
> 201326600: (268435456, 268435456, 201326592)! (268435467, 536870912, 
> 402653184)
> 402653185: (536870912, 536870912, 402653184)! (536870914, 1073741824, 
> 2147483647)
> 402653186: (536870912, 536870912, 402653184)! (536870915, 1073741824, 
> 2147483647)
> 402653187: (536870912, 536870912, 402653184)! (536870916, 1073741824, 
> 2147483647)
> 402653188: (536870912, 536870912, 402653184)! (536870918, 1073741824, 
> 2147483647)
> 402653189: (536870912, 536870912, 402653184)! (536870919, 1073741824, 
> 2147483647)
> 402653190: (536870912, 536870912, 402653184)! (536870920, 1073741824, 
> 2147483647)
> 402653191: (536870912, 536870912, 402653184)! (536870922, 1073741824, 
> 2147483647)
> 402653192: (536870912, 536870912, 402653184)! (536870923, 1073741824, 
> 2147483647)
> 402653193: (536870912, 536870912, 402653184)! (536870924, 1073741824, 
> 2147483647)
> 402653194: (536870912, 536870912, 402653184)! (536870926, 1073741824, 
> 2147483647)
> 402653195: (536870912, 536870912, 402653184)! (536870927, 1073741824, 
> 2147483647)
> 402653196: (536870912, 536870912, 402653184)! (536870928, 1073741824, 
> 2147483647)
> 402653197: (536870912, 536870912, 402653184)! (536870930, 1073741824, 
> 2147483647)
> 402653198: (536870912, 536870912, 402653184)! (536870931, 1073741824, 
> 2147483647)
> 402653199: (536870912, 536870912, 402653184)! (536870932, 1073741824, 
> 2147483647)
> 402653200: (536870912, 536870912, 402653184)! (536870934, 1073741824, 
> 2147483647)
>
>
> So as you see, for expectedSize < (Integer.MAX_VALUE - 2) / 4 (where 
> the alternative formula does not experience overflow and is enough for 
> Naoto's case) all miscalculations are due to the JDK/Guava formula 
> which in those cases calculates a value that is less than alternative 
> formula's value and too small to adequately pre-size the HashMap table.
>
>
> Voila, we have some bugs to fix or I am doing something wrong here.
>
>
> Regards, Peter
>
>
>>
>> On Tue, May 5, 2020 at 12:12 PM Peter Levart <peter.levart at gmail.com 
>> <mailto:peter.levart at gmail.com>> wrote:
>>
>>     Hi Martin,
>>
>>     On 5/5/20 8:26 PM, Martin Buchholz wrote:
>>>     See related:
>>>     https://guava.dev/releases/23.0/api/docs/com/google/common/collect/Maps.html#newHashMapWithExpectedSize-int-
>>
>>
>>     This is basically the same calculation (or at least gives same
>>     result) as Naoto did (without the max part):
>>
>>     Naoto: (int)(expectedSize / 0.75f) + 1
>>
>>     Guava: (int) ((float) expectedSize / 0.75F + 1.0F)
>>
>>     but in case expectedSize is a multiple of 3, it gives the result
>>     which is 1 more than needed. If what is needed is also a power of
>>     2, then twice the needed space is allocated in the HashMap
>>     backing table.
>>
>>
>>     Regards, Peter
>>
>>
>>>
>>>     On Tue, May 5, 2020 at 11:03 AM <naoto.sato at oracle.com
>>>     <mailto:naoto.sato at oracle.com>> wrote:
>>>
>>>         And here is the fix. Please review.
>>>
>>>         http://cr.openjdk.java.net/~naoto/8244459/webrev.00/
>>>
>>>         Naoto
>>>
>>>         On 5/5/20 10:25 AM, naoto.sato at oracle.com
>>>         <mailto:naoto.sato at oracle.com> wrote:
>>>         > Hi Peter,
>>>         >
>>>         > You are correct. Thanks. I'll remove that initial value of 16.
>>>         >
>>>         > Naoto
>>>         >
>>>         > On 5/5/20 9:37 AM, Peter Levart wrote:
>>>         >> Hi Naoto,
>>>         >>
>>>         >> On 4/30/20 12:18 AM, naoto.sato at oracle.com
>>>         <mailto:naoto.sato at oracle.com> wrote:
>>>         >>> Hello,
>>>         >>>
>>>         >>> Please review this small fix to the following issue:
>>>         >>>
>>>         >>> https://bugs.openjdk.java.net/browse/JDK-8244152
>>>         >>>
>>>         >>> The proposed changeset is located at:
>>>         >>>
>>>         >>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>>>         >>>
>>>         >>> The hash map used there didn't have initial capacity,
>>>         even though the
>>>         >>> exact numbers are known.
>>>         >>
>>>         >>
>>>         >> Well, it has to be calculated 1st (countTokens), but I
>>>         guess this pays
>>>         >> off when HashSet (the backing HashMap) does not have to
>>>         be rehashed then.
>>>         >>
>>>         >> The expression you use:
>>>         >>
>>>         >> ???? Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
>>>         >>
>>>         >> ...has a minimum value of 16. Why is that? 16 is just
>>>         HashMap's
>>>         >> default initialCapacity if not specified explicitly. But
>>>         if you only
>>>         >> want to store say 1 entry in the map, you can specify 2 as
>>>         >> initialCapacity and HashMap will happily work for such
>>>         case without
>>>         >> resizing.
>>>         >>
>>>         >>
>>>         >> So you could just use:
>>>         >>
>>>         >> ???? (int)(tokens.countTokens() / 0.75f) + 1
>>>         >>
>>>         >> And even this expression is sometimes overshooting the
>>>         minimal
>>>         >> required value by 1 (when # of tokens is "exact" multiple
>>>         of 0.75f,
>>>         >> say 6). I think the following could be used to optimally
>>>         pre-size the
>>>         >> HashMap with default load factor 0.75:
>>>         >>
>>>         >> ???? (tokens.countTokens() * 4 + 2) / 3
>>>         >>
>>>         >>
>>>         >> Regards, Peter
>>>         >>
>>>         >>>
>>>         >>> Naoto
>>>

From peter.levart at gmail.com  Tue May  5 20:50:49 2020
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 5 May 2020 22:50:49 +0200
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
 LocaleProviderAdapters
In-Reply-To: <0ad39c21-a5a4-6c38-b2ba-30d278b865a6@oracle.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
 <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
 <0ad39c21-a5a4-6c38-b2ba-30d278b865a6@oracle.com>
Message-ID: <87e924eb-7960-c8f9-c490-ca6e7b90cd4b@gmail.com>


On 5/5/20 8:29 PM, Joe Wang wrote:
> Hi Naoto,
>
> It seems you've missed the parentheses in :
> Set<String> tagset = new HashSet<>(tokens.countTokens() * 4 + 2 / 3);
> vs
> Set<String> tagset = new HashSet<>((tokens.countTokens() * 4 + 2) / 3);
>
> -Joe


Just a reminder to Naoto: In all this excitement, don't loose track on 
those parentheses (like Dirty Harry lost track of shots fired) ;-)

Regards, Peter


>
> On 5/5/2020 11:01 AM, naoto.sato at oracle.com wrote:
>> And here is the fix. Please review.
>>
>> http://cr.openjdk.java.net/~naoto/8244459/webrev.00/
>>
>> Naoto
>>
>> On 5/5/20 10:25 AM, naoto.sato at oracle.com wrote:
>>> Hi Peter,
>>>
>>> You are correct. Thanks. I'll remove that initial value of 16.
>>>
>>> Naoto
>>>
>>> On 5/5/20 9:37 AM, Peter Levart wrote:
>>>> Hi Naoto,
>>>>
>>>> On 4/30/20 12:18 AM, naoto.sato at oracle.com wrote:
>>>>> Hello,
>>>>>
>>>>> Please review this small fix to the following issue:
>>>>>
>>>>> https://bugs.openjdk.java.net/browse/JDK-8244152
>>>>>
>>>>> The proposed changeset is located at:
>>>>>
>>>>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>>>>>
>>>>> The hash map used there didn't have initial capacity, even though 
>>>>> the exact numbers are known.
>>>>
>>>>
>>>> Well, it has to be calculated 1st (countTokens), but I guess this 
>>>> pays off when HashSet (the backing HashMap) does not have to be 
>>>> rehashed then.
>>>>
>>>> The expression you use:
>>>>
>>>> ???? Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
>>>>
>>>> ...has a minimum value of 16. Why is that? 16 is just HashMap's 
>>>> default initialCapacity if not specified explicitly. But if you 
>>>> only want to store say 1 entry in the map, you can specify 2 as 
>>>> initialCapacity and HashMap will happily work for such case without 
>>>> resizing.
>>>>
>>>>
>>>> So you could just use:
>>>>
>>>> ???? (int)(tokens.countTokens() / 0.75f) + 1
>>>>
>>>> And even this expression is sometimes overshooting the minimal 
>>>> required value by 1 (when # of tokens is "exact" multiple of 0.75f, 
>>>> say 6). I think the following could be used to optimally pre-size 
>>>> the HashMap with default load factor 0.75:
>>>>
>>>> ???? (tokens.countTokens() * 4 + 2) / 3
>>>>
>>>>
>>>> Regards, Peter
>>>>
>>>>>
>>>>> Naoto
>

From naoto.sato at oracle.com  Tue May  5 21:08:46 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Tue, 5 May 2020 14:08:46 -0700
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
 LocaleProviderAdapters
In-Reply-To: <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
 <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
Message-ID: <5c22598c-cba4-bcb9-4c52-6994e95d4355@oracle.com>

Thanks, all. I didn't see this coming!

If I understand the discussion correctly, Peter's suggestion is the most 
optimal (Mark, your formula produces 1 for the expected size is 0, 
although it won't be happening in this particular case). And Joe, thank 
you for finding my silly mistake :-) So here is the updated webrev:

http://cr.openjdk.java.net/~naoto/8244459/webrev.01/

Naoto


On 5/5/20 11:01 AM, naoto.sato at oracle.com wrote:
> And here is the fix. Please review.
> 
> http://cr.openjdk.java.net/~naoto/8244459/webrev.00/
> 
> Naoto
> 
> On 5/5/20 10:25 AM, naoto.sato at oracle.com wrote:
>> Hi Peter,
>>
>> You are correct. Thanks. I'll remove that initial value of 16.
>>
>> Naoto
>>
>> On 5/5/20 9:37 AM, Peter Levart wrote:
>>> Hi Naoto,
>>>
>>> On 4/30/20 12:18 AM, naoto.sato at oracle.com wrote:
>>>> Hello,
>>>>
>>>> Please review this small fix to the following issue:
>>>>
>>>> https://bugs.openjdk.java.net/browse/JDK-8244152
>>>>
>>>> The proposed changeset is located at:
>>>>
>>>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>>>>
>>>> The hash map used there didn't have initial capacity, even though 
>>>> the exact numbers are known.
>>>
>>>
>>> Well, it has to be calculated 1st (countTokens), but I guess this 
>>> pays off when HashSet (the backing HashMap) does not have to be 
>>> rehashed then.
>>>
>>> The expression you use:
>>>
>>> ???? Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
>>>
>>> ...has a minimum value of 16. Why is that? 16 is just HashMap's 
>>> default initialCapacity if not specified explicitly. But if you only 
>>> want to store say 1 entry in the map, you can specify 2 as 
>>> initialCapacity and HashMap will happily work for such case without 
>>> resizing.
>>>
>>>
>>> So you could just use:
>>>
>>> ???? (int)(tokens.countTokens() / 0.75f) + 1
>>>
>>> And even this expression is sometimes overshooting the minimal 
>>> required value by 1 (when # of tokens is "exact" multiple of 0.75f, 
>>> say 6). I think the following could be used to optimally pre-size the 
>>> HashMap with default load factor 0.75:
>>>
>>> ???? (tokens.countTokens() * 4 + 2) / 3
>>>
>>>
>>> Regards, Peter
>>>
>>>>
>>>> Naoto

From huizhe.wang at oracle.com  Tue May  5 21:40:22 2020
From: huizhe.wang at oracle.com (Joe Wang)
Date: Tue, 5 May 2020 14:40:22 -0700
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
 LocaleProviderAdapters
In-Reply-To: <5c22598c-cba4-bcb9-4c52-6994e95d4355@oracle.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
 <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
 <5c22598c-cba4-bcb9-4c52-6994e95d4355@oracle.com>
Message-ID: <bf2fcda3-81b4-d219-b8f7-bc59241bc500@oracle.com>


On 5/5/2020 2:08 PM, naoto.sato at oracle.com wrote:
> Thanks, all. I didn't see this coming!

+1, just when one might think it was just a minor tweak... ;-)

>
> If I understand the discussion correctly, Peter's suggestion is the 
> most optimal (Mark, your formula produces 1 for the expected size is 
> 0, although it won't be happening in this particular case). And Joe, 
> thank you for finding my silly mistake :-) So here is the updated webrev:
>
> http://cr.openjdk.java.net/~naoto/8244459/webrev.01/

+1

-Joe

>
> Naoto
>
>
> On 5/5/20 11:01 AM, naoto.sato at oracle.com wrote:
>> And here is the fix. Please review.
>>
>> http://cr.openjdk.java.net/~naoto/8244459/webrev.00/
>>
>> Naoto
>>
>> On 5/5/20 10:25 AM, naoto.sato at oracle.com wrote:
>>> Hi Peter,
>>>
>>> You are correct. Thanks. I'll remove that initial value of 16.
>>>
>>> Naoto
>>>
>>> On 5/5/20 9:37 AM, Peter Levart wrote:
>>>> Hi Naoto,
>>>>
>>>> On 4/30/20 12:18 AM, naoto.sato at oracle.com wrote:
>>>>> Hello,
>>>>>
>>>>> Please review this small fix to the following issue:
>>>>>
>>>>> https://bugs.openjdk.java.net/browse/JDK-8244152
>>>>>
>>>>> The proposed changeset is located at:
>>>>>
>>>>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>>>>>
>>>>> The hash map used there didn't have initial capacity, even though 
>>>>> the exact numbers are known.
>>>>
>>>>
>>>> Well, it has to be calculated 1st (countTokens), but I guess this 
>>>> pays off when HashSet (the backing HashMap) does not have to be 
>>>> rehashed then.
>>>>
>>>> The expression you use:
>>>>
>>>> ???? Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
>>>>
>>>> ...has a minimum value of 16. Why is that? 16 is just HashMap's 
>>>> default initialCapacity if not specified explicitly. But if you 
>>>> only want to store say 1 entry in the map, you can specify 2 as 
>>>> initialCapacity and HashMap will happily work for such case without 
>>>> resizing.
>>>>
>>>>
>>>> So you could just use:
>>>>
>>>> ???? (int)(tokens.countTokens() / 0.75f) + 1
>>>>
>>>> And even this expression is sometimes overshooting the minimal 
>>>> required value by 1 (when # of tokens is "exact" multiple of 0.75f, 
>>>> say 6). I think the following could be used to optimally pre-size 
>>>> the HashMap with default load factor 0.75:
>>>>
>>>> ???? (tokens.countTokens() * 4 + 2) / 3
>>>>
>>>>
>>>> Regards, Peter
>>>>
>>>>>
>>>>> Naoto


From stuart.marks at oracle.com  Tue May  5 23:29:21 2020
From: stuart.marks at oracle.com (Stuart Marks)
Date: Tue, 5 May 2020 16:29:21 -0700
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
 LocaleProviderAdapters
In-Reply-To: <fecd7cee-31a9-7a1a-4cae-70822501f716@gmail.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
 <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
 <CA+kOe08MbbYsXb9zYJpM3oPoNYJmx3CNsrzz9-Se5RmrGubFmQ@mail.gmail.com>
 <ab4ad543-013b-1261-fa1c-3fc87c21402a@gmail.com>
 <CA+kOe0-YmLOWONqPMbGxKDsA9HMi9SbWZYaCZYD3VjgR4iBMBA@mail.gmail.com>
 <4e994519-c6bd-a6e9-2ef8-1e8be4ef6008@gmail.com>
 <fecd7cee-31a9-7a1a-4cae-70822501f716@gmail.com>
Message-ID: <2aa09b33-19ab-b1bd-83a9-29cd178b7b65@oracle.com>

Hm, interesting, good catch Peter! Very subtle. The time-honored

     (int) (expected / 0.75f) + 1

appears in several places around the JDK. I think most people (including me) 
just copy it, because it's "good enough" for most cases.

I'm a little concerned about

     (expectedSize * 4 + 2) / 3

because that wraps around to negative starting at about 2^29. Might I suggest 
the following instead?

     (int) Math.ceil(expectedSize / 0.75)

This expresses the actual intent more clearly, I think, and its results match 
Peter's expression for the range less than 2^29. It also saturates at 
Integer.MAX_VALUE instead of going negative. It does use double precision, 
though, as there's no float version of ceil(). At this point I think this is a 
small disadvantage.

**

As for Naoto's changeset, I don't think it should be held up while we bikeshed 
this. :-) I'm ok with whatever he decides.

s'marks




On 5/5/20 1:26 PM, Peter Levart wrote:
> There's more...
> 
> 
> Guava (and JDK in copy constructor) formula:
> 
>  ??? (int) ((float) expectedSize / 0.75f + 1.0f)
> 
> 
> is not the same as Naoto's formula:
> 
>  ??? (int) (expectedSize / 0.75f) + 1
> 
> 
> Notice that Naoto does addition of 1 in integer arithmetic after conversion to 
> int, while Guava/JDK does in floating point before conversion to int. If I put 
> Naoto's formula into my test program, no undercalculations are detected.
> 
> 
> So while Naoto's formula is sub-optimal for expectedSizes that are multiples of 
> 3, the Guava/JDK also has a bug.
> 
> 
> Regards, Peter
> 
> 
> On 5/5/20 10:01 PM, Peter Levart wrote:
>>
>>
>> On 5/5/20 9:41 PM, Martin Buchholz wrote:
>>> Hi Peter,
>>>
>>> Are you saying guava has a tiny bug?
>>
>>
>> If it was just 1 too much when expected size is a multiple of 3 then that 
>> would not be a bug, just sub-optimal calculation. And the same calculation is 
>> performed also in JDK when a copy constructor is called for example.
>>
>>
>> But I investigated further and what I found could be considered a bug. 
>> Sometimes, the following expression:
>>
>>
>> (int) ((float) expectedSize / 0.75f + 1.0f)
>>
>>
>> ...calculates a value that is not enough (due to floating point arithmetic and 
>> conversion to int) to store the expectedSize elements into the HashMap without 
>> re-hashing.
>>
>>
>> What HashMap does with initialCapacity parameter is to round it up to nearest 
>> power of 2:
>>
>> ??? static int tableSizeFor(int cap) {
>> ??????? int n = -1 >>> Integer.numberOfLeadingZeros(cap - 1);
>> ??????? return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ? MAXIMUM_CAPACITY : n + 1;
>> ??? }
>>
>> then it uses this as the initial backing table size. From that table size it 
>> calculates the threshold value:
>>
>> ??? static int threshold(int cap) {
>> ??????? float ft = (float) cap * 0.75f;
>> ??????? return (cap < MAXIMUM_CAPACITY && ft < (float) MAXIMUM_CAPACITY ?
>> ??????????????? (int) ft : Integer.MAX_VALUE);
>> ??? }
>>
>> ... and uses it as the max. number of elements that a HashMap can hold before 
>> it is re-hashed. So I did the following test (comparing the effectiveness of 
>> above formula with alternative (expectedSize*4+2)/3 formula):
>>
>>
>> public class HMTest {
>> ??? static final int MAXIMUM_CAPACITY = 1 << 30;
>>
>> ??? static int tableSizeFor(int cap) {
>> ??????? int n = -1 >>> Integer.numberOfLeadingZeros(cap - 1);
>> ??????? return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ? MAXIMUM_CAPACITY : n + 1;
>> ??? }
>>
>> ??? static int threshold(int cap) {
>> ??????? float ft = (float) cap * 0.75f;
>> ??????? return (cap < MAXIMUM_CAPACITY && ft < (float) MAXIMUM_CAPACITY ?
>> ??????????????? (int) ft : Integer.MAX_VALUE);
>> ??? }
>>
>> ??? public static void main(String[] args) {
>> ??????? for (int expectedSize = 0; expectedSize < (Integer.MAX_VALUE - 2) / 4; 
>> expectedSize++) {
>> ??????????? int cap1 = (int) ((float) expectedSize / 0.75f + 1.0f);
>> ??????????? int cap2 = (expectedSize * 4 + 2) / 3;
>> ??????????? int ts1 = tableSizeFor(cap1);
>> ??????????? int ts2 = tableSizeFor(cap2);
>> ??????????? int th1 = threshold(ts1);
>> ??????????? int th2 = threshold(ts2);
>>
>> ??????????? if (th1 < expectedSize || th2 < expectedSize) {
>> ??????????????? System.out.printf("%d: (%d, %d, %d)%s (%d, %d, %d)%s\n",
>> ??????????????????????? expectedSize,
>> ??????????????????????? cap1, ts1, th1, (th1 < expectedSize) ? "!" : " ",
>> ??????????????????????? cap2, ts2, th2, (th2 < expectedSize) ? "!" : " "
>> ??????????????? );
>> ??????????? }
>> ??????? }
>> ??? }
>> }
>>
>>
>> And what this prints is the following:
>>
>>
>> 25165825: (33554432, 33554432, 25165824)! (33554434, 67108864, 50331648)
>> 50331649: (67108864, 67108864, 50331648)! (67108866, 134217728, 100663296)
>> 50331650: (67108864, 67108864, 50331648)! (67108867, 134217728, 100663296)
>> 100663297: (134217728, 134217728, 100663296)! (134217730, 268435456, 201326592)
>> 100663298: (134217728, 134217728, 100663296)! (134217731, 268435456, 201326592)
>> 100663299: (134217728, 134217728, 100663296)! (134217732, 268435456, 201326592)
>> 100663300: (134217728, 134217728, 100663296)! (134217734, 268435456, 201326592)
>> 201326593: (268435456, 268435456, 201326592)! (268435458, 536870912, 402653184)
>> 201326594: (268435456, 268435456, 201326592)! (268435459, 536870912, 402653184)
>> 201326595: (268435456, 268435456, 201326592)! (268435460, 536870912, 402653184)
>> 201326596: (268435456, 268435456, 201326592)! (268435462, 536870912, 402653184)
>> 201326597: (268435456, 268435456, 201326592)! (268435463, 536870912, 402653184)
>> 201326598: (268435456, 268435456, 201326592)! (268435464, 536870912, 402653184)
>> 201326599: (268435456, 268435456, 201326592)! (268435466, 536870912, 402653184)
>> 201326600: (268435456, 268435456, 201326592)! (268435467, 536870912, 402653184)
>> 402653185: (536870912, 536870912, 402653184)! (536870914, 1073741824, 2147483647)
>> 402653186: (536870912, 536870912, 402653184)! (536870915, 1073741824, 2147483647)
>> 402653187: (536870912, 536870912, 402653184)! (536870916, 1073741824, 2147483647)
>> 402653188: (536870912, 536870912, 402653184)! (536870918, 1073741824, 2147483647)
>> 402653189: (536870912, 536870912, 402653184)! (536870919, 1073741824, 2147483647)
>> 402653190: (536870912, 536870912, 402653184)! (536870920, 1073741824, 2147483647)
>> 402653191: (536870912, 536870912, 402653184)! (536870922, 1073741824, 2147483647)
>> 402653192: (536870912, 536870912, 402653184)! (536870923, 1073741824, 2147483647)
>> 402653193: (536870912, 536870912, 402653184)! (536870924, 1073741824, 2147483647)
>> 402653194: (536870912, 536870912, 402653184)! (536870926, 1073741824, 2147483647)
>> 402653195: (536870912, 536870912, 402653184)! (536870927, 1073741824, 2147483647)
>> 402653196: (536870912, 536870912, 402653184)! (536870928, 1073741824, 2147483647)
>> 402653197: (536870912, 536870912, 402653184)! (536870930, 1073741824, 2147483647)
>> 402653198: (536870912, 536870912, 402653184)! (536870931, 1073741824, 2147483647)
>> 402653199: (536870912, 536870912, 402653184)! (536870932, 1073741824, 2147483647)
>> 402653200: (536870912, 536870912, 402653184)! (536870934, 1073741824, 2147483647)
>>
>>
>> So as you see, for expectedSize < (Integer.MAX_VALUE - 2) / 4 (where the 
>> alternative formula does not experience overflow and is enough for Naoto's 
>> case) all miscalculations are due to the JDK/Guava formula which in those 
>> cases calculates a value that is less than alternative formula's value and too 
>> small to adequately pre-size the HashMap table.
>>
>>
>> Voila, we have some bugs to fix or I am doing something wrong here.
>>
>>
>> Regards, Peter
>>
>>
>>>
>>> On Tue, May 5, 2020 at 12:12 PM Peter Levart <peter.levart at gmail.com 
>>> <mailto:peter.levart at gmail.com>> wrote:
>>>
>>> ??? Hi Martin,
>>>
>>> ??? On 5/5/20 8:26 PM, Martin Buchholz wrote:
>>>> ??? See related:
>>>>     
>>>> https://guava.dev/releases/23.0/api/docs/com/google/common/collect/Maps.html#newHashMapWithExpectedSize-int- 
>>>>
>>>
>>>
>>> ??? This is basically the same calculation (or at least gives same
>>> ??? result) as Naoto did (without the max part):
>>>
>>> ??? Naoto: (int)(expectedSize / 0.75f) + 1
>>>
>>> ??? Guava: (int) ((float) expectedSize / 0.75F + 1.0F)
>>>
>>> ??? but in case expectedSize is a multiple of 3, it gives the result
>>> ??? which is 1 more than needed. If what is needed is also a power of
>>> ??? 2, then twice the needed space is allocated in the HashMap
>>> ??? backing table.
>>>
>>>
>>> ??? Regards, Peter
>>>
>>>
>>>>
>>>> ??? On Tue, May 5, 2020 at 11:03 AM <naoto.sato at oracle.com
>>>> ??? <mailto:naoto.sato at oracle.com>> wrote:
>>>>
>>>> ??????? And here is the fix. Please review.
>>>>
>>>> ??????? http://cr.openjdk.java.net/~naoto/8244459/webrev.00/
>>>>
>>>> ??????? Naoto
>>>>
>>>> ??????? On 5/5/20 10:25 AM, naoto.sato at oracle.com
>>>> ??????? <mailto:naoto.sato at oracle.com> wrote:
>>>> ??????? > Hi Peter,
>>>> ??????? >
>>>> ??????? > You are correct. Thanks. I'll remove that initial value of 16.
>>>> ??????? >
>>>> ??????? > Naoto
>>>> ??????? >
>>>> ??????? > On 5/5/20 9:37 AM, Peter Levart wrote:
>>>> ??????? >> Hi Naoto,
>>>> ??????? >>
>>>> ??????? >> On 4/30/20 12:18 AM, naoto.sato at oracle.com
>>>> ??????? <mailto:naoto.sato at oracle.com> wrote:
>>>> ??????? >>> Hello,
>>>> ??????? >>>
>>>> ??????? >>> Please review this small fix to the following issue:
>>>> ??????? >>>
>>>> ??????? >>> https://bugs.openjdk.java.net/browse/JDK-8244152
>>>> ??????? >>>
>>>> ??????? >>> The proposed changeset is located at:
>>>> ??????? >>>
>>>> ??????? >>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>>>> ??????? >>>
>>>> ??????? >>> The hash map used there didn't have initial capacity,
>>>> ??????? even though the
>>>> ??????? >>> exact numbers are known.
>>>> ??????? >>
>>>> ??????? >>
>>>> ??????? >> Well, it has to be calculated 1st (countTokens), but I
>>>> ??????? guess this pays
>>>> ??????? >> off when HashSet (the backing HashMap) does not have to
>>>> ??????? be rehashed then.
>>>> ??????? >>
>>>> ??????? >> The expression you use:
>>>> ??????? >>
>>>> ??????? >> ???? Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
>>>> ??????? >>
>>>> ??????? >> ...has a minimum value of 16. Why is that? 16 is just
>>>> ??????? HashMap's
>>>> ??????? >> default initialCapacity if not specified explicitly. But
>>>> ??????? if you only
>>>> ??????? >> want to store say 1 entry in the map, you can specify 2 as
>>>> ??????? >> initialCapacity and HashMap will happily work for such
>>>> ??????? case without
>>>> ??????? >> resizing.
>>>> ??????? >>
>>>> ??????? >>
>>>> ??????? >> So you could just use:
>>>> ??????? >>
>>>> ??????? >> ???? (int)(tokens.countTokens() / 0.75f) + 1
>>>> ??????? >>
>>>> ??????? >> And even this expression is sometimes overshooting the
>>>> ??????? minimal
>>>> ??????? >> required value by 1 (when # of tokens is "exact" multiple
>>>> ??????? of 0.75f,
>>>> ??????? >> say 6). I think the following could be used to optimally
>>>> ??????? pre-size the
>>>> ??????? >> HashMap with default load factor 0.75:
>>>> ??????? >>
>>>> ??????? >> ???? (tokens.countTokens() * 4 + 2) / 3
>>>> ??????? >>
>>>> ??????? >>
>>>> ??????? >> Regards, Peter
>>>> ??????? >>
>>>> ??????? >>>
>>>> ??????? >>> Naoto
>>>>

From naoto.sato at oracle.com  Wed May  6 00:04:40 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Tue, 5 May 2020 17:04:40 -0700
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
 LocaleProviderAdapters
In-Reply-To: <2aa09b33-19ab-b1bd-83a9-29cd178b7b65@oracle.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
 <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
 <CA+kOe08MbbYsXb9zYJpM3oPoNYJmx3CNsrzz9-Se5RmrGubFmQ@mail.gmail.com>
 <ab4ad543-013b-1261-fa1c-3fc87c21402a@gmail.com>
 <CA+kOe0-YmLOWONqPMbGxKDsA9HMi9SbWZYaCZYD3VjgR4iBMBA@mail.gmail.com>
 <4e994519-c6bd-a6e9-2ef8-1e8be4ef6008@gmail.com>
 <fecd7cee-31a9-7a1a-4cae-70822501f716@gmail.com>
 <2aa09b33-19ab-b1bd-83a9-29cd178b7b65@oracle.com>
Message-ID: <1b2e71f3-52c4-169b-c7eb-36eb165311ad@oracle.com>

Hi Stuart,

On 5/5/20 4:29 PM, Stuart Marks wrote:
> As for Naoto's changeset, I don't think it should be held up while we 
> bikeshed this. :-) I'm ok with whatever he decides.

I don't think the number of language tags exceed 2^29, unless languages 
in the whole universe are counted :-) So I would go with the current 
version.

Naoto

> 
> s'marks
> 
> 
> 
> 
> On 5/5/20 1:26 PM, Peter Levart wrote:
>> There's more...
>>
>>
>> Guava (and JDK in copy constructor) formula:
>>
>> ???? (int) ((float) expectedSize / 0.75f + 1.0f)
>>
>>
>> is not the same as Naoto's formula:
>>
>> ???? (int) (expectedSize / 0.75f) + 1
>>
>>
>> Notice that Naoto does addition of 1 in integer arithmetic after 
>> conversion to int, while Guava/JDK does in floating point before 
>> conversion to int. If I put Naoto's formula into my test program, no 
>> undercalculations are detected.
>>
>>
>> So while Naoto's formula is sub-optimal for expectedSizes that are 
>> multiples of 3, the Guava/JDK also has a bug.
>>
>>
>> Regards, Peter
>>
>>
>> On 5/5/20 10:01 PM, Peter Levart wrote:
>>>
>>>
>>> On 5/5/20 9:41 PM, Martin Buchholz wrote:
>>>> Hi Peter,
>>>>
>>>> Are you saying guava has a tiny bug?
>>>
>>>
>>> If it was just 1 too much when expected size is a multiple of 3 then 
>>> that would not be a bug, just sub-optimal calculation. And the same 
>>> calculation is performed also in JDK when a copy constructor is 
>>> called for example.
>>>
>>>
>>> But I investigated further and what I found could be considered a 
>>> bug. Sometimes, the following expression:
>>>
>>>
>>> (int) ((float) expectedSize / 0.75f + 1.0f)
>>>
>>>
>>> ...calculates a value that is not enough (due to floating point 
>>> arithmetic and conversion to int) to store the expectedSize elements 
>>> into the HashMap without re-hashing.
>>>
>>>
>>> What HashMap does with initialCapacity parameter is to round it up to 
>>> nearest power of 2:
>>>
>>> ??? static int tableSizeFor(int cap) {
>>> ??????? int n = -1 >>> Integer.numberOfLeadingZeros(cap - 1);
>>> ??????? return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ? 
>>> MAXIMUM_CAPACITY : n + 1;
>>> ??? }
>>>
>>> then it uses this as the initial backing table size. From that table 
>>> size it calculates the threshold value:
>>>
>>> ??? static int threshold(int cap) {
>>> ??????? float ft = (float) cap * 0.75f;
>>> ??????? return (cap < MAXIMUM_CAPACITY && ft < (float) 
>>> MAXIMUM_CAPACITY ?
>>> ??????????????? (int) ft : Integer.MAX_VALUE);
>>> ??? }
>>>
>>> ... and uses it as the max. number of elements that a HashMap can 
>>> hold before it is re-hashed. So I did the following test (comparing 
>>> the effectiveness of above formula with alternative 
>>> (expectedSize*4+2)/3 formula):
>>>
>>>
>>> public class HMTest {
>>> ??? static final int MAXIMUM_CAPACITY = 1 << 30;
>>>
>>> ??? static int tableSizeFor(int cap) {
>>> ??????? int n = -1 >>> Integer.numberOfLeadingZeros(cap - 1);
>>> ??????? return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ? 
>>> MAXIMUM_CAPACITY : n + 1;
>>> ??? }
>>>
>>> ??? static int threshold(int cap) {
>>> ??????? float ft = (float) cap * 0.75f;
>>> ??????? return (cap < MAXIMUM_CAPACITY && ft < (float) 
>>> MAXIMUM_CAPACITY ?
>>> ??????????????? (int) ft : Integer.MAX_VALUE);
>>> ??? }
>>>
>>> ??? public static void main(String[] args) {
>>> ??????? for (int expectedSize = 0; expectedSize < (Integer.MAX_VALUE 
>>> - 2) / 4; expectedSize++) {
>>> ??????????? int cap1 = (int) ((float) expectedSize / 0.75f + 1.0f);
>>> ??????????? int cap2 = (expectedSize * 4 + 2) / 3;
>>> ??????????? int ts1 = tableSizeFor(cap1);
>>> ??????????? int ts2 = tableSizeFor(cap2);
>>> ??????????? int th1 = threshold(ts1);
>>> ??????????? int th2 = threshold(ts2);
>>>
>>> ??????????? if (th1 < expectedSize || th2 < expectedSize) {
>>> ??????????????? System.out.printf("%d: (%d, %d, %d)%s (%d, %d, %d)%s\n",
>>> ??????????????????????? expectedSize,
>>> ??????????????????????? cap1, ts1, th1, (th1 < expectedSize) ? "!" : 
>>> " ",
>>> ??????????????????????? cap2, ts2, th2, (th2 < expectedSize) ? "!" : " "
>>> ??????????????? );
>>> ??????????? }
>>> ??????? }
>>> ??? }
>>> }
>>>
>>>
>>> And what this prints is the following:
>>>
>>>
>>> 25165825: (33554432, 33554432, 25165824)! (33554434, 67108864, 50331648)
>>> 50331649: (67108864, 67108864, 50331648)! (67108866, 134217728, 
>>> 100663296)
>>> 50331650: (67108864, 67108864, 50331648)! (67108867, 134217728, 
>>> 100663296)
>>> 100663297: (134217728, 134217728, 100663296)! (134217730, 268435456, 
>>> 201326592)
>>> 100663298: (134217728, 134217728, 100663296)! (134217731, 268435456, 
>>> 201326592)
>>> 100663299: (134217728, 134217728, 100663296)! (134217732, 268435456, 
>>> 201326592)
>>> 100663300: (134217728, 134217728, 100663296)! (134217734, 268435456, 
>>> 201326592)
>>> 201326593: (268435456, 268435456, 201326592)! (268435458, 536870912, 
>>> 402653184)
>>> 201326594: (268435456, 268435456, 201326592)! (268435459, 536870912, 
>>> 402653184)
>>> 201326595: (268435456, 268435456, 201326592)! (268435460, 536870912, 
>>> 402653184)
>>> 201326596: (268435456, 268435456, 201326592)! (268435462, 536870912, 
>>> 402653184)
>>> 201326597: (268435456, 268435456, 201326592)! (268435463, 536870912, 
>>> 402653184)
>>> 201326598: (268435456, 268435456, 201326592)! (268435464, 536870912, 
>>> 402653184)
>>> 201326599: (268435456, 268435456, 201326592)! (268435466, 536870912, 
>>> 402653184)
>>> 201326600: (268435456, 268435456, 201326592)! (268435467, 536870912, 
>>> 402653184)
>>> 402653185: (536870912, 536870912, 402653184)! (536870914, 1073741824, 
>>> 2147483647)
>>> 402653186: (536870912, 536870912, 402653184)! (536870915, 1073741824, 
>>> 2147483647)
>>> 402653187: (536870912, 536870912, 402653184)! (536870916, 1073741824, 
>>> 2147483647)
>>> 402653188: (536870912, 536870912, 402653184)! (536870918, 1073741824, 
>>> 2147483647)
>>> 402653189: (536870912, 536870912, 402653184)! (536870919, 1073741824, 
>>> 2147483647)
>>> 402653190: (536870912, 536870912, 402653184)! (536870920, 1073741824, 
>>> 2147483647)
>>> 402653191: (536870912, 536870912, 402653184)! (536870922, 1073741824, 
>>> 2147483647)
>>> 402653192: (536870912, 536870912, 402653184)! (536870923, 1073741824, 
>>> 2147483647)
>>> 402653193: (536870912, 536870912, 402653184)! (536870924, 1073741824, 
>>> 2147483647)
>>> 402653194: (536870912, 536870912, 402653184)! (536870926, 1073741824, 
>>> 2147483647)
>>> 402653195: (536870912, 536870912, 402653184)! (536870927, 1073741824, 
>>> 2147483647)
>>> 402653196: (536870912, 536870912, 402653184)! (536870928, 1073741824, 
>>> 2147483647)
>>> 402653197: (536870912, 536870912, 402653184)! (536870930, 1073741824, 
>>> 2147483647)
>>> 402653198: (536870912, 536870912, 402653184)! (536870931, 1073741824, 
>>> 2147483647)
>>> 402653199: (536870912, 536870912, 402653184)! (536870932, 1073741824, 
>>> 2147483647)
>>> 402653200: (536870912, 536870912, 402653184)! (536870934, 1073741824, 
>>> 2147483647)
>>>
>>>
>>> So as you see, for expectedSize < (Integer.MAX_VALUE - 2) / 4 (where 
>>> the alternative formula does not experience overflow and is enough 
>>> for Naoto's case) all miscalculations are due to the JDK/Guava 
>>> formula which in those cases calculates a value that is less than 
>>> alternative formula's value and too small to adequately pre-size the 
>>> HashMap table.
>>>
>>>
>>> Voila, we have some bugs to fix or I am doing something wrong here.
>>>
>>>
>>> Regards, Peter
>>>
>>>
>>>>
>>>> On Tue, May 5, 2020 at 12:12 PM Peter Levart <peter.levart at gmail.com 
>>>> <mailto:peter.levart at gmail.com>> wrote:
>>>>
>>>> ??? Hi Martin,
>>>>
>>>> ??? On 5/5/20 8:26 PM, Martin Buchholz wrote:
>>>>> ??? See related:
>>>>> https://guava.dev/releases/23.0/api/docs/com/google/common/collect/Maps.html#newHashMapWithExpectedSize-int- 
>>>>>
>>>>
>>>>
>>>> ??? This is basically the same calculation (or at least gives same
>>>> ??? result) as Naoto did (without the max part):
>>>>
>>>> ??? Naoto: (int)(expectedSize / 0.75f) + 1
>>>>
>>>> ??? Guava: (int) ((float) expectedSize / 0.75F + 1.0F)
>>>>
>>>> ??? but in case expectedSize is a multiple of 3, it gives the result
>>>> ??? which is 1 more than needed. If what is needed is also a power of
>>>> ??? 2, then twice the needed space is allocated in the HashMap
>>>> ??? backing table.
>>>>
>>>>
>>>> ??? Regards, Peter
>>>>
>>>>
>>>>>
>>>>> ??? On Tue, May 5, 2020 at 11:03 AM <naoto.sato at oracle.com
>>>>> ??? <mailto:naoto.sato at oracle.com>> wrote:
>>>>>
>>>>> ??????? And here is the fix. Please review.
>>>>>
>>>>> ??????? http://cr.openjdk.java.net/~naoto/8244459/webrev.00/
>>>>>
>>>>> ??????? Naoto
>>>>>
>>>>> ??????? On 5/5/20 10:25 AM, naoto.sato at oracle.com
>>>>> ??????? <mailto:naoto.sato at oracle.com> wrote:
>>>>> ??????? > Hi Peter,
>>>>> ??????? >
>>>>> ??????? > You are correct. Thanks. I'll remove that initial value 
>>>>> of 16.
>>>>> ??????? >
>>>>> ??????? > Naoto
>>>>> ??????? >
>>>>> ??????? > On 5/5/20 9:37 AM, Peter Levart wrote:
>>>>> ??????? >> Hi Naoto,
>>>>> ??????? >>
>>>>> ??????? >> On 4/30/20 12:18 AM, naoto.sato at oracle.com
>>>>> ??????? <mailto:naoto.sato at oracle.com> wrote:
>>>>> ??????? >>> Hello,
>>>>> ??????? >>>
>>>>> ??????? >>> Please review this small fix to the following issue:
>>>>> ??????? >>>
>>>>> ??????? >>> https://bugs.openjdk.java.net/browse/JDK-8244152
>>>>> ??????? >>>
>>>>> ??????? >>> The proposed changeset is located at:
>>>>> ??????? >>>
>>>>> ??????? >>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
>>>>> ??????? >>>
>>>>> ??????? >>> The hash map used there didn't have initial capacity,
>>>>> ??????? even though the
>>>>> ??????? >>> exact numbers are known.
>>>>> ??????? >>
>>>>> ??????? >>
>>>>> ??????? >> Well, it has to be calculated 1st (countTokens), but I
>>>>> ??????? guess this pays
>>>>> ??????? >> off when HashSet (the backing HashMap) does not have to
>>>>> ??????? be rehashed then.
>>>>> ??????? >>
>>>>> ??????? >> The expression you use:
>>>>> ??????? >>
>>>>> ??????? >> ???? Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
>>>>> ??????? >>
>>>>> ??????? >> ...has a minimum value of 16. Why is that? 16 is just
>>>>> ??????? HashMap's
>>>>> ??????? >> default initialCapacity if not specified explicitly. But
>>>>> ??????? if you only
>>>>> ??????? >> want to store say 1 entry in the map, you can specify 2 as
>>>>> ??????? >> initialCapacity and HashMap will happily work for such
>>>>> ??????? case without
>>>>> ??????? >> resizing.
>>>>> ??????? >>
>>>>> ??????? >>
>>>>> ??????? >> So you could just use:
>>>>> ??????? >>
>>>>> ??????? >> ???? (int)(tokens.countTokens() / 0.75f) + 1
>>>>> ??????? >>
>>>>> ??????? >> And even this expression is sometimes overshooting the
>>>>> ??????? minimal
>>>>> ??????? >> required value by 1 (when # of tokens is "exact" multiple
>>>>> ??????? of 0.75f,
>>>>> ??????? >> say 6). I think the following could be used to optimally
>>>>> ??????? pre-size the
>>>>> ??????? >> HashMap with default load factor 0.75:
>>>>> ??????? >>
>>>>> ??????? >> ???? (tokens.countTokens() * 4 + 2) / 3
>>>>> ??????? >>
>>>>> ??????? >>
>>>>> ??????? >> Regards, Peter
>>>>> ??????? >>
>>>>> ??????? >>>
>>>>> ??????? >>> Naoto
>>>>>

From mark at macchiato.com  Wed May  6 04:02:41 2020
From: mark at macchiato.com (=?UTF-8?B?TWFyayBEYXZpcyDimJXvuI8=?=)
Date: Tue, 5 May 2020 21:02:41 -0700
Subject: <i18n dev> [15] RFR: 8244459: Optimize the hash map size in
	LocaleProviderAdapters
In-Reply-To: <1b2e71f3-52c4-169b-c7eb-36eb165311ad@oracle.com>
References: <e59313a6-6b7e-6085-dc53-d4fe8f460ae4@oracle.com>
 <40815512-2925-1140-e6f7-5aa026f427c5@gmail.com>
 <46715fee-ae7f-d281-37c7-6189004fe621@oracle.com>
 <71a3c4e7-4dd1-ec96-0699-830f1667b9c7@oracle.com>
 <CA+kOe08MbbYsXb9zYJpM3oPoNYJmx3CNsrzz9-Se5RmrGubFmQ@mail.gmail.com>
 <ab4ad543-013b-1261-fa1c-3fc87c21402a@gmail.com>
 <CA+kOe0-YmLOWONqPMbGxKDsA9HMi9SbWZYaCZYD3VjgR4iBMBA@mail.gmail.com>
 <4e994519-c6bd-a6e9-2ef8-1e8be4ef6008@gmail.com>
 <fecd7cee-31a9-7a1a-4cae-70822501f716@gmail.com>
 <2aa09b33-19ab-b1bd-83a9-29cd178b7b65@oracle.com>
 <1b2e71f3-52c4-169b-c7eb-36eb165311ad@oracle.com>
Message-ID: <CAJ2xs_FUDEiRP3uy0DnsWRL=L3qWq9Wqzkyh0wv6bsZ9M=zoeQ@mail.gmail.com>

I wouldn't worry too much about over 2^29 either.

However, the number of possible valid language tags is much larger than
people think, since any combination of multiple variants can occur). So
even not counting the locale extensions, it is over 2^129 (my rough
calculation).

Mark


On Tue, May 5, 2020 at 5:09 PM <naoto.sato at oracle.com> wrote:

> Hi Stuart,
>
> On 5/5/20 4:29 PM, Stuart Marks wrote:
> > As for Naoto's changeset, I don't think it should be held up while we
> > bikeshed this. :-) I'm ok with whatever he decides.
>
> I don't think the number of language tags exceed 2^29, unless languages
> in the whole universe are counted :-) So I would go with the current
> version.
>
> Naoto
>
> >
> > s'marks
> >
> >
> >
> >
> > On 5/5/20 1:26 PM, Peter Levart wrote:
> >> There's more...
> >>
> >>
> >> Guava (and JDK in copy constructor) formula:
> >>
> >>      (int) ((float) expectedSize / 0.75f + 1.0f)
> >>
> >>
> >> is not the same as Naoto's formula:
> >>
> >>      (int) (expectedSize / 0.75f) + 1
> >>
> >>
> >> Notice that Naoto does addition of 1 in integer arithmetic after
> >> conversion to int, while Guava/JDK does in floating point before
> >> conversion to int. If I put Naoto's formula into my test program, no
> >> undercalculations are detected.
> >>
> >>
> >> So while Naoto's formula is sub-optimal for expectedSizes that are
> >> multiples of 3, the Guava/JDK also has a bug.
> >>
> >>
> >> Regards, Peter
> >>
> >>
> >> On 5/5/20 10:01 PM, Peter Levart wrote:
> >>>
> >>>
> >>> On 5/5/20 9:41 PM, Martin Buchholz wrote:
> >>>> Hi Peter,
> >>>>
> >>>> Are you saying guava has a tiny bug?
> >>>
> >>>
> >>> If it was just 1 too much when expected size is a multiple of 3 then
> >>> that would not be a bug, just sub-optimal calculation. And the same
> >>> calculation is performed also in JDK when a copy constructor is
> >>> called for example.
> >>>
> >>>
> >>> But I investigated further and what I found could be considered a
> >>> bug. Sometimes, the following expression:
> >>>
> >>>
> >>> (int) ((float) expectedSize / 0.75f + 1.0f)
> >>>
> >>>
> >>> ...calculates a value that is not enough (due to floating point
> >>> arithmetic and conversion to int) to store the expectedSize elements
> >>> into the HashMap without re-hashing.
> >>>
> >>>
> >>> What HashMap does with initialCapacity parameter is to round it up to
> >>> nearest power of 2:
> >>>
> >>>     static int tableSizeFor(int cap) {
> >>>         int n = -1 >>> Integer.numberOfLeadingZeros(cap - 1);
> >>>         return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ?
> >>> MAXIMUM_CAPACITY : n + 1;
> >>>     }
> >>>
> >>> then it uses this as the initial backing table size. From that table
> >>> size it calculates the threshold value:
> >>>
> >>>     static int threshold(int cap) {
> >>>         float ft = (float) cap * 0.75f;
> >>>         return (cap < MAXIMUM_CAPACITY && ft < (float)
> >>> MAXIMUM_CAPACITY ?
> >>>                 (int) ft : Integer.MAX_VALUE);
> >>>     }
> >>>
> >>> ... and uses it as the max. number of elements that a HashMap can
> >>> hold before it is re-hashed. So I did the following test (comparing
> >>> the effectiveness of above formula with alternative
> >>> (expectedSize*4+2)/3 formula):
> >>>
> >>>
> >>> public class HMTest {
> >>>     static final int MAXIMUM_CAPACITY = 1 << 30;
> >>>
> >>>     static int tableSizeFor(int cap) {
> >>>         int n = -1 >>> Integer.numberOfLeadingZeros(cap - 1);
> >>>         return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ?
> >>> MAXIMUM_CAPACITY : n + 1;
> >>>     }
> >>>
> >>>     static int threshold(int cap) {
> >>>         float ft = (float) cap * 0.75f;
> >>>         return (cap < MAXIMUM_CAPACITY && ft < (float)
> >>> MAXIMUM_CAPACITY ?
> >>>                 (int) ft : Integer.MAX_VALUE);
> >>>     }
> >>>
> >>>     public static void main(String[] args) {
> >>>         for (int expectedSize = 0; expectedSize < (Integer.MAX_VALUE
> >>> - 2) / 4; expectedSize++) {
> >>>             int cap1 = (int) ((float) expectedSize / 0.75f + 1.0f);
> >>>             int cap2 = (expectedSize * 4 + 2) / 3;
> >>>             int ts1 = tableSizeFor(cap1);
> >>>             int ts2 = tableSizeFor(cap2);
> >>>             int th1 = threshold(ts1);
> >>>             int th2 = threshold(ts2);
> >>>
> >>>             if (th1 < expectedSize || th2 < expectedSize) {
> >>>                 System.out.printf("%d: (%d, %d, %d)%s (%d, %d,
> %d)%s\n",
> >>>                         expectedSize,
> >>>                         cap1, ts1, th1, (th1 < expectedSize) ? "!" :
> >>> " ",
> >>>                         cap2, ts2, th2, (th2 < expectedSize) ? "!" : "
> "
> >>>                 );
> >>>             }
> >>>         }
> >>>     }
> >>> }
> >>>
> >>>
> >>> And what this prints is the following:
> >>>
> >>>
> >>> 25165825: (33554432, 33554432, 25165824)! (33554434, 67108864,
> 50331648)
> >>> 50331649: (67108864, 67108864, 50331648)! (67108866, 134217728,
> >>> 100663296)
> >>> 50331650: (67108864, 67108864, 50331648)! (67108867, 134217728,
> >>> 100663296)
> >>> 100663297: (134217728, 134217728, 100663296)! (134217730, 268435456,
> >>> 201326592)
> >>> 100663298: (134217728, 134217728, 100663296)! (134217731, 268435456,
> >>> 201326592)
> >>> 100663299: (134217728, 134217728, 100663296)! (134217732, 268435456,
> >>> 201326592)
> >>> 100663300: (134217728, 134217728, 100663296)! (134217734, 268435456,
> >>> 201326592)
> >>> 201326593: (268435456, 268435456, 201326592)! (268435458, 536870912,
> >>> 402653184)
> >>> 201326594: (268435456, 268435456, 201326592)! (268435459, 536870912,
> >>> 402653184)
> >>> 201326595: (268435456, 268435456, 201326592)! (268435460, 536870912,
> >>> 402653184)
> >>> 201326596: (268435456, 268435456, 201326592)! (268435462, 536870912,
> >>> 402653184)
> >>> 201326597: (268435456, 268435456, 201326592)! (268435463, 536870912,
> >>> 402653184)
> >>> 201326598: (268435456, 268435456, 201326592)! (268435464, 536870912,
> >>> 402653184)
> >>> 201326599: (268435456, 268435456, 201326592)! (268435466, 536870912,
> >>> 402653184)
> >>> 201326600: (268435456, 268435456, 201326592)! (268435467, 536870912,
> >>> 402653184)
> >>> 402653185: (536870912, 536870912, 402653184)! (536870914, 1073741824,
> >>> 2147483647)
> >>> 402653186: (536870912, 536870912, 402653184)! (536870915, 1073741824,
> >>> 2147483647)
> >>> 402653187: (536870912, 536870912, 402653184)! (536870916, 1073741824,
> >>> 2147483647)
> >>> 402653188: (536870912, 536870912, 402653184)! (536870918, 1073741824,
> >>> 2147483647)
> >>> 402653189: (536870912, 536870912, 402653184)! (536870919, 1073741824,
> >>> 2147483647)
> >>> 402653190: (536870912, 536870912, 402653184)! (536870920, 1073741824,
> >>> 2147483647)
> >>> 402653191: (536870912, 536870912, 402653184)! (536870922, 1073741824,
> >>> 2147483647)
> >>> 402653192: (536870912, 536870912, 402653184)! (536870923, 1073741824,
> >>> 2147483647)
> >>> 402653193: (536870912, 536870912, 402653184)! (536870924, 1073741824,
> >>> 2147483647)
> >>> 402653194: (536870912, 536870912, 402653184)! (536870926, 1073741824,
> >>> 2147483647)
> >>> 402653195: (536870912, 536870912, 402653184)! (536870927, 1073741824,
> >>> 2147483647)
> >>> 402653196: (536870912, 536870912, 402653184)! (536870928, 1073741824,
> >>> 2147483647)
> >>> 402653197: (536870912, 536870912, 402653184)! (536870930, 1073741824,
> >>> 2147483647)
> >>> 402653198: (536870912, 536870912, 402653184)! (536870931, 1073741824,
> >>> 2147483647)
> >>> 402653199: (536870912, 536870912, 402653184)! (536870932, 1073741824,
> >>> 2147483647)
> >>> 402653200: (536870912, 536870912, 402653184)! (536870934, 1073741824,
> >>> 2147483647)
> >>>
> >>>
> >>> So as you see, for expectedSize < (Integer.MAX_VALUE - 2) / 4 (where
> >>> the alternative formula does not experience overflow and is enough
> >>> for Naoto's case) all miscalculations are due to the JDK/Guava
> >>> formula which in those cases calculates a value that is less than
> >>> alternative formula's value and too small to adequately pre-size the
> >>> HashMap table.
> >>>
> >>>
> >>> Voila, we have some bugs to fix or I am doing something wrong here.
> >>>
> >>>
> >>> Regards, Peter
> >>>
> >>>
> >>>>
> >>>> On Tue, May 5, 2020 at 12:12 PM Peter Levart <peter.levart at gmail.com
> >>>> <mailto:peter.levart at gmail.com>> wrote:
> >>>>
> >>>>     Hi Martin,
> >>>>
> >>>>     On 5/5/20 8:26 PM, Martin Buchholz wrote:
> >>>>>     See related:
> >>>>>
> https://guava.dev/releases/23.0/api/docs/com/google/common/collect/Maps.html#newHashMapWithExpectedSize-int-
> >>>>>
> >>>>
> >>>>
> >>>>     This is basically the same calculation (or at least gives same
> >>>>     result) as Naoto did (without the max part):
> >>>>
> >>>>     Naoto: (int)(expectedSize / 0.75f) + 1
> >>>>
> >>>>     Guava: (int) ((float) expectedSize / 0.75F + 1.0F)
> >>>>
> >>>>     but in case expectedSize is a multiple of 3, it gives the result
> >>>>     which is 1 more than needed. If what is needed is also a power of
> >>>>     2, then twice the needed space is allocated in the HashMap
> >>>>     backing table.
> >>>>
> >>>>
> >>>>     Regards, Peter
> >>>>
> >>>>
> >>>>>
> >>>>>     On Tue, May 5, 2020 at 11:03 AM <naoto.sato at oracle.com
> >>>>>     <mailto:naoto.sato at oracle.com>> wrote:
> >>>>>
> >>>>>         And here is the fix. Please review.
> >>>>>
> >>>>>         http://cr.openjdk.java.net/~naoto/8244459/webrev.00/
> >>>>>
> >>>>>         Naoto
> >>>>>
> >>>>>         On 5/5/20 10:25 AM, naoto.sato at oracle.com
> >>>>>         <mailto:naoto.sato at oracle.com> wrote:
> >>>>>         > Hi Peter,
> >>>>>         >
> >>>>>         > You are correct. Thanks. I'll remove that initial value
> >>>>> of 16.
> >>>>>         >
> >>>>>         > Naoto
> >>>>>         >
> >>>>>         > On 5/5/20 9:37 AM, Peter Levart wrote:
> >>>>>         >> Hi Naoto,
> >>>>>         >>
> >>>>>         >> On 4/30/20 12:18 AM, naoto.sato at oracle.com
> >>>>>         <mailto:naoto.sato at oracle.com> wrote:
> >>>>>         >>> Hello,
> >>>>>         >>>
> >>>>>         >>> Please review this small fix to the following issue:
> >>>>>         >>>
> >>>>>         >>> https://bugs.openjdk.java.net/browse/JDK-8244152
> >>>>>         >>>
> >>>>>         >>> The proposed changeset is located at:
> >>>>>         >>>
> >>>>>         >>> https://cr.openjdk.java.net/~naoto/8244152/webrev.00/
> >>>>>         >>>
> >>>>>         >>> The hash map used there didn't have initial capacity,
> >>>>>         even though the
> >>>>>         >>> exact numbers are known.
> >>>>>         >>
> >>>>>         >>
> >>>>>         >> Well, it has to be calculated 1st (countTokens), but I
> >>>>>         guess this pays
> >>>>>         >> off when HashSet (the backing HashMap) does not have to
> >>>>>         be rehashed then.
> >>>>>         >>
> >>>>>         >> The expression you use:
> >>>>>         >>
> >>>>>         >>      Math.max((int)(tokens.countTokens() / 0.75f) + 1, 16)
> >>>>>         >>
> >>>>>         >> ...has a minimum value of 16. Why is that? 16 is just
> >>>>>         HashMap's
> >>>>>         >> default initialCapacity if not specified explicitly. But
> >>>>>         if you only
> >>>>>         >> want to store say 1 entry in the map, you can specify 2 as
> >>>>>         >> initialCapacity and HashMap will happily work for such
> >>>>>         case without
> >>>>>         >> resizing.
> >>>>>         >>
> >>>>>         >>
> >>>>>         >> So you could just use:
> >>>>>         >>
> >>>>>         >>      (int)(tokens.countTokens() / 0.75f) + 1
> >>>>>         >>
> >>>>>         >> And even this expression is sometimes overshooting the
> >>>>>         minimal
> >>>>>         >> required value by 1 (when # of tokens is "exact" multiple
> >>>>>         of 0.75f,
> >>>>>         >> say 6). I think the following could be used to optimally
> >>>>>         pre-size the
> >>>>>         >> HashMap with default load factor 0.75:
> >>>>>         >>
> >>>>>         >>      (tokens.countTokens() * 4 + 2) / 3
> >>>>>         >>
> >>>>>         >>
> >>>>>         >> Regards, Peter
> >>>>>         >>
> >>>>>         >>>
> >>>>>         >>> Naoto
> >>>>>
>

From naoto.sato at oracle.com  Wed May  6 20:44:11 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Wed, 6 May 2020 13:44:11 -0700
Subject: <i18n dev> [15]: RFR: 8244245: localizedBy() should override
 localized values with default values
Message-ID: <d15af92c-c9f7-0af2-34c1-263bf95d2a51@oracle.com>

Hello,

Please review the fix to the following issue:

https://bugs.openjdk.java.net/browse/JDK-8244245

The CSR and proposed changeset are located at:

https://bugs.openjdk.java.net/browse/JDK-8244246
https://cr.openjdk.java.net/~naoto/8244245/webrev.00/

This stems from the closed issue 
(https://bugs.openjdk.java.net/browse/JDK-8243162), and the rationale 
for this fix is discussed there.

Naoto


From huizhe.wang at oracle.com  Thu May  7 06:32:20 2020
From: huizhe.wang at oracle.com (Joe Wang)
Date: Wed, 6 May 2020 23:32:20 -0700
Subject: <i18n dev> [15]: RFR: 8244245: localizedBy() should override
 localized values with default values
In-Reply-To: <d15af92c-c9f7-0af2-34c1-263bf95d2a51@oracle.com>
References: <d15af92c-c9f7-0af2-34c1-263bf95d2a51@oracle.com>
Message-ID: <2b681167-85d0-631f-60e2-641a28a0a0a0@oracle.com>

Hi Naoto,

The Javadoc states:
 ??? If the locale contains the "ca" (calendar), "nu" (numbering 
system), "rg" (region override), and/or "tz" (timezone) Unicode 
extensions, the chronology, numbering system and/or the zone are overridden.

If you remove the two statements that check whether the specified locale 
contains "ca" or "nu", would you need to update the Javadoc as well?

Best,
Joe

On 5/6/2020 1:44 PM, naoto.sato at oracle.com wrote:
> Hello,
>
> Please review the fix to the following issue:
>
> https://bugs.openjdk.java.net/browse/JDK-8244245
>
> The CSR and proposed changeset are located at:
>
> https://bugs.openjdk.java.net/browse/JDK-8244246
> https://cr.openjdk.java.net/~naoto/8244245/webrev.00/
>
> This stems from the closed issue 
> (https://bugs.openjdk.java.net/browse/JDK-8243162), and the rationale 
> for this fix is discussed there.
>
> Naoto
>


From Roger.Riggs at oracle.com  Thu May  7 14:43:40 2020
From: Roger.Riggs at oracle.com (Roger Riggs)
Date: Thu, 7 May 2020 10:43:40 -0400
Subject: <i18n dev> [15]: RFR: 8244245: localizedBy() should override
 localized values with default values
In-Reply-To: <d15af92c-c9f7-0af2-34c1-263bf95d2a51@oracle.com>
References: <d15af92c-c9f7-0af2-34c1-263bf95d2a51@oracle.com>
Message-ID: <9edc738f-2da2-98d4-866c-3217461d3cde@oracle.com>

Hi Naoto,

Looks fine with a small source edit below.

TestUnicodeExtension.java: Please wrap the excessively long lines; 
string concatination will put them together for the test.

Thanks, Roger


On 5/6/20 4:44 PM, naoto.sato at oracle.com wrote:
> Hello,
>
> Please review the fix to the following issue:
>
> https://bugs.openjdk.java.net/browse/JDK-8244245
>
> The CSR and proposed changeset are located at:
>
> https://bugs.openjdk.java.net/browse/JDK-8244246
> https://cr.openjdk.java.net/~naoto/8244245/webrev.00/
>
> This stems from the closed issue 
> (https://bugs.openjdk.java.net/browse/JDK-8243162), and the rationale 
> for this fix is discussed there.
>
> Naoto
>


From naoto.sato at oracle.com  Thu May  7 16:03:43 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Thu, 7 May 2020 09:03:43 -0700
Subject: <i18n dev> [15]: RFR: 8244245: localizedBy() should override
 localized values with default values
In-Reply-To: <2b681167-85d0-631f-60e2-641a28a0a0a0@oracle.com>
References: <d15af92c-c9f7-0af2-34c1-263bf95d2a51@oracle.com>
 <2b681167-85d0-631f-60e2-641a28a0a0a0@oracle.com>
Message-ID: <ee6cbf74-4932-48b9-d483-aaa5f783728e@oracle.com>

Hi Joe,

Thank you for the review. The removed check was explicitly avoiding the 
default chrono/number in the locale overriding the current locale 
values, which is exactly what this issue is trying to remove. As Stephen 
wrote in another email, Unicode Extensions are correctly dealt in 
Chronology.ofLocale()/DecimalStyle.of() methods indirectly, so I believe 
no doc change is warranted.

Naoto

On 5/6/20 11:32 PM, Joe Wang wrote:
> Hi Naoto,
> 
> The Javadoc states:
>  ??? If the locale contains the "ca" (calendar), "nu" (numbering 
> system), "rg" (region override), and/or "tz" (timezone) Unicode 
> extensions, the chronology, numbering system and/or the zone are 
> overridden.
> 
> If you remove the two statements that check whether the specified locale 
> contains "ca" or "nu", would you need to update the Javadoc as well?
> 
> Best,
> Joe
> 
> On 5/6/2020 1:44 PM, naoto.sato at oracle.com wrote:
>> Hello,
>>
>> Please review the fix to the following issue:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8244245
>>
>> The CSR and proposed changeset are located at:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8244246
>> https://cr.openjdk.java.net/~naoto/8244245/webrev.00/
>>
>> This stems from the closed issue 
>> (https://bugs.openjdk.java.net/browse/JDK-8243162), and the rationale 
>> for this fix is discussed there.
>>
>> Naoto
>>
> 

From naoto.sato at oracle.com  Thu May  7 16:06:02 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Thu, 7 May 2020 09:06:02 -0700
Subject: <i18n dev> [15]: RFR: 8244245: localizedBy() should override
 localized values with default values
In-Reply-To: <9edc738f-2da2-98d4-866c-3217461d3cde@oracle.com>
References: <d15af92c-c9f7-0af2-34c1-263bf95d2a51@oracle.com>
 <9edc738f-2da2-98d4-866c-3217461d3cde@oracle.com>
Message-ID: <e3f6efc9-2301-42b7-fe59-155aa946eaa5@oracle.com>

Hi Roger,

Thank you for the review. Wrapped the long lines as suggested, along 
with some typo fixes in the comments:

https://cr.openjdk.java.net/~naoto/8244245/webrev.01/

Naoto

On 5/7/20 7:43 AM, Roger Riggs wrote:
> Hi Naoto,
> 
> Looks fine with a small source edit below.
> 
> TestUnicodeExtension.java: Please wrap the excessively long lines; 
> string concatination will put them together for the test.
> 
> Thanks, Roger
> 
> 
> On 5/6/20 4:44 PM, naoto.sato at oracle.com wrote:
>> Hello,
>>
>> Please review the fix to the following issue:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8244245
>>
>> The CSR and proposed changeset are located at:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8244246
>> https://cr.openjdk.java.net/~naoto/8244245/webrev.00/
>>
>> This stems from the closed issue 
>> (https://bugs.openjdk.java.net/browse/JDK-8243162), and the rationale 
>> for this fix is discussed there.
>>
>> Naoto
>>
> 

From Roger.Riggs at oracle.com  Thu May  7 16:07:03 2020
From: Roger.Riggs at oracle.com (Roger Riggs)
Date: Thu, 7 May 2020 12:07:03 -0400
Subject: <i18n dev> [15]: RFR: 8244245: localizedBy() should override
 localized values with default values
In-Reply-To: <e3f6efc9-2301-42b7-fe59-155aa946eaa5@oracle.com>
References: <d15af92c-c9f7-0af2-34c1-263bf95d2a51@oracle.com>
 <9edc738f-2da2-98d4-866c-3217461d3cde@oracle.com>
 <e3f6efc9-2301-42b7-fe59-155aa946eaa5@oracle.com>
Message-ID: <dcbd56dc-e4c5-2ac4-c8bc-18607c4575c3@oracle.com>

Looks good, thanks

On 5/7/20 12:06 PM, naoto.sato at oracle.com wrote:
> Hi Roger,
>
> Thank you for the review. Wrapped the long lines as suggested, along 
> with some typo fixes in the comments:
>
> https://cr.openjdk.java.net/~naoto/8244245/webrev.01/
>
> Naoto
>
> On 5/7/20 7:43 AM, Roger Riggs wrote:
>> Hi Naoto,
>>
>> Looks fine with a small source edit below.
>>
>> TestUnicodeExtension.java: Please wrap the excessively long lines; 
>> string concatination will put them together for the test.
>>
>> Thanks, Roger
>>
>>
>> On 5/6/20 4:44 PM, naoto.sato at oracle.com wrote:
>>> Hello,
>>>
>>> Please review the fix to the following issue:
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8244245
>>>
>>> The CSR and proposed changeset are located at:
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8244246
>>> https://cr.openjdk.java.net/~naoto/8244245/webrev.00/
>>>
>>> This stems from the closed issue 
>>> (https://bugs.openjdk.java.net/browse/JDK-8243162), and the 
>>> rationale for this fix is discussed there.
>>>
>>> Naoto
>>>
>>


From huizhe.wang at oracle.com  Thu May  7 16:18:29 2020
From: huizhe.wang at oracle.com (Joe Wang)
Date: Thu, 7 May 2020 09:18:29 -0700
Subject: <i18n dev> [15]: RFR: 8244245: localizedBy() should override
 localized values with default values
In-Reply-To: <ee6cbf74-4932-48b9-d483-aaa5f783728e@oracle.com>
References: <d15af92c-c9f7-0af2-34c1-263bf95d2a51@oracle.com>
 <2b681167-85d0-631f-60e2-641a28a0a0a0@oracle.com>
 <ee6cbf74-4932-48b9-d483-aaa5f783728e@oracle.com>
Message-ID: <e5643651-2c48-4f0c-ab38-bad9122d4eb6@oracle.com>



On 5/7/2020 9:03 AM, naoto.sato at oracle.com wrote:
> Hi Joe,
>
> Thank you for the review. The removed check was explicitly avoiding 
> the default chrono/number in the locale overriding the current locale 
> values, which is exactly what this issue is trying to remove. As 
> Stephen wrote in another email, Unicode Extensions are correctly dealt 
> in Chronology.ofLocale()/DecimalStyle.of() methods indirectly, so I 
> believe no doc change is warranted.

Ok, thanks for the explanation.

-Joe

>
> Naoto
>
> On 5/6/20 11:32 PM, Joe Wang wrote:
>> Hi Naoto,
>>
>> The Javadoc states:
>> ???? If the locale contains the "ca" (calendar), "nu" (numbering 
>> system), "rg" (region override), and/or "tz" (timezone) Unicode 
>> extensions, the chronology, numbering system and/or the zone are 
>> overridden.
>>
>> If you remove the two statements that check whether the specified 
>> locale contains "ca" or "nu", would you need to update the Javadoc as 
>> well?
>>
>> Best,
>> Joe
>>
>> On 5/6/2020 1:44 PM, naoto.sato at oracle.com wrote:
>>> Hello,
>>>
>>> Please review the fix to the following issue:
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8244245
>>>
>>> The CSR and proposed changeset are located at:
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8244246
>>> https://cr.openjdk.java.net/~naoto/8244245/webrev.00/
>>>
>>> This stems from the closed issue 
>>> (https://bugs.openjdk.java.net/browse/JDK-8243162), and the 
>>> rationale for this fix is discussed there.
>>>
>>> Naoto
>>>
>>


From naoto.sato at oracle.com  Thu May  7 21:59:12 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Thu, 7 May 2020 14:59:12 -0700
Subject: <i18n dev> [15] RFR: 8239383: Support for Unicode 13.0
Message-ID: <e0339fbf-1219-3e4e-4d05-9840c42cc21f@oracle.com>

Hello,

Please review the fix to the following issue:

https://bugs.openjdk.java.net/browse/JDK-8239383

Its CSR and proposed changeset are located at:

https://bugs.openjdk.java.net/browse/JDK-8239504
https://cr.openjdk.java.net/~naoto/8239383/webrev.00/

This updates java.lang.Character class to support Unicode Character 
Database 13.0, as well as incorporating ICU4J version 67.1, which 
corresponds to Unicode 13.0, for upgrading java.text.BiDi/Normalizer 
support. Grapheme support in java.util.regex package is also upgraded to 
13.0 level.

Naoto

From naoto.sato at oracle.com  Wed May 13 15:50:23 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Wed, 13 May 2020 08:50:23 -0700
Subject: <i18n dev> [15] RFR: 8239383: Support for Unicode 13.0
In-Reply-To: <e0339fbf-1219-3e4e-4d05-9840c42cc21f@oracle.com>
References: <e0339fbf-1219-3e4e-4d05-9840c42cc21f@oracle.com>
Message-ID: <102402fa-02c1-fba1-5d48-a6fa76188b87@oracle.com>

Ping.

Naoto

On 5/7/20 2:59 PM, naoto.sato at oracle.com wrote:
> Hello,
> 
> Please review the fix to the following issue:
> 
> https://bugs.openjdk.java.net/browse/JDK-8239383
> 
> Its CSR and proposed changeset are located at:
> 
> https://bugs.openjdk.java.net/browse/JDK-8239504
> https://cr.openjdk.java.net/~naoto/8239383/webrev.00/
> 
> This updates java.lang.Character class to support Unicode Character 
> Database 13.0, as well as incorporating ICU4J version 67.1, which 
> corresponds to Unicode 13.0, for upgrading java.text.BiDi/Normalizer 
> support. Grapheme support in java.util.regex package is also upgraded to 
> 13.0 level.
> 
> Naoto

From naoto.sato at oracle.com  Wed May 20 17:29:48 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Wed, 20 May 2020 10:29:48 -0700
Subject: <i18n dev> [15] RFR: 8245241: Incorrect locale provider preference
	is not logged
Message-ID: <4ab22033-8eda-e96b-3898-0525a43e29fd@oracle.com>

Hello,

Please review the fix to the following issue:

https://bugs.openjdk.java.net/browse/JDK-8245241

The proposed changeset is located at:

https://cr.openjdk.java.net/~naoto/8245241/webrev.00/

Incorrect user-provided provider preference is supposed to be logged. 
However it is not so because it is using PlatformLogger(SurrogateLogger) 
which uses the default logging level. I changed it to use j.l.System's 
logger and bumped it to INFO level (it should notify the user by 
default). By taking this opportunity, I did some clean-up in locale 
provider adapter's logging related code as well.

Naoto

From naoto.sato at oracle.com  Wed May 20 21:31:39 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Wed, 20 May 2020 14:31:39 -0700
Subject: <i18n dev> [15] RFR: 8239480: Support for CLDR version 37
Message-ID: <f5268ec0-a7a9-02c0-95b0-4ccf0d79e7b3@oracle.com>

Hello,

Please review the fix to the following issue:

https://bugs.openjdk.java.net/browse/JDK-8239480

The proposed changeset, and the release note draft are located at:

https://cr.openjdk.java.net/~naoto/8239480/webrev.00/
https://bugs.openjdk.java.net/browse/JDK-8243581

This updates the CLDR locale data to version 37. Although the changeset 
is huge, this mainly consists of the locale data and licenses changes. 
Other than those, only one test case was updated 
(IncludeLocalesPluginTest.java) due to the locale data change.

Naoto

From huizhe.wang at oracle.com  Wed May 20 23:10:13 2020
From: huizhe.wang at oracle.com (Joe Wang)
Date: Wed, 20 May 2020 16:10:13 -0700
Subject: <i18n dev> [15] RFR: 8245241: Incorrect locale provider
 preference is not logged
In-Reply-To: <4ab22033-8eda-e96b-3898-0525a43e29fd@oracle.com>
References: <4ab22033-8eda-e96b-3898-0525a43e29fd@oracle.com>
Message-ID: <c6789ae3-3f7d-b06c-adce-2134683132f9@oracle.com>

Hi Naoto,

I don't seem to see the DateFormat class or the getDateInstance methods 
specify how errors may be handled (or logged). Is that stated somewhere 
else?? In other cases, I see that you've changed it to throw 
ServiceConfigurationError, that looks to me may be better than a log as 
a configuration error? (or specifying wrong provider) sounds to me more 
severe than info.

Of the three HostLocaleProviderAdapterImpl, the one for unix is deleted, 
is there a specific reason?

Best,
Joe

On 5/20/2020 10:29 AM, naoto.sato at oracle.com wrote:
> Hello,
>
> Please review the fix to the following issue:
>
> https://bugs.openjdk.java.net/browse/JDK-8245241
>
> The proposed changeset is located at:
>
> https://cr.openjdk.java.net/~naoto/8245241/webrev.00/
>
> Incorrect user-provided provider preference is supposed to be logged. 
> However it is not so because it is using 
> PlatformLogger(SurrogateLogger) which uses the default logging level. 
> I changed it to use j.l.System's logger and bumped it to INFO level 
> (it should notify the user by default). By taking this opportunity, I 
> did some clean-up in locale provider adapter's logging related code as 
> well.
>
> Naoto


From naoto.sato at oracle.com  Wed May 20 23:49:55 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Wed, 20 May 2020 16:49:55 -0700
Subject: <i18n dev> [15] RFR: 8245241: Incorrect locale provider
 preference is not logged
In-Reply-To: <c6789ae3-3f7d-b06c-adce-2134683132f9@oracle.com>
References: <4ab22033-8eda-e96b-3898-0525a43e29fd@oracle.com>
 <c6789ae3-3f7d-b06c-adce-2134683132f9@oracle.com>
Message-ID: <72d71433-1c05-1c10-bd08-8e7b28384dfd@oracle.com>

Hi Joe, thanks for the review.

On 5/20/20 4:10 PM, Joe Wang wrote:
> Hi Naoto,
> 
> I don't seem to see the DateFormat class or the getDateInstance methods 
> specify how errors may be handled (or logged). Is that stated somewhere 
> else?

Description of the system property "java.locale.providers" is in the 
class description of java.util.spi.LocaleServiceProvider class, in which 
the possible provider names can only be CLDR/COMPAT/SPI/HOST/JRE. So 
possible error with this system property is user's typo of the provider 
names. Although the behavior is not described, wrong names have been 
simply ignored at runtime. The problem here is that user cannot tell 
that he's done typo, and this fix is exactly to address it.

>? In other cases, I see that you've changed it to throw 
> ServiceConfigurationError, that looks to me may be better than a log as 
> a configuration error? (or specifying wrong provider) sounds to me more 
> severe than info.

Those exceptions will never happen in normal situation, since those 
locations are loading SPI/HostLocaleProviderAdapter class(es) that are 
the JDK classes (classes/methods are known to exist).
On the other hand, I kept exception handlers in LocaleProviderAdapter to 
generate Level.INFO log, as this is for the user's typo of the provider 
names (explained above). Again this should be ignored and 
replaced/continue with proper default behavior, and letting user know 
the typo. If this were throwing SCError, it would break compatibility.

> 
> Of the three HostLocaleProviderAdapterImpl, the one for unix is deleted, 
> is there a specific reason?

Because it is simply not necessary (empty class). It was meant to be 
implemented later, but the host provider on Unixes has never been 
requested. Probably because OS date/time/number settings in Unixes are 
less important to Java clients.

HTH,
Naoto

> 
> Best,
> Joe
> 
> On 5/20/2020 10:29 AM, naoto.sato at oracle.com wrote:
>> Hello,
>>
>> Please review the fix to the following issue:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8245241
>>
>> The proposed changeset is located at:
>>
>> https://cr.openjdk.java.net/~naoto/8245241/webrev.00/
>>
>> Incorrect user-provided provider preference is supposed to be logged. 
>> However it is not so because it is using 
>> PlatformLogger(SurrogateLogger) which uses the default logging level. 
>> I changed it to use j.l.System's logger and bumped it to INFO level 
>> (it should notify the user by default). By taking this opportunity, I 
>> did some clean-up in locale provider adapter's logging related code as 
>> well.
>>
>> Naoto
> 

From huizhe.wang at oracle.com  Thu May 21 01:09:20 2020
From: huizhe.wang at oracle.com (Joe Wang)
Date: Wed, 20 May 2020 18:09:20 -0700
Subject: <i18n dev> [15] RFR: 8245241: Incorrect locale provider
 preference is not logged
In-Reply-To: <72d71433-1c05-1c10-bd08-8e7b28384dfd@oracle.com>
References: <4ab22033-8eda-e96b-3898-0525a43e29fd@oracle.com>
 <c6789ae3-3f7d-b06c-adce-2134683132f9@oracle.com>
 <72d71433-1c05-1c10-bd08-8e7b28384dfd@oracle.com>
Message-ID: <adbd7959-f05e-de0b-c3ab-980bbc710ad1@oracle.com>

Hi Naoto,

Thanks for the explanation. I agree on the needs for maintaining 
backward compatibility.? The patch looks good me then.

Best,
Joe

On 5/20/2020 4:49 PM, naoto.sato at oracle.com wrote:
> Hi Joe, thanks for the review.
>
> On 5/20/20 4:10 PM, Joe Wang wrote:
>> Hi Naoto,
>>
>> I don't seem to see the DateFormat class or the getDateInstance 
>> methods specify how errors may be handled (or logged). Is that stated 
>> somewhere else?
>
> Description of the system property "java.locale.providers" is in the 
> class description of java.util.spi.LocaleServiceProvider class, in 
> which the possible provider names can only be 
> CLDR/COMPAT/SPI/HOST/JRE. So possible error with this system property 
> is user's typo of the provider names. Although the behavior is not 
> described, wrong names have been simply ignored at runtime. The 
> problem here is that user cannot tell that he's done typo, and this 
> fix is exactly to address it.
>
>> ? In other cases, I see that you've changed it to throw 
>> ServiceConfigurationError, that looks to me may be better than a log 
>> as a configuration error? (or specifying wrong provider) sounds to me 
>> more severe than info.
>
> Those exceptions will never happen in normal situation, since those 
> locations are loading SPI/HostLocaleProviderAdapter class(es) that are 
> the JDK classes (classes/methods are known to exist).
> On the other hand, I kept exception handlers in LocaleProviderAdapter 
> to generate Level.INFO log, as this is for the user's typo of the 
> provider names (explained above). Again this should be ignored and 
> replaced/continue with proper default behavior, and letting user know 
> the typo. If this were throwing SCError, it would break compatibility.
>
>>
>> Of the three HostLocaleProviderAdapterImpl, the one for unix is 
>> deleted, is there a specific reason?
>
> Because it is simply not necessary (empty class). It was meant to be 
> implemented later, but the host provider on Unixes has never been 
> requested. Probably because OS date/time/number settings in Unixes are 
> less important to Java clients.
>
> HTH,
> Naoto
>
>>
>> Best,
>> Joe
>>
>> On 5/20/2020 10:29 AM, naoto.sato at oracle.com wrote:
>>> Hello,
>>>
>>> Please review the fix to the following issue:
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8245241
>>>
>>> The proposed changeset is located at:
>>>
>>> https://cr.openjdk.java.net/~naoto/8245241/webrev.00/
>>>
>>> Incorrect user-provided provider preference is supposed to be 
>>> logged. However it is not so because it is using 
>>> PlatformLogger(SurrogateLogger) which uses the default logging 
>>> level. I changed it to use j.l.System's logger and bumped it to INFO 
>>> level (it should notify the user by default). By taking this 
>>> opportunity, I did some clean-up in locale provider adapter's 
>>> logging related code as well.
>>>
>>> Naoto
>>


From huizhe.wang at oracle.com  Thu May 21 01:51:38 2020
From: huizhe.wang at oracle.com (Joe Wang)
Date: Wed, 20 May 2020 18:51:38 -0700
Subject: <i18n dev> [15] RFR: 8239480: Support for CLDR version 37
In-Reply-To: <f5268ec0-a7a9-02c0-95b0-4ccf0d79e7b3@oracle.com>
References: <f5268ec0-a7a9-02c0-95b0-4ccf0d79e7b3@oracle.com>
Message-ID: <60df81f9-0b16-9739-6817-55bfefef76a3@oracle.com>

Looks good to me.

A minor comment: in the license files, Terms of Use lost its original 
format <https://www.unicode.org/copyright.html> (e.g. headings and 
numbering), lines are also super long. But I think we've seen them in 
the previous update, it's up to you whether you'd want to reformat them. 
Either way, no need to regenerate the huge webrev :-)

Best,
Joe

On 5/20/2020 2:31 PM, naoto.sato at oracle.com wrote:
> Hello,
>
> Please review the fix to the following issue:
>
> https://bugs.openjdk.java.net/browse/JDK-8239480
>
> The proposed changeset, and the release note draft are located at:
>
> https://cr.openjdk.java.net/~naoto/8239480/webrev.00/
> https://bugs.openjdk.java.net/browse/JDK-8243581
>
> This updates the CLDR locale data to version 37. Although the 
> changeset is huge, this mainly consists of the locale data and 
> licenses changes. Other than those, only one test case was updated 
> (IncludeLocalesPluginTest.java) due to the locale data change.
>
> Naoto


From naoto.sato at oracle.com  Thu May 21 02:06:03 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Wed, 20 May 2020 19:06:03 -0700
Subject: <i18n dev> [15] RFR: 8239480: Support for CLDR version 37
In-Reply-To: <60df81f9-0b16-9739-6817-55bfefef76a3@oracle.com>
References: <f5268ec0-a7a9-02c0-95b0-4ccf0d79e7b3@oracle.com>
 <60df81f9-0b16-9739-6817-55bfefef76a3@oracle.com>
Message-ID: <a57c2302-a148-761a-bf25-b59db86ac80f@oracle.com>

Hi Joe, thanks again!

The license file "unicode-license.txt" is actually included in the 
version 37 artifact:

https://github.com/unicode-org/cldr/blob/release-37/unicode-license.txt

The changeset uses the file as is, so I believe it is OK.

Naoto

On 5/20/20 6:51 PM, Joe Wang wrote:
> Looks good to me.
> 
> A minor comment: in the license files, Terms of Use lost its original 
> format <https://www.unicode.org/copyright.html> (e.g. headings and 
> numbering), lines are also super long. But I think we've seen them in 
> the previous update, it's up to you whether you'd want to reformat them. 
> Either way, no need to regenerate the huge webrev :-)
> 
> Best,
> Joe
> 
> On 5/20/2020 2:31 PM, naoto.sato at oracle.com wrote:
>> Hello,
>>
>> Please review the fix to the following issue:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8239480
>>
>> The proposed changeset, and the release note draft are located at:
>>
>> https://cr.openjdk.java.net/~naoto/8239480/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8243581
>>
>> This updates the CLDR locale data to version 37. Although the 
>> changeset is huge, this mainly consists of the locale data and 
>> licenses changes. Other than those, only one test case was updated 
>> (IncludeLocalesPluginTest.java) due to the locale data change.
>>
>> Naoto
> 

From huizhe.wang at oracle.com  Thu May 21 05:56:19 2020
From: huizhe.wang at oracle.com (Joe Wang)
Date: Wed, 20 May 2020 22:56:19 -0700
Subject: <i18n dev> [15] RFR: 8239480: Support for CLDR version 37
In-Reply-To: <a57c2302-a148-761a-bf25-b59db86ac80f@oracle.com>
References: <f5268ec0-a7a9-02c0-95b0-4ccf0d79e7b3@oracle.com>
 <60df81f9-0b16-9739-6817-55bfefef76a3@oracle.com>
 <a57c2302-a148-761a-bf25-b59db86ac80f@oracle.com>
Message-ID: <1e195b3d-fa14-ed44-52d8-1d4695527bb0@oracle.com>



On 5/20/2020 7:06 PM, naoto.sato at oracle.com wrote:
> Hi Joe, thanks again!

No problem.
>
> The license file "unicode-license.txt" is actually included in the 
> version 37 artifact:
>
> https://github.com/unicode-org/cldr/blob/release-37/unicode-license.txt
>
> The changeset uses the file as is, so I believe it is OK.

I meant the md files that include the license text with the additional 
"Terms of Use", starting from line 55. That looks like a copy of this: 
https://www.unicode.org/copyright.html, but headings and numbering were 
lost. Plus, without html format, some of the lines stretched very long 
<https://cr.openjdk.java.net/~naoto/8239480/webrev.00/src/jdk.localedata/share/legal/cldr.md.html>. 
But again, the format of the md files might not be that important. Your 
call.

-Joe

>
> Naoto
>
> On 5/20/20 6:51 PM, Joe Wang wrote:
>> Looks good to me.
>>
>> A minor comment: in the license files, Terms of Use lost its original 
>> format <https://www.unicode.org/copyright.html> (e.g. headings and 
>> numbering), lines are also super long. But I think we've seen them in 
>> the previous update, it's up to you whether you'd want to reformat 
>> them. Either way, no need to regenerate the huge webrev :-)
>>
>> Best,
>> Joe
>>
>> On 5/20/2020 2:31 PM, naoto.sato at oracle.com wrote:
>>> Hello,
>>>
>>> Please review the fix to the following issue:
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8239480
>>>
>>> The proposed changeset, and the release note draft are located at:
>>>
>>> https://cr.openjdk.java.net/~naoto/8239480/webrev.00/
>>> https://bugs.openjdk.java.net/browse/JDK-8243581
>>>
>>> This updates the CLDR locale data to version 37. Although the 
>>> changeset is huge, this mainly consists of the locale data and 
>>> licenses changes. Other than those, only one test case was updated 
>>> (IncludeLocalesPluginTest.java) due to the locale data change.
>>>
>>> Naoto
>>


From daniel.fuchs at oracle.com  Thu May 21 10:42:52 2020
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Thu, 21 May 2020 11:42:52 +0100
Subject: <i18n dev> [15] RFR: 8245241: Incorrect locale provider
 preference is not logged
In-Reply-To: <4ab22033-8eda-e96b-3898-0525a43e29fd@oracle.com>
References: <4ab22033-8eda-e96b-3898-0525a43e29fd@oracle.com>
Message-ID: <7f4db42f-1873-3d08-2537-b20279c74ab0@oracle.com>

Hi Naoto,

Logging uses:

ZonedDateTime zdt = ZonedDateTime.ofInstant(
                 record.getInstant(), ZoneId.systemDefault());

and then

String.format("%1$tb %1$td, %1$tY %1$tl:%1$tM:%1$tS %1$Tp %2$s%n%4$s: 
%5$s%6$s%n", ...)

to format the date.

If the locale provider can't be loaded, is that going to cause problem,
when logging is invoked to log the info/warning?

In particular:
java.base/share/classes/sun/util/locale/provider/LocaleProviderAdapter.java

  197 
getLogger(LocaleProviderAdapter.class.getCanonicalName())
  198                             .log(Logger.Level.INFO, e);
  199                     adapterInstances.putIfAbsent(type, 
NONEXISTENT_ADAPTER);
  200                     if (defaultLocaleProviderAdapter == type) {
  201                         defaultLocaleProviderAdapter = Type.FALLBACK;
  202                     }

Maybe logging should only be invoked *after* lines 199-202?

Same potential issue in
src/java.base/share/classes/sun/util/locale/provider/LocaleServiceProviderPool.java

lines 367-372

best regards,

-- daniel

On 20/05/2020 18:29, naoto.sato at oracle.com wrote:
> Hello,
> 
> Please review the fix to the following issue:
> 
> https://bugs.openjdk.java.net/browse/JDK-8245241
> 
> The proposed changeset is located at:
> 
> https://cr.openjdk.java.net/~naoto/8245241/webrev.00/
> 
> Incorrect user-provided provider preference is supposed to be logged. 
> However it is not so because it is using PlatformLogger(SurrogateLogger) 
> which uses the default logging level. I changed it to use j.l.System's 
> logger and bumped it to INFO level (it should notify the user by 
> default). By taking this opportunity, I did some clean-up in locale 
> provider adapter's logging related code as well.
> 
> Naoto


From naoto.sato at oracle.com  Thu May 21 19:00:42 2020
From: naoto.sato at oracle.com (naoto.sato at oracle.com)
Date: Thu, 21 May 2020 12:00:42 -0700
Subject: <i18n dev> [15] RFR: 8245241: Incorrect locale provider
 preference is not logged
In-Reply-To: <7f4db42f-1873-3d08-2537-b20279c74ab0@oracle.com>
References: <4ab22033-8eda-e96b-3898-0525a43e29fd@oracle.com>
 <7f4db42f-1873-3d08-2537-b20279c74ab0@oracle.com>
Message-ID: <2714c058-ba41-2f26-52fa-f2b5b8eef382@oracle.com>

Hi Daniel,

Thank you for your review!

On 5/21/20 3:42 AM, Daniel Fuchs wrote:
> Hi Naoto,
> 
> Logging uses:
> 
> ZonedDateTime zdt = ZonedDateTime.ofInstant(
>  ??????????????? record.getInstant(), ZoneId.systemDefault());
> 
> and then
> 
> String.format("%1$tb %1$td, %1$tY %1$tl:%1$tM:%1$tS %1$Tp %2$s%n%4$s: 
> %5$s%6$s%n", ...)
> 
> to format the date.
> 
> If the locale provider can't be loaded, is that going to cause problem,
> when logging is invoked to log the info/warning?
> 
> In particular:
> java.base/share/classes/sun/util/locale/provider/LocaleProviderAdapter.java
> 
>  ?197 getLogger(LocaleProviderAdapter.class.getCanonicalName())
>  ?198???????????????????????????? .log(Logger.Level.INFO, e);
>  ?199???????????????????? adapterInstances.putIfAbsent(type, 
> NONEXISTENT_ADAPTER);
>  ?200???????????????????? if (defaultLocaleProviderAdapter == type) {
>  ?201???????????????????????? defaultLocaleProviderAdapter = Type.FALLBACK;
>  ?202???????????????????? }
> 
> Maybe logging should only be invoked *after* lines 199-202?

In fact, this piece of code should not happen as those adapter classes 
are all JDK provided classes. I replaced the above code with 
ServiceConfigurationError, like other similar locations (e.g, 
HostLocaleProviderAdapter - line 64-65). Updated the webrev:

http://cr.openjdk.java.net/~naoto/8245241/webrev.01/

> 
> Same potential issue in
> src/java.base/share/classes/sun/util/locale/provider/LocaleServiceProviderPool.java
> 
> 
> lines 367-372

This should be fine, as LocaleProviderAdapter should properly have been 
initialized at this point.

Naoto

> 
> best regards,
> 
> -- daniel
> 
> On 20/05/2020 18:29, naoto.sato at oracle.com wrote:
>> Hello,
>>
>> Please review the fix to the following issue:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8245241
>>
>> The proposed changeset is located at:
>>
>> https://cr.openjdk.java.net/~naoto/8245241/webrev.00/
>>
>> Incorrect user-provided provider preference is supposed to be logged. 
>> However it is not so because it is using 
>> PlatformLogger(SurrogateLogger) which uses the default logging level. 
>> I changed it to use j.l.System's logger and bumped it to INFO level 
>> (it should notify the user by default). By taking this opportunity, I 
>> did some clean-up in locale provider adapter's logging related code as 
>> well.
>>
>> Naoto
> 

From daniel.fuchs at oracle.com  Thu May 21 20:00:16 2020
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Thu, 21 May 2020 21:00:16 +0100
Subject: <i18n dev> [15] RFR: 8245241: Incorrect locale provider
 preference is not logged
In-Reply-To: <2714c058-ba41-2f26-52fa-f2b5b8eef382@oracle.com>
References: <4ab22033-8eda-e96b-3898-0525a43e29fd@oracle.com>
 <7f4db42f-1873-3d08-2537-b20279c74ab0@oracle.com>
 <2714c058-ba41-2f26-52fa-f2b5b8eef382@oracle.com>
Message-ID: <b62b3167-e717-8576-8d42-ee7ec7300493@oracle.com>

Hi Naoto,

On 21/05/2020 20:00, naoto.sato at oracle.com wrote:
> In fact, this piece of code should not happen as those adapter classes 
> are all JDK provided classes. I replaced the above code with 
> ServiceConfigurationError, like other similar locations (e.g, 
> HostLocaleProviderAdapter - line 64-65). Updated the webrev:
> 
> http://cr.openjdk.java.net/~naoto/8245241/webrev.01/
> 

That addresses my concern.

Thanks!

best regards,

-- daniel