From kevinw at openjdk.org  Tue Apr  1 10:39:28 2025
From: kevinw at openjdk.org (Kevin Walls)
Date: Tue, 1 Apr 2025 10:39:28 GMT
Subject: jmx-dev RFR: 8353231: Test
 com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad still fails
 intermittently
Message-ID: <PIhaXE0X5mOumCzQefIhCwiC-BZ-C3lAZPOknY0Vaac=.077220d2-7b53-47bb-aad1-0a42d0a664ba@github.com>

Test com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad still fails intermittently.
On failure, 10 attempts with sleep(200) each time, only read -1 from mbean.getProcessCpuLoad().
The method is documented to return -1 when info is not available, but want to avoid the test accepting a -1 and masking real problems.

Test failures are happening when multiple CPU load reding tests ran on the same host, at the same second.
Add a TEST.properties file containing: exclusiveAccess.dirs=.

-------------

Commit messages:
 - 8353231: Test com/sun/management/OperatingSystemMXBean cpuLoad still fails intermittently

Changes: https://git.openjdk.org/jdk/pull/24352/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=24352&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8353231
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/24352.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/24352/head:pull/24352

PR: https://git.openjdk.org/jdk/pull/24352

From dholmes at openjdk.org  Mon Apr  7 07:39:54 2025
From: dholmes at openjdk.org (David Holmes)
Date: Mon, 7 Apr 2025 07:39:54 GMT
Subject: jmx-dev RFR: 8353231: Test
 com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad still fails
 intermittently
In-Reply-To: <PIhaXE0X5mOumCzQefIhCwiC-BZ-C3lAZPOknY0Vaac=.077220d2-7b53-47bb-aad1-0a42d0a664ba@github.com>
References: <PIhaXE0X5mOumCzQefIhCwiC-BZ-C3lAZPOknY0Vaac=.077220d2-7b53-47bb-aad1-0a42d0a664ba@github.com>
Message-ID: <igVb1Z-pPZqIy-PDGAjOcfzGqF5Z5H18X6ZjE_ET0B0=.a32a7260-e715-4a15-b742-48265abfb26b@github.com>

On Tue, 1 Apr 2025 10:27:39 GMT, Kevin Walls <kevinw at openjdk.org> wrote:

> Test com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad still fails intermittently.
> On failure, 10 attempts with sleep(200) each time, only read -1 from mbean.getProcessCpuLoad().
> The method is documented to return -1 when info is not available, but want to avoid the test accepting a -1 and masking real problems.
> 
> Test failures are happening when multiple CPU load reding tests ran on the same host, at the same second.
> Add a TEST.properties file containing: exclusiveAccess.dirs=.

Okay lets give this a try.

Thanks

-------------

Marked as reviewed by dholmes (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/24352#pullrequestreview-2745797954

From kevinw at openjdk.org  Mon Apr  7 11:36:55 2025
From: kevinw at openjdk.org (Kevin Walls)
Date: Mon, 7 Apr 2025 11:36:55 GMT
Subject: jmx-dev Integrated: 8353231: Test
 com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad still fails
 intermittently
In-Reply-To: <PIhaXE0X5mOumCzQefIhCwiC-BZ-C3lAZPOknY0Vaac=.077220d2-7b53-47bb-aad1-0a42d0a664ba@github.com>
References: <PIhaXE0X5mOumCzQefIhCwiC-BZ-C3lAZPOknY0Vaac=.077220d2-7b53-47bb-aad1-0a42d0a664ba@github.com>
Message-ID: <V_ARdTAlMs3I4dlAmat9_bLYdUlPh_coT8GFch3YSnw=.a286ae4a-3982-4bde-81dc-59e0f90e2bbb@github.com>

On Tue, 1 Apr 2025 10:27:39 GMT, Kevin Walls <kevinw at openjdk.org> wrote:

> Test com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad still fails intermittently.
> On failure, 10 attempts with sleep(200) each time, only read -1 from mbean.getProcessCpuLoad().
> The method is documented to return -1 when info is not available, but want to avoid the test accepting a -1 and masking real problems.
> 
> Test failures are happening when multiple CPU load reding tests ran on the same host, at the same second.
> Add a TEST.properties file containing: exclusiveAccess.dirs=.

This pull request has now been integrated.

Changeset: e8c9e5c6
Author:    Kevin Walls <kevinw at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/e8c9e5c6cd3c844765c27c068022a018914fdf4e
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

8353231: Test com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad still fails intermittently

Reviewed-by: dholmes

-------------

PR: https://git.openjdk.org/jdk/pull/24352

From kevinw at openjdk.org  Mon Apr  7 11:36:55 2025
From: kevinw at openjdk.org (Kevin Walls)
Date: Mon, 7 Apr 2025 11:36:55 GMT
Subject: jmx-dev RFR: 8353231: Test
 com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad still fails
 intermittently
In-Reply-To: <PIhaXE0X5mOumCzQefIhCwiC-BZ-C3lAZPOknY0Vaac=.077220d2-7b53-47bb-aad1-0a42d0a664ba@github.com>
References: <PIhaXE0X5mOumCzQefIhCwiC-BZ-C3lAZPOknY0Vaac=.077220d2-7b53-47bb-aad1-0a42d0a664ba@github.com>
Message-ID: <yaqEkj2Fk1u956WX3ypqBrIKv8NwV_65JsvhS11EaN4=.65028f30-0db3-484c-a409-e3408e3ea07c@github.com>

On Tue, 1 Apr 2025 10:27:39 GMT, Kevin Walls <kevinw at openjdk.org> wrote:

> Test com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad still fails intermittently.
> On failure, 10 attempts with sleep(200) each time, only read -1 from mbean.getProcessCpuLoad().
> The method is documented to return -1 when info is not available, but want to avoid the test accepting a -1 and masking real problems.
> 
> Test failures are happening when multiple CPU load reding tests ran on the same host, at the same second.
> Add a TEST.properties file containing: exclusiveAccess.dirs=.

Thanks David!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/24352#issuecomment-2783014959

From ihse at openjdk.org  Wed Apr  9 15:09:58 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Wed, 9 Apr 2025 15:09:58 GMT
Subject: jmx-dev RFR: 8301991: Convert l10n properties resource bundles
 to UTF-8 native [v6]
In-Reply-To: <cqbVzCGKX2oiVnvlF5eU47L-vXRpPXyuI7DK6Eu9Nf0=.7375f607-3413-4171-a51b-b841908e0684@github.com>
References: <0MB7FLFNfaGEWssr9X54UJ_iZNFWBJkxQ1yusP7fsuY=.3f9f3de5-fe84-48e6-9449-626cac42da0b@github.com>
 <cqbVzCGKX2oiVnvlF5eU47L-vXRpPXyuI7DK6Eu9Nf0=.7375f607-3413-4171-a51b-b841908e0684@github.com>
Message-ID: <_YOUyzMbSEXFduCKVgyis37kwTlGSjBbP8VlFu3xQpU=.9b668e2a-8f91-476d-8914-13dc33a0b9e5@github.com>

On Thu, 11 May 2023 20:21:57 GMT, Justin Lu <jlu at openjdk.org> wrote:

>> This PR converts Unicode sequences to UTF-8 native in .properties file. (Excluding the Unicode space and tab sequence). The conversion was done using native2ascii.
>> 
>> In addition, the build logic is adjusted to support reading in the .properties files as UTF-8 during the conversion from .properties file to .java ListResourceBundle file.
>
> Justin Lu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits:
> 
>  - Convert the merged master changes to UTF-8
>  - Merge master and fix conflicts
>  - Close streams when finished loading into props
>  - Adjust CF test to read in with UTF-8 to fix failing test
>  - Reconvert CS.properties to UTF-8
>  - Revert all changes to CurrencySymbols.properties
>  - Bug6204853 should not be converted
>  - Copyright year for CompileProperties
>  - Redo translation for CS.properties
>  - Spot convert CurrencySymbols.properties
>  - ... and 6 more: https://git.openjdk.org/jdk/compare/4386d42d...f15b373a

src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties line 22:

> 20: # Peter Smolik
> 21: Cp1250 WINDOWS-1250 0x00FF
> 22: # Patch attributed to havardw at underdusken.no (H?vard Wigtil)

This does not seem to have been a correct conversion.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/12726#discussion_r2035582242

From jlu at openjdk.org  Wed Apr  9 21:28:41 2025
From: jlu at openjdk.org (Justin Lu)
Date: Wed, 9 Apr 2025 21:28:41 GMT
Subject: jmx-dev RFR: 8301991: Convert l10n properties resource bundles
 to UTF-8 native [v6]
In-Reply-To: <_YOUyzMbSEXFduCKVgyis37kwTlGSjBbP8VlFu3xQpU=.9b668e2a-8f91-476d-8914-13dc33a0b9e5@github.com>
References: <0MB7FLFNfaGEWssr9X54UJ_iZNFWBJkxQ1yusP7fsuY=.3f9f3de5-fe84-48e6-9449-626cac42da0b@github.com>
 <cqbVzCGKX2oiVnvlF5eU47L-vXRpPXyuI7DK6Eu9Nf0=.7375f607-3413-4171-a51b-b841908e0684@github.com>
 <_YOUyzMbSEXFduCKVgyis37kwTlGSjBbP8VlFu3xQpU=.9b668e2a-8f91-476d-8914-13dc33a0b9e5@github.com>
Message-ID: <f565K9bPPbDC3a6M34NDH-DLKzxunYzBaSTkcOAF8bM=.0d00299f-48f7-43ef-8b70-f037bd1441c1@github.com>

On Wed, 9 Apr 2025 15:06:32 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

>> Justin Lu has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits:
>> 
>>  - Convert the merged master changes to UTF-8
>>  - Merge master and fix conflicts
>>  - Close streams when finished loading into props
>>  - Adjust CF test to read in with UTF-8 to fix failing test
>>  - Reconvert CS.properties to UTF-8
>>  - Revert all changes to CurrencySymbols.properties
>>  - Bug6204853 should not be converted
>>  - Copyright year for CompileProperties
>>  - Redo translation for CS.properties
>>  - Spot convert CurrencySymbols.properties
>>  - ... and 6 more: https://git.openjdk.org/jdk/compare/4386d42d...f15b373a
>
> src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties line 22:
> 
>> 20: # Peter Smolik
>> 21: Cp1250 WINDOWS-1250 0x00FF
>> 22: # Patch attributed to havardw at underdusken.no (H?vard Wigtil)
> 
> This does not seem to have been a correct conversion.

Right, that `?` looks to have been incorrectly converted during the ISO-8859-1 to UTF-8 conversion. (I can't find the script used for conversion as this change is from some time ago.)

Since the change occurs in a comment (thankfully), it should be harmless and the next upstream update of this file would overwrite this incorrect change. However, this file does not seem to be updated that often, so I can also file an issue to correct this if you would prefer that.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/12726#discussion_r2036165417

From ihse at openjdk.org  Thu Apr 10 07:34:37 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Thu, 10 Apr 2025 07:34:37 GMT
Subject: jmx-dev RFR: 8301991: Convert l10n properties resource bundles
 to UTF-8 native [v6]
In-Reply-To: <f565K9bPPbDC3a6M34NDH-DLKzxunYzBaSTkcOAF8bM=.0d00299f-48f7-43ef-8b70-f037bd1441c1@github.com>
References: <0MB7FLFNfaGEWssr9X54UJ_iZNFWBJkxQ1yusP7fsuY=.3f9f3de5-fe84-48e6-9449-626cac42da0b@github.com>
 <cqbVzCGKX2oiVnvlF5eU47L-vXRpPXyuI7DK6Eu9Nf0=.7375f607-3413-4171-a51b-b841908e0684@github.com>
 <_YOUyzMbSEXFduCKVgyis37kwTlGSjBbP8VlFu3xQpU=.9b668e2a-8f91-476d-8914-13dc33a0b9e5@github.com>
 <f565K9bPPbDC3a6M34NDH-DLKzxunYzBaSTkcOAF8bM=.0d00299f-48f7-43ef-8b70-f037bd1441c1@github.com>
Message-ID: <tfpN5b1ld7DNt2HZDgxqWONwyUdyV9GZ27pW5Wjj5mc=.9bcea0db-9d33-40d3-89b3-f8d329da29e7@github.com>

On Wed, 9 Apr 2025 21:26:15 GMT, Justin Lu <jlu at openjdk.org> wrote:

>> src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties line 22:
>> 
>>> 20: # Peter Smolik
>>> 21: Cp1250 WINDOWS-1250 0x00FF
>>> 22: # Patch attributed to havardw at underdusken.no (H?vard Wigtil)
>> 
>> This does not seem to have been a correct conversion.
>
> Right, that `?` looks to have been incorrectly converted during the ISO-8859-1 to UTF-8 conversion. (I can't find the script used for conversion as this change is from some time ago.)
> 
> Since the change occurs in a comment (thankfully), it should be harmless and the next upstream update of this file would overwrite this incorrect change. However, this file does not seem to be updated that often, so I can also file an issue to correct this if you would prefer that.

You don't have to do that, I'm working on an omnibus UTF-8 fixing PR right now, where I will include a fix for this as well.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/12726#discussion_r2036695622

From ihse at openjdk.org  Thu Apr 10 07:34:37 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Thu, 10 Apr 2025 07:34:37 GMT
Subject: jmx-dev RFR: 8301991: Convert l10n properties resource bundles
 to UTF-8 native [v6]
In-Reply-To: <tfpN5b1ld7DNt2HZDgxqWONwyUdyV9GZ27pW5Wjj5mc=.9bcea0db-9d33-40d3-89b3-f8d329da29e7@github.com>
References: <0MB7FLFNfaGEWssr9X54UJ_iZNFWBJkxQ1yusP7fsuY=.3f9f3de5-fe84-48e6-9449-626cac42da0b@github.com>
 <cqbVzCGKX2oiVnvlF5eU47L-vXRpPXyuI7DK6Eu9Nf0=.7375f607-3413-4171-a51b-b841908e0684@github.com>
 <_YOUyzMbSEXFduCKVgyis37kwTlGSjBbP8VlFu3xQpU=.9b668e2a-8f91-476d-8914-13dc33a0b9e5@github.com>
 <f565K9bPPbDC3a6M34NDH-DLKzxunYzBaSTkcOAF8bM=.0d00299f-48f7-43ef-8b70-f037bd1441c1@github.com>
 <tfpN5b1ld7DNt2HZDgxqWONwyUdyV9GZ27pW5Wjj5mc=.9bcea0db-9d33-40d3-89b3-f8d329da29e7@github.com>
Message-ID: <Ivm7XuioTIWOn0303Mm7LUqVPTRhzYRMd8IL8l7EZzA=.463fee71-99a5-420d-827b-eefe7f819877@github.com>

On Thu, 10 Apr 2025 07:31:37 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

>> Right, that `?` looks to have been incorrectly converted during the ISO-8859-1 to UTF-8 conversion. (I can't find the script used for conversion as this change is from some time ago.)
>> 
>> Since the change occurs in a comment (thankfully), it should be harmless and the next upstream update of this file would overwrite this incorrect change. However, this file does not seem to be updated that often, so I can also file an issue to correct this if you would prefer that.
>
> You don't have to do that, I'm working on an omnibus UTF-8 fixing PR right now, where I will include a fix for this as well.

If anything, I might be a bit worried that there are more incorrect conversions stemming from this PR, that my automated tools and manual scanning has not revealed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/12726#discussion_r2036696723

From eirbjo at openjdk.org  Thu Apr 10 08:10:42 2025
From: eirbjo at openjdk.org (Eirik =?UTF-8?B?QmrDuHJzbsO4cw==?=)
Date: Thu, 10 Apr 2025 08:10:42 GMT
Subject: jmx-dev RFR: 8301991: Convert l10n properties resource bundles
 to UTF-8 native [v6]
In-Reply-To: <Ivm7XuioTIWOn0303Mm7LUqVPTRhzYRMd8IL8l7EZzA=.463fee71-99a5-420d-827b-eefe7f819877@github.com>
References: <0MB7FLFNfaGEWssr9X54UJ_iZNFWBJkxQ1yusP7fsuY=.3f9f3de5-fe84-48e6-9449-626cac42da0b@github.com>
 <cqbVzCGKX2oiVnvlF5eU47L-vXRpPXyuI7DK6Eu9Nf0=.7375f607-3413-4171-a51b-b841908e0684@github.com>
 <_YOUyzMbSEXFduCKVgyis37kwTlGSjBbP8VlFu3xQpU=.9b668e2a-8f91-476d-8914-13dc33a0b9e5@github.com>
 <f565K9bPPbDC3a6M34NDH-DLKzxunYzBaSTkcOAF8bM=.0d00299f-48f7-43ef-8b70-f037bd1441c1@github.com>
 <tfpN5b1ld7DNt2HZDgxqWONwyUdyV9GZ27pW5Wjj5mc=.9bcea0db-9d33-40d3-89b3-f8d329da29e7@github.com>
 <Ivm7XuioTIWOn0303Mm7LUqVPTRhzYRMd8IL8l7EZzA=.463fee71-99a5-420d-827b-eefe7f819877@github.com>
Message-ID: <6c6DqyCqyPonBZgUU8BpYJR3JQvMXjWm9ulq4SN25Do=.77775825-716d-4908-ae24-c4cf1ead78a5@github.com>

On Thu, 10 Apr 2025 07:32:18 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

>> You don't have to do that, I'm working on an omnibus UTF-8 fixing PR right now, where I will include a fix for this as well.
>
> If anything, I might be a bit worried that there are more incorrect conversions stemming from this PR, that my automated tools and manual scanning has not revealed.

Some observations: 

1: This PR seems to have been abondoned, so perhaps this discussion belongs in #15694 ?

2: The `?` (Unicode 'Latin small letter a with ring above' U+00E5) was correctly encoded as 0xEF in ISO-8859-1 previous to this change.

3: The conversion changed this `0xEF` to the three-byte sequence `ef bf bd`

4: This is as-if the file was incorrctly decoded using UTF-8, then encoded using UTF-8:


byte[] origBytes = "?".getBytes(StandardCharsets.ISO_8859_1);
String decoded = new String(origBytes, StandardCharsets.UTF_8);
byte[] encoded = decoded.getBytes(StandardCharsets.UTF_8);
String hex = HexFormat.of().formatHex(encoded);
assertEquals("efbfbd", hex);
``` 

Like @magicus I'm worried that similar incorrect decoding could have been introduced by the same script in other files.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/12726#discussion_r2036767319

From ihse at openjdk.org  Thu Apr 10 08:38:38 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Thu, 10 Apr 2025 08:38:38 GMT
Subject: jmx-dev RFR: 8301991: Convert l10n properties resource bundles
 to UTF-8 native [v6]
In-Reply-To: <6c6DqyCqyPonBZgUU8BpYJR3JQvMXjWm9ulq4SN25Do=.77775825-716d-4908-ae24-c4cf1ead78a5@github.com>
References: <0MB7FLFNfaGEWssr9X54UJ_iZNFWBJkxQ1yusP7fsuY=.3f9f3de5-fe84-48e6-9449-626cac42da0b@github.com>
 <cqbVzCGKX2oiVnvlF5eU47L-vXRpPXyuI7DK6Eu9Nf0=.7375f607-3413-4171-a51b-b841908e0684@github.com>
 <_YOUyzMbSEXFduCKVgyis37kwTlGSjBbP8VlFu3xQpU=.9b668e2a-8f91-476d-8914-13dc33a0b9e5@github.com>
 <f565K9bPPbDC3a6M34NDH-DLKzxunYzBaSTkcOAF8bM=.0d00299f-48f7-43ef-8b70-f037bd1441c1@github.com>
 <tfpN5b1ld7DNt2HZDgxqWONwyUdyV9GZ27pW5Wjj5mc=.9bcea0db-9d33-40d3-89b3-f8d329da29e7@github.com>
 <Ivm7XuioTIWOn0303Mm7LUqVPTRhzYRMd8IL8l7EZzA=.463fee71-99a5-420d-827b-eefe7f819877@github.com>
 <6c6DqyCqyPonBZgUU8BpYJR3JQvMXjWm9ulq4SN25Do=.77775825-716d-4908-ae24-c4cf1ead78a5@github.com>
Message-ID: <qG3Fpev16BBWYvs2zSE5CeqmTcfRF4h_l-PAM3PVHns=.4a1e25de-c3e2-4876-a8e5-c3f32cfef4c1@github.com>

On Thu, 10 Apr 2025 08:08:02 GMT, Eirik Bj?rsn?s <eirbjo at openjdk.org> wrote:

>> If anything, I might be a bit worried that there are more incorrect conversions stemming from this PR, that my automated tools and manual scanning has not revealed.
>
> Some observations: 
> 
> 1: This PR seems to have been abondoned, so perhaps this discussion belongs in #15694 ?
> 
> 2: The `?` (Unicode 'Latin small letter a with ring above' U+00E5) was correctly encoded as 0xEF in ISO-8859-1 previous to this change.
> 
> 3: The conversion changed this `0xEF` to the three-byte sequence `ef bf bd`
> 
> 4: This is as-if the file was incorrctly decoded using UTF-8, then encoded using UTF-8:
> 
> 
> byte[] origBytes = "?".getBytes(StandardCharsets.ISO_8859_1);
> String decoded = new String(origBytes, StandardCharsets.UTF_8);
> byte[] encoded = decoded.getBytes(StandardCharsets.UTF_8);
> String hex = HexFormat.of().formatHex(encoded);
> assertEquals("efbfbd", hex);
> ``` 
> 
> Like @magicus I'm worried that similar incorrect decoding could have been introduced by the same script in other files.

> This PR seems to have been abondoned, so perhaps this discussion belongs in https://github.com/openjdk/jdk/pull/15694 ?

Oh, I didn't notice this was supplanted by another PR. It might be better to continue there, yes. Even if closed PRs seldom are the best places to conduct discussions, I think it might be a good idea to scrutinize all files modified by this script.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/12726#discussion_r2036820765

From ihse at openjdk.org  Thu Apr 10 08:41:45 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Thu, 10 Apr 2025 08:41:45 GMT
Subject: jmx-dev RFR: 8301991: Convert l10n properties resource bundles
 to UTF-8 native [v2]
In-Reply-To: <wnD0vkZ1qWZcUbEguZ4ixnJYH0tEx3b960Vxv6dcqRw=.3140a102-55c6-4ec4-bea9-3a9deca7e852@github.com>
References: <FLJUcoMTKEHcWQyslyz5MmH36l2DJESXgsvvSTpQVGI=.33d28019-5a2a-4a0e-8f99-d89b39dbf86e@github.com>
 <wnD0vkZ1qWZcUbEguZ4ixnJYH0tEx3b960Vxv6dcqRw=.3140a102-55c6-4ec4-bea9-3a9deca7e852@github.com>
Message-ID: <yO1q1NVjOInbzcWCkJra3sYGkfv3OBhg__dKO0w--s0=.d7a4b185-6ed1-4ae6-bdfe-01b3204da46a@github.com>

On Wed, 13 Sep 2023 17:38:28 GMT, Justin Lu <jlu at openjdk.org> wrote:

>> JDK .properties files still use ISO-8859-1 encoding with escape sequences. It would improve readability to see the native characters instead of escape sequences (especially for the L10n process). The majority of files changed are localized resource files.
>> 
>> This change converts the Unicode escape sequences in the JDK .properties files (both in src and test) to UTF-8 native characters. Additionally, the build logic is adjusted to read the .properties files in UTF-8 while generating the ListResourceBundle files.
>> 
>> The only escape sequence not converted was `\u0020` as this is used to denote intentional trailing white space. (E.g. `key=This is the value:\u0020`)
>> 
>> The conversion was done using native2ascii with options `-reverse -encoding UTF-8`.
>> 
>> If this PR is integrated, the IDE default encoding for .properties files need to be updated to UTF-8. (IntelliJ IDEA locks .properties files as ISO-8859-1 unless manually changed).
>
> Justin Lu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Replace InputStreamReader with BufferedReader

Continuing the discussion that was started at a predecessor to this PR, https://github.com/openjdk/jdk/pull/12726#discussion_r2035582242. At least one incorrect conversion has been found in this PR. It might be worthwhile to double- and triple-check all the other conversions as well.

As part of https://bugs.openjdk.org/browse/JDK-8301971 I am trying various ways of detecting files without UTF-8 encoding, but it is still a bit of hit and miss, since there are no surefire way of telling which encoding a file has, only heuristics. So finding and following up potential sources of error is important.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15694#issuecomment-2791991649
PR Comment: https://git.openjdk.org/jdk/pull/15694#issuecomment-2791997157

From eirbjo at openjdk.org  Thu Apr 10 08:48:37 2025
From: eirbjo at openjdk.org (Eirik =?UTF-8?B?QmrDuHJzbsO4cw==?=)
Date: Thu, 10 Apr 2025 08:48:37 GMT
Subject: jmx-dev RFR: 8301991: Convert l10n properties resource bundles
 to UTF-8 native [v2]
In-Reply-To: <wnD0vkZ1qWZcUbEguZ4ixnJYH0tEx3b960Vxv6dcqRw=.3140a102-55c6-4ec4-bea9-3a9deca7e852@github.com>
References: <FLJUcoMTKEHcWQyslyz5MmH36l2DJESXgsvvSTpQVGI=.33d28019-5a2a-4a0e-8f99-d89b39dbf86e@github.com>
 <wnD0vkZ1qWZcUbEguZ4ixnJYH0tEx3b960Vxv6dcqRw=.3140a102-55c6-4ec4-bea9-3a9deca7e852@github.com>
Message-ID: <0q0gTsqIsYtmzAfNYbBXksUXKdZh2uzQ9yvSETKAP88=.137372e6-d63e-4539-b196-4bd9ef1ddd16@github.com>

On Wed, 13 Sep 2023 17:38:28 GMT, Justin Lu <jlu at openjdk.org> wrote:

>> JDK .properties files still use ISO-8859-1 encoding with escape sequences. It would improve readability to see the native characters instead of escape sequences (especially for the L10n process). The majority of files changed are localized resource files.
>> 
>> This change converts the Unicode escape sequences in the JDK .properties files (both in src and test) to UTF-8 native characters. Additionally, the build logic is adjusted to read the .properties files in UTF-8 while generating the ListResourceBundle files.
>> 
>> The only escape sequence not converted was `\u0020` as this is used to denote intentional trailing white space. (E.g. `key=This is the value:\u0020`)
>> 
>> The conversion was done using native2ascii with options `-reverse -encoding UTF-8`.
>> 
>> If this PR is integrated, the IDE default encoding for .properties files need to be updated to UTF-8. (IntelliJ IDEA locks .properties files as ISO-8859-1 unless manually changed).
>
> Justin Lu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Replace InputStreamReader with BufferedReader

FWIW, I checked out the revision of the commit previous to this change and found the following:


% git checkout b55e418a077791b39992042411cde97f68dc39fe^ 
% find src -name "*.properties" | xargs file | grep -v ASCII
src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties: 
  ISO-8859 text
src/java.xml.crypto/share/classes/com/sun/org/apache/xml/internal/security/resource/xmlsecurity_de.properties:
  Unicode text, UTF-8 text, with very long lines (322)


Which indicates that that this is the only non-ASCII, non-UTF-8 property file. So we may be lucky.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15694#issuecomment-2792014164

From ihse at openjdk.org  Thu Apr 10 09:45:56 2025
From: ihse at openjdk.org (Magnus Ihse Bursie)
Date: Thu, 10 Apr 2025 09:45:56 GMT
Subject: jmx-dev RFR: 8301991: Convert l10n properties resource bundles
 to UTF-8 native [v2]
In-Reply-To: <wnD0vkZ1qWZcUbEguZ4ixnJYH0tEx3b960Vxv6dcqRw=.3140a102-55c6-4ec4-bea9-3a9deca7e852@github.com>
References: <FLJUcoMTKEHcWQyslyz5MmH36l2DJESXgsvvSTpQVGI=.33d28019-5a2a-4a0e-8f99-d89b39dbf86e@github.com>
 <wnD0vkZ1qWZcUbEguZ4ixnJYH0tEx3b960Vxv6dcqRw=.3140a102-55c6-4ec4-bea9-3a9deca7e852@github.com>
Message-ID: <GlAmyjF3ZmTQHUhkTXRmmrijCWM_4JoredRJOuyVyIc=.775fbc8d-80da-4b0b-b2e1-e5b7e75b3d52@github.com>

On Wed, 13 Sep 2023 17:38:28 GMT, Justin Lu <jlu at openjdk.org> wrote:

>> JDK .properties files still use ISO-8859-1 encoding with escape sequences. It would improve readability to see the native characters instead of escape sequences (especially for the L10n process). The majority of files changed are localized resource files.
>> 
>> This change converts the Unicode escape sequences in the JDK .properties files (both in src and test) to UTF-8 native characters. Additionally, the build logic is adjusted to read the .properties files in UTF-8 while generating the ListResourceBundle files.
>> 
>> The only escape sequence not converted was `\u0020` as this is used to denote intentional trailing white space. (E.g. `key=This is the value:\u0020`)
>> 
>> The conversion was done using native2ascii with options `-reverse -encoding UTF-8`.
>> 
>> If this PR is integrated, the IDE default encoding for .properties files need to be updated to UTF-8. (IntelliJ IDEA locks .properties files as ISO-8859-1 unless manually changed).
>
> Justin Lu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Replace InputStreamReader with BufferedReader

Thanks for checking!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15694#issuecomment-2792170460

From jlu at openjdk.org  Thu Apr 10 18:47:53 2025
From: jlu at openjdk.org (Justin Lu)
Date: Thu, 10 Apr 2025 18:47:53 GMT
Subject: jmx-dev RFR: 8301991: Convert l10n properties resource bundles
 to UTF-8 native [v2]
In-Reply-To: <0q0gTsqIsYtmzAfNYbBXksUXKdZh2uzQ9yvSETKAP88=.137372e6-d63e-4539-b196-4bd9ef1ddd16@github.com>
References: <FLJUcoMTKEHcWQyslyz5MmH36l2DJESXgsvvSTpQVGI=.33d28019-5a2a-4a0e-8f99-d89b39dbf86e@github.com>
 <wnD0vkZ1qWZcUbEguZ4ixnJYH0tEx3b960Vxv6dcqRw=.3140a102-55c6-4ec4-bea9-3a9deca7e852@github.com>
 <0q0gTsqIsYtmzAfNYbBXksUXKdZh2uzQ9yvSETKAP88=.137372e6-d63e-4539-b196-4bd9ef1ddd16@github.com>
Message-ID: <9aQcWun5KNgHgELVwkc3478_RtqfhRL1Cxvyn2Yl0Nw=.07ee596f-e738-4796-8d27-14621ed8860c@github.com>

On Thu, 10 Apr 2025 08:44:28 GMT, Eirik Bj?rsn?s <eirbjo at openjdk.org> wrote:

>> Justin Lu has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Replace InputStreamReader with BufferedReader
>
> FWIW, I checked out the revision of the commit previous to this change and found the following:
> 
> 
> % git checkout b55e418a077791b39992042411cde97f68dc39fe^ 
> % find src -name "*.properties" | xargs file | grep -v ASCII
> src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties: 
>   ISO-8859 text
> src/java.xml.crypto/share/classes/com/sun/org/apache/xml/internal/security/resource/xmlsecurity_de.properties:
>   Unicode text, UTF-8 text, with very long lines (322)
> 
> 
> Which indicates that that this is the only non-ASCII, non-UTF-8 property file. So we may be lucky.

This conversion was performed under the assumption of ASCII set and Unicode escape sequences, which is the format we expect for the translation process for .properties files. That file should have been omitted from this change. Thank you @eirbjo and @magicus for the analysis and checking!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15694#issuecomment-2794828598