RFR(S): 8201540: [AIX] Extend the set of supported charsets in java.base

[Bhaktavatsal R Maram]

Hi All,

Please review.

Bug: https://bugs.openjdk.java.net/browse/JDK-8201540
webrev: http://cr.openjdk.java.net/~gromero/8201540/v1/webrev/

In this patch,

1. Default charsets Big5, Big5_Solaris, Big5_HKSCS, GB18030, IBM856, IBM921, IBM922, IBM942, IBM942C, IBM943, IBM943C, IBM950, IBM970, IBM1046, IBM1124, IBM1383, ISO_8859_6, ISO_8859_8, MS1252, TIS_620 for different locales supported on AIX are added to standard charsets in java.base module

2. More aliases are added to existing charsets.

3. Source files for IBM-942C and IBM-943C are changed to template to support #1

4. Modified file make/jdk/src/classes/build/tools/charsetmapping/SPI.java to increase the Hashtable capacity that holds aliases of standard charsets. As the no.of charsets we include in standard charsets are more in this patch, existing capacity of Hashtable used to hold aliases is not efficient. Without change to this file, JDK won't get built and following error is seen

Exception in thread "main" java.lang.RuntimeException: Cannot find a suitable size within given constraints
        at build.tools.charsetmapping.Hasher.build(Hasher.java:122)
        at build.tools.charsetmapping.Hasher.genClass(Hasher.java:261)
        at build.tools.charsetmapping.SPI.genClass(SPI.java:130)
        at build.tools.charsetmapping.Main.main(Main.java:99)

Please note that webrev/index.html is not showing below 2 files I deleted. However, webrev/jdk11.changeset file show them deleted properly.
  src/jdk.charsets/share/classes/sun/nio/cs/ext/IBM942C.java
  src/jdk.charsets/share/classes/sun/nio/cs/ext/IBM943C.java

I've built the JDK with this patch on both Linux and Aix.


Thanks,
Bhaktavatsal Reddy


  

-----Volker Simonis <volker.simonis at gmail.com> wrote: -----
To: Bhaktavatsal R Maram <bhamaram at in.ibm.com>
From: Volker Simonis <volker.simonis at gmail.com>
Date: 04/13/2018 08:51PM
Cc: Alan Bateman <Alan.Bateman at oracle.com>, Java Core Libs <core-libs-dev at openjdk.java.net>, Tim Ellison <Tim_Ellison at uk.ibm.com>, ppc-aix-port-dev at openjdk.java.net
Subject: Re: Missing many locales support on AIX platform

Hi Bhaktavatsal Reddy,

thanks for addressing this long standing issue. I've opened "8201540:
[AIX] Extend the set of supported charsets in java.base" [1] to track
this issue.

As I wrote in the bug report, this problem is the consequence of an
emergency fix (JDK-8081332) I did back in 2015 to fix the build on AIX
after the integration of the modularity support change (see
discussion: [2]). At that time I only added the minimal set of
charsets which were required to fix the build.

It would be great if you can get in touch with your IBM colleagues to
find out which are the default extended charsets supported by IBM J9
on AIX. I think we should try to use a similar set here in our OpenJDK
port which is also used by the OpenJ9 for building OpenJ9 on AIX.

Also, your IBM colleagues can help you to host a webrev which will
make the further review of these changes much easier. I've but Tim
from IBM on CC which should have an overview of all the IBM people
involved in the OpenJDK project. Besides that I know at least the
following people who might help you:

Michihiro Horie: HORIE at jp.ibm.com
Hiroshi H Horii: horii at jp.ibm.com
Gustavo Romero: gromero at linux.vnet.ibm.com
Matthew Brandyberyy: mbrandy at linux.vnet.ibm.com

Thank you and best regards,
Volker

[1] https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8201540&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg&m=cieXGANE8bD3liEC2gJzl5hfZorR2qIfggL6U9t-Et8&s=hC4gcA6uomYgY26uR74VelobSIK1ReDsjRZGmI46UBY&e=
[2] https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_2d-2Ddev_2015-2DMay_005431.html&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg&m=cieXGANE8bD3liEC2gJzl5hfZorR2qIfggL6U9t-Et8&s=qfYLaVVn7tapNDhv5bY6yE72nhngaWqte4wtRhQ3wB8&e=

On Fri, Apr 13, 2018 at 2:42 PM, Bhaktavatsal R Maram
<bhamaram at in.ibm.com> wrote:
> Hi Alan,
>
> Thank you for your response. I'm happy that my patch was attached. But, I don't see attachment. So, I inlined patch which contain diffs from 2 changesets in mail text. If a Jira bug is opened for this issue, probably I can attach complete and consolidated patch there.
>
> At high level, I'm adding following charsets to standard charset in java.base. For this, I have to change IBM943C and IBM942C from source to template to handle java package and aliases. It is also required to add codepage 932 as alias for IBM942C because both are one and same.
>
> Big5, Big5_HKSCS, GB18030, IBM942, IBM942C, IBM943, IBM943C, IBM950, IBM970, IBM1124, TIS_620
>
>
> These are default charsets for some of locales supported by Operating System (AIX). Since these are not available in standard charset, JDK can't be used in those locale even if they are available in jdk.charset module (java -version fails).
>
> I've followed some of the discussions around this in community and understand that default charset of a locale should be made available in java.charset module by using stdcs-* mechanism. On Linux, they were added to java.base in similar way. As it is missing for AIX, I've added them to enable JDK support for more locales.
>
>
> Thanks,
> Bhaktavatsal Reddy
>
> -----Alan Bateman <Alan.Bateman at oracle.com> wrote: -----
> To: Bhaktavatsal R Maram <bhamaram at in.ibm.com>, core-libs-dev at openjdk.java.net
> From: Alan Bateman <Alan.Bateman at oracle.com>
> Date: 04/13/2018 03:52PM
> Subject: Re: Missing many locales support on AIX platform
>
> On 13/04/2018 10:35, Bhaktavatsal R Maram wrote:
>> Given that patch is big, I am sending patch as attachment again after changing some mail settings. Hopefully, it will make to community this time.
> Your patch was attached.
>
> If I read it correctly, you've switched IBM943C to a template but there
> aren't additional aliases so this part is effectively a no-op, is that
> right? For IBM932C, you've moved it to be template and added several
> aliases.
>
> The rest is AIX specific and I hope the SAP engineers that maintain the
> AIX port can help you. It may be that you are testing with locales that
> aren't supported configurations for the AIX port in OpenJDK. As a
> general point, we try to keep as many of the exotic and multibyte
> charsets out of java.base. They are of course still available to
> applications via the API and the jdk.charsets service provider module.
>
> -Alan
>
>