Codereview request for 7130915: File.equals does not give expected results when path contains Non-English characters on Mac OS X

[Xueming Shen]

The fix for jdk8 is now in the TL repository, should be in the "next" 
promotion jdk8
very soon. The plan is for this change to be baked in jdk8 for a while 
before backport
to a 7ux release.

-Sherman

On 07/23/2012 12:56 AM, David Kocher wrote:
> I really would appreciate an answer here and a possible resolution to backport this fix. Others are having the same issue [1] and are posting to the non maintained JIRA issue tracker.
>
> -- David
>
>
> [1] http://java.net/jira/browse/MACOSX_PORT-165
>
> On 04.07.2012, at 18:40, David Kocher<dkocher at sudo.ch>  wrote:
>
>> Is there a corresponding ticket in bugs.sun.com that allows to track this issue?
>>
>> -- David
>>
>> On 27.06.2012, at 09:28, David Kocher wrote:
>>
>>> I welcome this issue is getting some serious attention then. When will this be backported to 7u?
>>>
>>> -- David
>>>
>>> On 24.06.2012, at 18:58, Xueming Shen wrote:
>>>
>>>> Yes, I believe the issue described in MACOSX_PORT-165 is the
>>>> same issue this patch is trying to solve.
>>>>
>>>> Btw, it appears there are typos in the note(2), my mini keyboard obviously
>>>> is too sticky:-)
>>>>
>>>> (2) normalize the resulting file name from macosx fs APIs from nfd->nfc before passing
>>>>   back to java.io.File (File.list() and canonicalize()), so we deal with nfc file name
>>>>   (as "usual")  for java.io classes/APIs.
>>>>
>>>> -sherman
>>>>
>>>> On 6/24/12 7:50 AM, David Kocher wrote:
>>>>> Will this address issue MACOSX_PORT-165 [1]?
>>>>>
>>>>> [1] http://java.net/jira/browse/MACOSX_PORT-165
>>>>>
>>>>> -- David
>>>>>
>>>>> On 22.06.2012, at 19:01, Xueming Shen wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> Here is the proposed change to support Unicode nfd/nfc and case insensitive
>>>>>> file path on MacOSX file system.
>>>>>>
>>>>>> 7130915: File.equals does not give expected results when path contains Non-English characters on Mac OS X
>>>>>> 7168427: FileInputStream cannot open file where the file path contains asian characters [macosx]
>>>>>>
>>>>>> While these two bug reports are only against java.io, we have the same issue in javax.nio.file.
>>>>>> Here is the webrev
>>>>>>
>>>>>> http://cr.openjdk.java.net/~sherman/7130915_7168427/webrev/
>>>>>>
>>>>>> Here is the brief summary of the changes
>>>>>>
>>>>>> java.io.File:
>>>>>> (1) removed nfc->nfd conversion in io_util.h/WITH_PLATFORM_STRING, which means
>>>>>>    we are now passing nfc/composite characters directly into macosx file system APIs
>>>>>>    without  normalize them to nfd. It appears macosx fs APIs do take nfc, though it uses
>>>>>>    nfd.
>>>>>>
>>>>>> (2) normalize the resulting file name from macosx fs APIs from nfd->nfd before passing
>>>>>>    back to java.io.File (File.list() and canonicalize()), so we deal with nfdc file name
>>>>>>    (as "usual")  for java.io classes/APIs.
>>>>>>
>>>>>> (3) fs.compare()/hashCode() was updated to be case insensitive
>>>>>>
>>>>>> (4) hasCode() was updated to use the new String.hash32().
>>>>>>
>>>>>> java.nio.file:
>>>>>>
>>>>>> (5) added a setof MacOSXFile... on top of existing BsdFile... classes. An alternative is to
>>>>>> update those BsdFile... classes directly to address the macosx specific issues. But given
>>>>>> there might be developers over there might work on open jdk BSD port and have dependency
>>>>>> on these classes, it might be desirable to have another separate layer of MacOSXFile...
>>>>>> classes. So now the default FileSystem/Provider is MacOSXFileSystemProvider and
>>>>>> MacOSXFileSystem.
>>>>>>
>>>>>> (6)  the "main" changes are in MacOSXFileSystem, in which the corresponding methods
>>>>>> were added to handle, case insensitive and nfd<=>nfc normalization, including the
>>>>>> pathmatcher.
>>>>>>
>>>>>> (7) compare is now are case-insensitive
>>>>>>
>>>>>> (8) hashCode is now murmur3_32(), this is true for all Solaris/Unix/Linux and maxosx.
>>>>>>
>>>>>>
>>>>>> Though lots of files have been touched, but the line of changes are actually relatively
>>>>>> small.
>>>>>>
>>>>>> The proposed change only deals with the default case-sensitiveness seting, which is
>>>>>> case insensitive. On MaxOSX, you actually can configure the HFS+ file system or the
>>>>>> mounted vol to be case-sensitive. A possible approach is to have some extra FileStore
>>>>>> attributes, such as a isCaseSensitive and to use case sensitive compare/equal on
>>>>>> such fs, but this can be dealt with separately later.
>>>>>>
>>>>>> Thanks,
>>>>>> -Sherman
>>>>>>
>>>>