Codereview request for 7130915: File.equals does not give expected results when path contains Non-English characters on Mac OS X

[David Kocher]

I really would appreciate an answer here and a possible resolution to backport this fix. Others are having the same issue [1] and are posting to the non maintained JIRA issue tracker.

-- David


[1] http://java.net/jira/browse/MACOSX_PORT-165

On 04.07.2012, at 18:40, David Kocher <dkocher at sudo.ch> wrote:

> Is there a corresponding ticket in bugs.sun.com that allows to track this issue?
> 
> -- David
> 
> On 27.06.2012, at 09:28, David Kocher wrote:
> 
>> I welcome this issue is getting some serious attention then. When will this be backported to 7u?
>> 
>> -- David
>> 
>> On 24.06.2012, at 18:58, Xueming Shen wrote:
>> 
>>> 
>>> Yes, I believe the issue described in MACOSX_PORT-165 is the
>>> same issue this patch is trying to solve.
>>> 
>>> Btw, it appears there are typos in the note(2), my mini keyboard obviously
>>> is too sticky:-)
>>> 
>>> (2) normalize the resulting file name from macosx fs APIs from nfd->nfc before passing
>>>  back to java.io.File (File.list() and canonicalize()), so we deal with nfc file name
>>>  (as "usual")  for java.io classes/APIs.
>>> 
>>> -sherman
>>> 
>>> On 6/24/12 7:50 AM, David Kocher wrote:
>>>> Will this address issue MACOSX_PORT-165 [1]?
>>>> 
>>>> [1] http://java.net/jira/browse/MACOSX_PORT-165
>>>> 
>>>> -- David
>>>> 
>>>> On 22.06.2012, at 19:01, Xueming Shen wrote:
>>>> 
>>>>> Hi
>>>>> 
>>>>> Here is the proposed change to support Unicode nfd/nfc and case insensitive
>>>>> file path on MacOSX file system.
>>>>> 
>>>>> 7130915: File.equals does not give expected results when path contains Non-English characters on Mac OS X
>>>>> 7168427: FileInputStream cannot open file where the file path contains asian characters [macosx]
>>>>> 
>>>>> While these two bug reports are only against java.io, we have the same issue in javax.nio.file.
>>>>> Here is the webrev
>>>>> 
>>>>> http://cr.openjdk.java.net/~sherman/7130915_7168427/webrev/
>>>>> 
>>>>> Here is the brief summary of the changes
>>>>> 
>>>>> java.io.File:
>>>>> (1) removed nfc->nfd conversion in io_util.h/WITH_PLATFORM_STRING, which means
>>>>>   we are now passing nfc/composite characters directly into macosx file system APIs
>>>>>   without  normalize them to nfd. It appears macosx fs APIs do take nfc, though it uses
>>>>>   nfd.
>>>>> 
>>>>> (2) normalize the resulting file name from macosx fs APIs from nfd->nfd before passing
>>>>>   back to java.io.File (File.list() and canonicalize()), so we deal with nfdc file name
>>>>>   (as "usual")  for java.io classes/APIs.
>>>>> 
>>>>> (3) fs.compare()/hashCode() was updated to be case insensitive
>>>>> 
>>>>> (4) hasCode() was updated to use the new String.hash32().
>>>>> 
>>>>> java.nio.file:
>>>>> 
>>>>> (5) added a setof MacOSXFile... on top of existing BsdFile... classes. An alternative is to
>>>>> update those BsdFile... classes directly to address the macosx specific issues. But given
>>>>> there might be developers over there might work on open jdk BSD port and have dependency
>>>>> on these classes, it might be desirable to have another separate layer of MacOSXFile...
>>>>> classes. So now the default FileSystem/Provider is MacOSXFileSystemProvider and
>>>>> MacOSXFileSystem.
>>>>> 
>>>>> (6)  the "main" changes are in MacOSXFileSystem, in which the corresponding methods
>>>>> were added to handle, case insensitive and nfd<=>nfc normalization, including the
>>>>> pathmatcher.
>>>>> 
>>>>> (7) compare is now are case-insensitive
>>>>> 
>>>>> (8) hashCode is now murmur3_32(), this is true for all Solaris/Unix/Linux and maxosx.
>>>>> 
>>>>> 
>>>>> Though lots of files have been touched, but the line of changes are actually relatively
>>>>> small.
>>>>> 
>>>>> The proposed change only deals with the default case-sensitiveness seting, which is
>>>>> case insensitive. On MaxOSX, you actually can configure the HFS+ file system or the
>>>>> mounted vol to be case-sensitive. A possible approach is to have some extra FileStore
>>>>> attributes, such as a isCaseSensitive and to use case sensitive compare/equal on
>>>>> such fs, but this can be dealt with separately later.
>>>>> 
>>>>> Thanks,
>>>>> -Sherman
>>>>> 
>>> 
>>> 
>> 
>