Codereview request for 7130915: File.equals does not give expected results when path contains Non-English characters on Mac OS X

[David Kocher]

I (and certainly others not even aware of the issue) would welcome if the backport could be given higher priority as this is a show stopper bug to bundle applications with a current jre7 runtime.

-- David

On 24.07.2012, at 00:27, Xueming Shen <xueming.shen at oracle.com> wrote:

> 
> The fix for jdk8 is now in the TL repository, should be in the "next" promotion jdk8
> very soon. The plan is for this change to be baked in jdk8 for a while before backport
> to a 7ux release.
> 
> -Sherman
> 
> On 07/23/2012 12:56 AM, David Kocher wrote:
>> I really would appreciate an answer here and a possible resolution to backport this fix. Others are having the same issue [1] and are posting to the non maintained JIRA issue tracker.
>> 
>> -- David
>> 
>> 
>> [1] http://java.net/jira/browse/MACOSX_PORT-165
>> 
>> On 04.07.2012, at 18:40, David Kocher<dkocher at sudo.ch>  wrote:
>> 
>>> Is there a corresponding ticket in bugs.sun.com that allows to track this issue?
>>> 
>>> -- David
>>> 
>>> On 27.06.2012, at 09:28, David Kocher wrote:
>>> 
>>>> I welcome this issue is getting some serious attention then. When will this be backported to 7u?
>>>> 
>>>> -- David
>>>> 
>>>> On 24.06.2012, at 18:58, Xueming Shen wrote:
>>>> 
>>>>> Yes, I believe the issue described in MACOSX_PORT-165 is the
>>>>> same issue this patch is trying to solve.
>>>>> 
>>>>> Btw, it appears there are typos in the note(2), my mini keyboard obviously
>>>>> is too sticky:-)
>>>>> 
>>>>> (2) normalize the resulting file name from macosx fs APIs from nfd->nfc before passing
>>>>>  back to java.io.File (File.list() and canonicalize()), so we deal with nfc file name
>>>>>  (as "usual")  for java.io classes/APIs.
>>>>> 
>>>>> -sherman
>>>>> 
>>>>> On 6/24/12 7:50 AM, David Kocher wrote:
>>>>>> Will this address issue MACOSX_PORT-165 [1]?
>>>>>> 
>>>>>> [1] http://java.net/jira/browse/MACOSX_PORT-165
>>>>>> 
>>>>>> -- David
>>>>>> 
>>>>>> On 22.06.2012, at 19:01, Xueming Shen wrote:
>>>>>> 
>>>>>>> Hi
>>>>>>> 
>>>>>>> Here is the proposed change to support Unicode nfd/nfc and case insensitive
>>>>>>> file path on MacOSX file system.
>>>>>>> 
>>>>>>> 7130915: File.equals does not give expected results when path contains Non-English characters on Mac OS X
>>>>>>> 7168427: FileInputStream cannot open file where the file path contains asian characters [macosx]
>>>>>>> 
>>>>>>> While these two bug reports are only against java.io, we have the same issue in javax.nio.file.
>>>>>>> Here is the webrev
>>>>>>> 
>>>>>>> http://cr.openjdk.java.net/~sherman/7130915_7168427/webrev/
>>>>>>> 
>>>>>>> Here is the brief summary of the changes
>>>>>>> 
>>>>>>> java.io.File:
>>>>>>> (1) removed nfc->nfd conversion in io_util.h/WITH_PLATFORM_STRING, which means
>>>>>>>   we are now passing nfc/composite characters directly into macosx file system APIs
>>>>>>>   without  normalize them to nfd. It appears macosx fs APIs do take nfc, though it uses
>>>>>>>   nfd.
>>>>>>> 
>>>>>>> (2) normalize the resulting file name from macosx fs APIs from nfd->nfd before passing
>>>>>>>   back to java.io.File (File.list() and canonicalize()), so we deal with nfdc file name
>>>>>>>   (as "usual")  for java.io classes/APIs.
>>>>>>> 
>>>>>>> (3) fs.compare()/hashCode() was updated to be case insensitive
>>>>>>> 
>>>>>>> (4) hasCode() was updated to use the new String.hash32().
>>>>>>> 
>>>>>>> java.nio.file:
>>>>>>> 
>>>>>>> (5) added a setof MacOSXFile... on top of existing BsdFile... classes. An alternative is to
>>>>>>> update those BsdFile... classes directly to address the macosx specific issues. But given
>>>>>>> there might be developers over there might work on open jdk BSD port and have dependency
>>>>>>> on these classes, it might be desirable to have another separate layer of MacOSXFile...
>>>>>>> classes. So now the default FileSystem/Provider is MacOSXFileSystemProvider and
>>>>>>> MacOSXFileSystem.
>>>>>>> 
>>>>>>> (6)  the "main" changes are in MacOSXFileSystem, in which the corresponding methods
>>>>>>> were added to handle, case insensitive and nfd<=>nfc normalization, including the
>>>>>>> pathmatcher.
>>>>>>> 
>>>>>>> (7) compare is now are case-insensitive
>>>>>>> 
>>>>>>> (8) hashCode is now murmur3_32(), this is true for all Solaris/Unix/Linux and maxosx.
>>>>>>> 
>>>>>>> 
>>>>>>> Though lots of files have been touched, but the line of changes are actually relatively
>>>>>>> small.
>>>>>>> 
>>>>>>> The proposed change only deals with the default case-sensitiveness seting, which is
>>>>>>> case insensitive. On MaxOSX, you actually can configure the HFS+ file system or the
>>>>>>> mounted vol to be case-sensitive. A possible approach is to have some extra FileStore
>>>>>>> attributes, such as a isCaseSensitive and to use case sensitive compare/equal on
>>>>>>> such fs, but this can be dealt with separately later.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> -Sherman
>>>>>>> 
>>>>> 
>