JDK 1.8.0_66, diacritics and file problems, follow up

[Fabrizio Giudici]

Hello.

A year ago I asked about problems with the handling of files with  
diacritics in their name, as per this thread:

	http://mail.openjdk.java.net/pipermail/openjfx-dev/2015-April/017183.html

For almost the whole past year I suspended the investigation and moved to  
other things to do, but I've resumed the thing a few days ago, trying to  
create a more controlled scenario.

Among a number of things to discuss, I think that I discovered a bug in  
the JDK. I recall that I was answered that some problems are related to  
the filesystem and Java can't do much, but in this case it seems there's  
an inconsistency in the behaviour beetween the old IO and NIO. Consider  
the test below, that given a path probes the existence and try to read  
bytes by using java.io.File and java.nio.file.*:

     private void tryToOpen (final Path path)
       throws IOException
       {
         final boolean filesExists = Files.exists(path);
         final boolean pathToFileExists = path.toFile().exists();
         boolean canBeOpenedNio = false;

         try
           {
             Files.readAllBytes(path);
             canBeOpenedNio = true;
           }
         catch (IOException e)
           {
           }

         boolean canBeOpenedIo = false;

         try (final InputStream is = new FileInputStream(path.toFile()))
           {
             is.read();
             canBeOpenedIo = true;
           }
         catch (IOException e)
           {
           }

         final String result =
             String.format("Files.exists(): %s, toFile.exists(): %s,  
canBeOpened NIO: %s canBeOpened IO: %s ",
                     filesExists, pathToFileExists, canBeOpenedNio,  
canBeOpenedIo)
                 + toDebugString(path.toString());

         assertThat(result + toDebugString(path.toString()), filesExists,  
is(true));
         assertThat(result + toDebugString(path.toString()),  
pathToFileExists, is(true));
         assertThat(result + toDebugString(path.toString()), canBeOpenedIo,  
is(true));
         assertThat(result + toDebugString(path.toString()),  
canBeOpenedNio, is(true));
       }

The Paths submitted to the test are discovered by Files.walk().

What I'm seeing is that all the troubled files show now no access problems  
with NIO, while IO fails (e.g. toFile().exists() returns false while  
Files.exists(path) returns true).

Message: Files.exists(): true, toFile.exists(): false, canBeOpened NIO:  
true canBeOpened IO: false  
/var/test-sets/MusicTestSets/iTunes-fg-20160504-1-iconv/Music/Ivo  
Pogorelich/Ravel_ Gaspard De La Nuit; Prokofiev_ Piano Sonata #6/03 Ravel_  
Gaspard De La Nuit - Scarbo_  
Mod??r??.mp3/var/test-sets/MusicTestSets/iTunes-fg-20160504-1-iconv/Music/Ivo  
Pogorelich/Ravel_ Gaspard De La Nuit; Prokofiev_ Piano Sonata #6/03 Ravel_  
Gaspard De La Nuit - Scarbo_ Mod??r??.mp3 Expected: is <true> but: was  
<false>

Message: Files.exists(): true, toFile.exists(): false, canBeOpened NIO:  
true canBeOpened IO: false  
/var/test-sets/MusicTestSets/iTunes-fg-20160504-1-noiconv/Music/Ivo  
Pogorelich/Ravel_ Gaspard De La Nuit; Prokofiev_ Piano Sonata #6/03 Ravel_  
Gaspard De La Nuit - Scarbo_  
Mode??re??.mp3/var/test-sets/MusicTestSets/iTunes-fg-20160504-1-noiconv/Music/Ivo  
Pogorelich/Ravel_ Gaspard De La Nuit; Prokofiev_ Piano Sonata #6/03 Ravel_  
Gaspard De La Nuit - Scarbo_ Mode??re??.mp3 Expected: is <true> but: was  
<false>


The following code thus can be used to probe a troubled Path (so far I see  
that the problem only happens with the EXT4 filesytem; no problems with  
e.g. BTRFS and HFS+):

     private static boolean isTroubled (final @Nonnull Path path)
       {
         return Files.exists(path) != path.toFile().exists();
       }

I was able to implement a first workaround for libraries I'm using that  
unfortunately are based on java.io.File: I create a temporary symbolic  
link without diacritics, use it in place of the original, then delete it  
after the processing, and it works.

There are other remaining troubles with file names, but I'd like to first  
have a feedback on this point.

Thanks.


--
Fabrizio Giudici - Java Architect @ Tidalwave s.a.s.
"We make Java work. Everywhere."
http://tidalwave.it/fabrizio/blog - fabrizio.giudici at tidalwave.it