From brian.burkhalter at oracle.com Wed Dec 2 01:45:15 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 1 Dec 2015 17:45:15 -0800 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: <5626EA1D.1070005@oracle.com> References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> Message-ID: Reprising this review after some time. On Oct 20, 2015, at 6:27 PM, Alan Bateman wrote: > On 21/10/2015 02:05, Brian Burkhalter wrote: >> Please review at your convenience. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8139133 >> Patch: http://cr.openjdk.java.net/~bpb/8139133/webrev.00/ >> >> The exception is caused by the native open(2) system call's failing with errno ENXIO, "No such device or address.? The fix is to use equivalent methods which do not depend on the file descriptor?s being available. >> > Catching IOException doesn't look right. Instead, I think this will need re-visiting openForAttributeAccess so that you can examine the errno and only fallback to the file path for ENXIO and maybe a few other errors. > > A minor comment is renaming isFdValid to haveFd could make the usages a bit clearer. > > Does the test have a race? It looks like it might attempt to accept the socket file before the nc command creates it. Also can the test be moved to attribute/BasicFileAttributeView as that is where the tests for setTimes are (it's not a test for the FileTime class). An updated version of the patch may be found here: http://cr.openjdk.java.net/~bpb/8139133/webrev.01/ Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Dec 2 07:56:23 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 2 Dec 2015 07:56:23 +0000 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> Message-ID: <565EA427.7090104@oracle.com> On 02/12/2015 01:45, Brian Burkhalter wrote: > An updated version of the patch may be found here: > > http://cr.openjdk.java.net/~bpb/8139133/webrev.01/ > > This results in FileSystemException bring throw with the JDK-internal UnixException as the cause so it means a non-accessible cause will leak out. In the previous round on this issue then I thought we were leaning towards changing openForAttributeAccess? -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Wed Dec 2 09:20:34 2015 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 2 Dec 2015 10:20:34 +0100 Subject: RFR 6856817: Poor performance of Writer#append with CharBuffer In-Reply-To: <5655C5B0.2050305@oracle.com> References: <5655C5B0.2050305@oracle.com> Message-ID: <565EB7E2.9060708@oracle.com> Hi Vyom, Looks good to me too. If you don't get more comments from reviewers, send me the exported changeset and I'll push it for you. best regards, -- daniel On 11/25/15 3:29 PM, vyom wrote: > Hi All, > > Please review my changes for below bug. > > Bug: JDK-6856817 : Poor performance of Writer#append with > CharBuffer > Webrev: http://cr.openjdk.java.net/~vtewari/6856817/webrev0.4/ > > > This change will avoid the redundant toString/CharBuffer.wrap in case > of OutputStreamWriter.append(CharBuffer). > > In case of OutputStreamWriter.append(CharBuffer) we are doing > CharBuffer->String->char[]->CharBuffer conversion this change will > avoid redundant conversion. > > Thanks, > Vyom > From brian.burkhalter at oracle.com Wed Dec 2 16:11:15 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 2 Dec 2015 08:11:15 -0800 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: <565EA427.7090104@oracle.com> References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> <565EA427.7090104@oracle.com> Message-ID: On Dec 1, 2015, at 11:56 PM, Alan Bateman wrote: > On 02/12/2015 01:45, Brian Burkhalter wrote: >> An updated version of the patch may be found here: >> >> http://cr.openjdk.java.net/~bpb/8139133/webrev.01/ >> > This results in FileSystemException bring throw with the JDK-internal UnixException as the cause so it means a non-accessible cause will leak out. Oh, you?re right: that would not be good. > In the previous round on this issue then I thought we were leaning towards changing openForAttributeAccess? That was one idea. I?ll investigate it. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Dec 3 01:51:44 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 2 Dec 2015 17:51:44 -0800 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: <565EA427.7090104@oracle.com> References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> <565EA427.7090104@oracle.com> Message-ID: On Dec 1, 2015, at 11:56 PM, Alan Bateman wrote: > On 02/12/2015 01:45, Brian Burkhalter wrote: >> An updated version of the patch may be found here: >> >> http://cr.openjdk.java.net/~bpb/8139133/webrev.01/ >> > This results in FileSystemException bring throw with the JDK-internal UnixException as the cause so it means a non-accessible cause will leak out. In the previous round on this issue then I thought we were leaning towards changing openForAttributeAccess? Here is an updated version with a modified openForAttributeAccess(): http://cr.openjdk.java.net/~bpb/8139133/webrev.02/ NIO regression tests pass on all platforms with this change applied. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Dec 3 08:55:46 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 3 Dec 2015 08:55:46 +0000 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> <565EA427.7090104@oracle.com> Message-ID: <56600392.30809@oracle.com> On 03/12/2015 01:51, Brian Burkhalter wrote: > > Here is an updated version with a modified openForAttributeAccess(): > > http://cr.openjdk.java.net/~bpb/8139133/webrev.02/ > > > NIO regression tests pass on all platforms with this change applied. > openForAttributeAccess taking an int[] parameters looks very hacky, can't we just change this to throw UnixException and let the callers access the errno. This is where I thought we were going in the initial rounds on this issue. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Dec 3 16:24:57 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 3 Dec 2015 08:24:57 -0800 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: <56600392.30809@oracle.com> References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> <565EA427.7090104@oracle.com> <56600392.30809@oracle.com> Message-ID: <1B9A2778-DF6C-47DC-B196-1FC74A6C443A@oracle.com> On Dec 3, 2015, at 12:55 AM, Alan Bateman wrote: > On 03/12/2015 01:51, Brian Burkhalter wrote: >> >> Here is an updated version with a modified openForAttributeAccess(): >> >> http://cr.openjdk.java.net/~bpb/8139133/webrev.02/ >> >> NIO regression tests pass on all platforms with this change applied. >> > openForAttributeAccess taking an int[] parameters looks very hacky, can't we just change this to throw UnixException and let the callers access the errno. This is where I thought we were going in the initial rounds on this issue. Sure I can change it to that. I was attempting to avoid replicating the code in the catch(UnixException) block of openForAttributeAccess(). Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Dec 4 01:56:53 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 3 Dec 2015 17:56:53 -0800 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: <1B9A2778-DF6C-47DC-B196-1FC74A6C443A@oracle.com> References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> <565EA427.7090104@oracle.com> <56600392.30809@oracle.com> <1B9A2778-DF6C-47DC-B196-1FC74A6C443A@oracle.com> Message-ID: On Dec 3, 2015, at 8:24 AM, Brian Burkhalter wrote: > Sure I can change it to that. I was attempting to avoid replicating the code in the catch(UnixException) block of openForAttributeAccess(). All right, here is another version of the fix: http://cr.openjdk.java.net/~bpb/8139133/webrev.03/ Having openForAttributeAccess() throw a UnixException required minor changes to several linux- and solaris-specific classes. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Sun Dec 6 07:17:50 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 6 Dec 2015 07:17:50 +0000 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> <565EA427.7090104@oracle.com> <56600392.30809@oracle.com> <1B9A2778-DF6C-47DC-B196-1FC74A6C443A@oracle.com> Message-ID: <5663E11E.4030405@oracle.com> On 04/12/2015 01:56, Brian Burkhalter wrote: > > : > > Having openForAttributeAccess() throw a UnixException required minor > changes to several linux- and solaris-specific classes. > Getting there, but I think the handling of ELOOP needs to move from openForAttributeAccess to UnixException::translateToIOException so that openForAttributeAccess can only throw UnixException. This is should get to look a bit cleaner. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve.drach at oracle.com Mon Dec 7 23:23:58 2015 From: steve.drach at oracle.com (Steve Drach) Date: Mon, 7 Dec 2015 15:23:58 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files Message-ID: Hi, Please review the changes and additions to ZipFileSystem to support multi-release jars as described in JEP 238 . The strategy is, if a file system is created over a multi-release jar, a map of root entries to versioned entries for a specific platform release version is constructed. The desired platform release version is configured through the the env parameter given to the FileSystem.newFileSystem method. Then when a root entry is requested, the map is consulted and if a mapping is found, the corresponding versioned entry is returned. Very similar to symbolic links. Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 Change to modules.xml: http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.00/index.html Changes to ZipFileSystem: http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.00/index.html Thanks, Steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Dec 8 01:09:49 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 7 Dec 2015 17:09:49 -0800 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: <5663E11E.4030405@oracle.com> References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> <565EA427.7090104@oracle.com> <56600392.30809@oracle.com> <1B9A2778-DF6C-47DC-B196-1FC74A6C443A@oracle.com> <5663E11E.4030405@oracle.com> Message-ID: On Dec 5, 2015, at 11:17 PM, Alan Bateman wrote: >> Having openForAttributeAccess() throw a UnixException required minor changes to several linux- and solaris-specific classes. >> > Getting there, but I think the handling of ELOOP needs to move from openForAttributeAccess to UnixException::translateToIOException so that openForAttributeAccess can only throw UnixException. This is should get to look a bit cleaner. OK so here?s one more go with the above change added: http://cr.openjdk.java.net/~bpb/8139133/webrev.04/ Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Dec 8 13:11:55 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 8 Dec 2015 13:11:55 +0000 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> <565EA427.7090104@oracle.com> <56600392.30809@oracle.com> <1B9A2778-DF6C-47DC-B196-1FC74A6C443A@oracle.com> <5663E11E.4030405@oracle.com> Message-ID: <5666D71B.5090109@oracle.com> On 08/12/2015 01:09, Brian Burkhalter wrote: > > OK so here?s one more go with the above change added: > > http://cr.openjdk.java.net/~bpb/8139133/webrev.04/ > > Good, this is where I assumed it was going when we initially discussed this one back in October. One thing we could do to make this a bit clearer is for close to be a no-op when called with -1. I realize this requires renaming the native methods so not a big issue if you decide not to do it in this patch. In setTimes then it might be a bit cleaner if you rename useFdToSetTimes to useFutimes. The test is still a concern, esp. the sleep. There are also a lot of catching of IOException that I assume should be removed to allow the test to fail. Also the message "Failed to remove" isn't quite right as deleteIfExists returns false when the file does not exist. I think you can remove the try-catch around this can be removed too. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Dec 9 01:03:57 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 8 Dec 2015 17:03:57 -0800 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: <5666D71B.5090109@oracle.com> References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> <565EA427.7090104@oracle.com> <56600392.30809@oracle.com> <1B9A2778-DF6C-47DC-B196-1FC74A6C443A@oracle.com> <5663E11E.4030405@oracle.com> <5666D71B.5090109@oracle.com> Message-ID: <5C77FED2-AFC9-4C96-B9C7-6E2F3DD9330A@oracle.com> On Dec 8, 2015, at 5:11 AM, Alan Bateman wrote: > Good, this is where I assumed it was going when we initially discussed this one back in October. Here?s yet another iteration: http://cr.openjdk.java.net/~bpb/8139133/webrev.05/ > One thing we could do to make this a bit clearer is for close to be a no-op when called with -1. I realize this requires renaming the native methods so not a big issue if you decide not to do it in this patch. UnixNativeDispatcher modified accordingly. > In setTimes then it might be a bit cleaner if you rename useFdToSetTimes to useFutimes. So changed. > The test is still a concern, esp. the sleep. Excised the sleep calls. The existence of the socket file is determined instead by creating it in a new subdirectory of the test directory and monitoring the subdirectory for new entries using a WatchService. Don?t know whether this is the best way to handle it but it works. > There are also a lot of catching of IOException that I assume should be removed to allow the test to fail. The test would still fail as the IOEs were re-thrown but I?ve cleaned them up anyway as it does look less messy. > Also the message "Failed to remove" isn't quite right as deleteIfExists returns false when the file does not exist. I think you can remove the try-catch around this can be removed too. Done. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Dec 10 10:35:11 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 10 Dec 2015 10:35:11 +0000 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: <5C77FED2-AFC9-4C96-B9C7-6E2F3DD9330A@oracle.com> References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> <565EA427.7090104@oracle.com> <56600392.30809@oracle.com> <1B9A2778-DF6C-47DC-B196-1FC74A6C443A@oracle.com> <5663E11E.4030405@oracle.com> <5666D71B.5090109@oracle.com> <5C77FED2-AFC9-4C96-B9C7-6E2F3DD9330A@oracle.com> Message-ID: <5669555F.4060600@oracle.com> On 09/12/2015 01:03, Brian Burkhalter wrote: > > UnixNativeDispatcher modified accordingly. This looks good now. To be consistent then close0 can be private and the declaration moved to below close (you'll see this convention is used in this source file). No need for an updated webrev on this. > : > >> The test is still a concern, esp. the sleep. > > Excised the sleep calls. The existence of the socket file is > determined instead by creating it in a new subdirectory of the test > directory and monitoring the subdirectory for new entries using a > WatchService. Don?t know whether this is the best way to handle it but > it works. > Looks better but I worry about the reliability of tests like this. I don't object to you pushing this, you just might want to fix up some of the odd line breaks to make it looks a bit nicer before pushing. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From fgaliegue at gmail.com Thu Dec 10 17:34:48 2015 From: fgaliegue at gmail.com (Francis Galiegue) Date: Thu, 10 Dec 2015 18:34:48 +0100 Subject: Clarification about ClosedFileSystemException Message-ID: Hello, The javadoc says: Unchecked exception thrown when an attempt is made to invoke an operation on a file and the file system is closed So, what it means that an operation is "made on a file"? What happens if, for instance, I have a FileSystem instance which is closed and I attempt to call, say, .newWatchService() on it? A FileSystem is not a file... -- Francis Galiegue, fgaliegue at gmail.com, https://github.com/fge JSON Schema in Java: http://json-schema-validator.herokuapp.com Parsers in pure Java: https://github.com/fge/grappa From brian.burkhalter at oracle.com Fri Dec 11 00:01:34 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 10 Dec 2015 16:01:34 -0800 Subject: [9] RFR of 8139133: Changing the modification time on a unix domain socket file fails In-Reply-To: <5669555F.4060600@oracle.com> References: <9910F845-0B46-4446-8320-B1BFBA19E017@oracle.com> <5626EA1D.1070005@oracle.com> <565EA427.7090104@oracle.com> <56600392.30809@oracle.com> <1B9A2778-DF6C-47DC-B196-1FC74A6C443A@oracle.com> <5663E11E.4030405@oracle.com> <5666D71B.5090109@oracle.com> <5C77FED2-AFC9-4C96-B9C7-6E2F3DD9330A@oracle.com> <5669555F.4060600@oracle.com> Message-ID: <68840859-4BDB-4FB7-90AC-3B20C40A5DE8@oracle.com> On Dec 10, 2015, at 2:35 AM, Alan Bateman wrote: >> UnixNativeDispatcher modified accordingly. > This looks good now. To be consistent then close0 can be private and the declaration moved to below close (you'll see this convention is used in this source file). No need for an updated webrev on this. Modified as suggested. >> Excised the sleep calls. The existence of the socket file is determined instead by creating it in a new subdirectory of the test directory and monitoring the subdirectory for new entries using a WatchService. Don?t know whether this is the best way to handle it but it works. >> > Looks better but I worry about the reliability of tests like this. I don't object to you pushing this, you just might want to fix up some of the odd line breaks to make it looks a bit nicer before pushing. Went back over the code and added a number of comments and format improvement; looks much better now. The patch has been pushed. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Dec 11 01:29:28 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 10 Dec 2015 17:29:28 -0800 Subject: [9] RFR of 8133986: (fs) Remove file deletion from test/java/nio/file/FileSystem/Basic.java checkNoUOE() method In-Reply-To: References: Message-ID: This RFR was neither approved nor rejected. Thanks, Brian On Aug 19, 2015, at 5:06 PM, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8133986 > Diff: > > --- a/test/java/nio/file/FileSystem/Basic.java > +++ b/test/java/nio/file/FileSystem/Basic.java > @@ -67,7 +67,6 @@ > } catch (ProviderNotFoundException pnfe) { > System.out.println("Expected ProviderNotFoundException caught: " > + "\"" + pnfe.getMessage() + "\""); > - } finally { > FileUtils.deleteFileWithRetry(path); > } > } > b > > Summary: Only delete files for which an unexpected exception did not occur. > > Thanks, > > Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Dec 11 06:39:35 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 11 Dec 2015 06:39:35 +0000 Subject: [9] RFR of 8133986: (fs) Remove file deletion from test/java/nio/file/FileSystem/Basic.java checkNoUOE() method In-Reply-To: References: Message-ID: <566A6FA7.60501@oracle.com> I assume the usage of FileUtils.deleteFileWithRetry can simply be removed, it shouldn't be needed here. -Alan On 11/12/2015 01:29, Brian Burkhalter wrote: > This RFR was neither approved nor rejected. > > Thanks, > > Brian > > On Aug 19, 2015, at 5:06 PM, Brian Burkhalter > > wrote: > >> Please review at your convenience. >> >> Issue:https://bugs.openjdk.java.net/browse/JDK-8133986 >> Diff: >> >> --- a/test/java/nio/file/FileSystem/Basic.java >> +++ b/test/java/nio/file/FileSystem/Basic.java >> @@ -67,7 +67,6 @@ >> } catch (ProviderNotFoundException pnfe) { >> System.out.println("Expected ProviderNotFoundException >> caught: " >> + "\"" + pnfe.getMessage() + "\""); >> - } finally { >> FileUtils.deleteFileWithRetry(path); >> } >> } >> b >> >> Summary: Only delete files for which an unexpected exception did not >> occur. >> >> Thanks, >> >> Brian > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vyom.tewari at oracle.com Fri Dec 11 06:46:58 2015 From: vyom.tewari at oracle.com (Vyom Tewari) Date: Fri, 11 Dec 2015 12:16:58 +0530 Subject: RFR 4823133: RandomAccessFile.length() is not thread-safe In-Reply-To: <56696EF8.4060800@oracle.com> References: <562A9DE1.4070208@oracle.com> <56695984.7030204@oracle.com> <56696EF8.4060800@oracle.com> Message-ID: <566A7162.2060805@oracle.com> +nio-dev On 12/10/2015 5:54 PM, David Holmes wrote: > Hi Vyom, > > Should this be reviewed by the NIO folk? It isn't obvious whether this > change impacts the interaction with the associated FileChannel - see > also https://bugs.openjdk.java.net/browse/JDK-6371642 > > Thanks, > David > > On 10/12/2015 8:52 PM, vyom wrote: >> Hi All, >> >> Please review my changes for below bug. >> >> Bug: JDK-4823133 : RandomAccessFile.length() is not thread-safe >> >> Webrev:http://cr.openjdk.java.net/~vtewari/4823133/webrev0.0/ >> >> >> This change ensure that length() does not temporarily changes the file >> pointer and it will make sure that there is no race >> condition in case of multi thread uses. >> >> Thanks, >> Vyom >> >> >> >> From brian.burkhalter at oracle.com Fri Dec 11 16:13:49 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 11 Dec 2015 08:13:49 -0800 Subject: [9] RFR of 8133986: (fs) Remove file deletion from test/java/nio/file/FileSystem/Basic.java checkNoUOE() method In-Reply-To: <566A6FA7.60501@oracle.com> References: <566A6FA7.60501@oracle.com> Message-ID: OK, here?s that version: diff --git a/test/java/nio/file/FileSystem/Basic.java b/test/java/nio/file/FileSystem/Basic.java --- a/test/java/nio/file/FileSystem/Basic.java +++ b/test/java/nio/file/FileSystem/Basic.java @@ -67,8 +67,6 @@ } catch (ProviderNotFoundException pnfe) { System.out.println("Expected ProviderNotFoundException caught: " + "\"" + pnfe.getMessage() + "\""); - } finally { - FileUtils.deleteFileWithRetry(path); } } Thanks, Brian On Dec 10, 2015, at 10:39 PM, Alan Bateman wrote: > I assume the usage of FileUtils.deleteFileWithRetry can simply be removed, it shouldn't be needed here. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fgaliegue at gmail.com Sat Dec 12 23:03:51 2015 From: fgaliegue at gmail.com (Francis Galiegue) Date: Sun, 13 Dec 2015 00:03:51 +0100 Subject: Path's .relativize documentation is wrong on a crucial point Message-ID: Let us have a FileSystemProvider which provides FileSystems with Unix-like Paths. Let us have two FileSystems from that same provider fs1 and fs2. Let us have p issued from FileSystem fs1 which is "a". Let us have q issued from FileSystem fs2 which is "b". Let us call r the result of p.resolve(q). Since this is a Unix-like filesystem, none of p and q have root components (that would be /), and the resulting path r would be "a/b", on filesystem fs1. Now, let us do: p.relativize(r) Both p (which is "a") and r (which is "a/b") are on fs1; the result is "b", which is on... fs1. Which contradicts the equality of the Path's .relativize() which says that: ---- For any two normalized paths p and q, where q does not have a root component, p.relativize(p.resolve(q)).equals(q) ---- This is clearly not true in this case. Path's .equals() says it explicitly. And this was easy to demonstrate, without even an implementation... Now, case in point: I have implemented JSR 203 for both DropBox and FTP where this just cannot be true: I can issue several FileSystems from the same providers (one for DropBox, the other for FTP) for which even _trying to resolve_ Paths from different FileSystems but the same provider does not make any sense at all (you may end up accessing _the wrong entity altogether_); this is why I define FileSystemMismatchException. I consider this to be a bug in the API definition. What do you think? Regards, -- Francis Galiegue, fgaliegue at gmail.com, https://github.com/fge JSON Schema in Java: http://json-schema-validator.herokuapp.com Parsers in pure Java: https://github.com/fge/grappa From Alan.Bateman at oracle.com Sun Dec 13 07:00:54 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 13 Dec 2015 07:00:54 +0000 Subject: Path's .relativize documentation is wrong on a crucial point In-Reply-To: References: Message-ID: <566D17A6.2040203@oracle.com> On 12/12/2015 23:03, Francis Galiegue wrote: > Let us have a FileSystemProvider which provides FileSystems with > Unix-like Paths. > > Let us have two FileSystems from that same provider fs1 and fs2. > > Let us have p issued from FileSystem fs1 which is "a". > > Let us have q issued from FileSystem fs2 which is "b". > > Let us call r the result of p.resolve(q). Since this is a Unix-like > filesystem, none of p and q have root components (that would be /), > and the resulting path r would be "a/b", on filesystem fs1. > > Now, let us do: > > p.relativize(r) > > Both p (which is "a") and r (which is "a/b") are on fs1; the result is > "b", which is on... fs1. > > Which contradicts the equality of the Path's .relativize() which says that: > > ---- > > For any two normalized paths p and q, where q does not have a root component, > > p.relativize(p.resolve(q)).equals(q) > ---- > > Yes, there is a spec issue here as this should make it clear that and paths p and q are the same FileSystem. I will create a bug for this, assuming we don't have one already. -Alan From fgaliegue at gmail.com Mon Dec 14 15:09:29 2015 From: fgaliegue at gmail.com (Francis Galiegue) Date: Mon, 14 Dec 2015 16:09:29 +0100 Subject: Path's .relativize documentation is wrong on a crucial point In-Reply-To: <566D17A6.2040203@oracle.com> References: <566D17A6.2040203@oracle.com> Message-ID: Hello, On Sun, Dec 13, 2015 at 8:00 AM, Alan Bateman wrote: [...] >> >> > Yes, there is a spec issue here as this should make it clear that and paths > p and q are the same FileSystem. I will create a bug for this, assuming we > don't have one already. > There is one more method in FileSystemProvider which is also making me pause: .copy(). Theoretically, it can be used if and only if two paths are from the same provider; and in my case we get into the situation where even that does not make any sense. If my two filesystems are different dropbox accounts for instance, I cannot use the direct copy command from the dropbox sdk; I have to go through a plain InputStream to OutputStream copy. -- Francis Galiegue, fgaliegue at gmail.com, https://github.com/fge JSON Schema in Java: http://json-schema-validator.herokuapp.com Parsers in pure Java: https://github.com/fge/grappa From Alan.Bateman at oracle.com Mon Dec 14 15:17:52 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 14 Dec 2015 15:17:52 +0000 Subject: Path's .relativize documentation is wrong on a crucial point In-Reply-To: References: <566D17A6.2040203@oracle.com> Message-ID: <566EDDA0.803@oracle.com> On 14/12/2015 15:09, Francis Galiegue wrote: > : > There is one more method in FileSystemProvider which is also making me > pause: .copy(). > > Theoretically, it can be used if and only if two paths are from the > same provider; and in my case we get into the situation where even > that does not make any sense. > > If my two filesystems are different dropbox accounts for instance, I > cannot use the direct copy command from the dropbox sdk; I have to go > through a plain InputStream to OutputStream copy. > Is this a problem? Files.copy will delegate to the FileSystemProvider when the source and target are the same provider. It is up to the file system provider as to how the copy is done. -Alan From fgaliegue at gmail.com Mon Dec 14 15:24:11 2015 From: fgaliegue at gmail.com (Francis Galiegue) Date: Mon, 14 Dec 2015 16:24:11 +0100 Subject: Path's .relativize documentation is wrong on a crucial point In-Reply-To: <566EDDA0.803@oracle.com> References: <566D17A6.2040203@oracle.com> <566EDDA0.803@oracle.com> Message-ID: On Mon, Dec 14, 2015 at 4:17 PM, Alan Bateman wrote: > > On 14/12/2015 15:09, Francis Galiegue wrote: >> >> : >> There is one more method in FileSystemProvider which is also making me >> pause: .copy(). >> >> Theoretically, it can be used if and only if two paths are from the >> same provider; and in my case we get into the situation where even >> that does not make any sense. >> >> If my two filesystems are different dropbox accounts for instance, I >> cannot use the direct copy command from the dropbox sdk; I have to go >> through a plain InputStream to OutputStream copy. >> > Is this a problem? Files.copy will delegate to the FileSystemProvider when > the source and target are the same provider. It is up to the file system > provider as to how the copy is done. > Yes, I know; I did exactly that. Still, this is not very practical to having to replicate that mechanism so many times... But speaking of this same method again, here's another scenario: * you have a fs wil more than one filestore; * you have path p1 from filestore s1 which is read write; * you have path p2 from filestore s2 which is read only; * you try and Files.copy(p1, p2). What is the exception here? It cannot be ReadOnlyFileSystemException since at least one file store from the filesystem is read write... -- Francis Galiegue, fgaliegue at gmail.com, https://github.com/fge JSON Schema in Java: http://json-schema-validator.herokuapp.com Parsers in pure Java: https://github.com/fge/grappa From Alan.Bateman at oracle.com Mon Dec 14 15:40:32 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 14 Dec 2015 15:40:32 +0000 Subject: Path's .relativize documentation is wrong on a crucial point In-Reply-To: References: <566D17A6.2040203@oracle.com> <566EDDA0.803@oracle.com> Message-ID: <566EE2F0.4000601@oracle.com> On 14/12/2015 15:24, Francis Galiegue wrote: > > But speaking of this same method again, here's another scenario: > > * you have a fs wil more than one filestore; > * you have path p1 from filestore s1 which is read write; > * you have path p2 from filestore s2 which is read only; > * you try and Files.copy(p1, p2). > > What is the exception here? It cannot be ReadOnlyFileSystemException > since at least one file store from the filesystem is read write... > There isn't a specific FileSystemException for this case. Can you throw FileSystemException with a useful message? That is what the default provider does. -Alan From fgaliegue at gmail.com Mon Dec 14 15:56:48 2015 From: fgaliegue at gmail.com (Francis Galiegue) Date: Mon, 14 Dec 2015 16:56:48 +0100 Subject: Path's .relativize documentation is wrong on a crucial point In-Reply-To: <566EE2F0.4000601@oracle.com> References: <566D17A6.2040203@oracle.com> <566EDDA0.803@oracle.com> <566EE2F0.4000601@oracle.com> Message-ID: Hello, On Mon, Dec 14, 2015 at 4:40 PM, Alan Bateman wrote: [...] >> > There isn't a specific FileSystemException for this case. Can you throw > FileSystemException with a useful message? That is what the default provider > does. > Well, right now my API is limited to one FileStore per FileSystem, so the problem does not apply to me (I was asking the question on theoretical grounds only); similarly I only support one root. Regards, -- Francis Galiegue, fgaliegue at gmail.com, https://github.com/fge JSON Schema in Java: http://json-schema-validator.herokuapp.com Parsers in pure Java: https://github.com/fge/grappa From steve.drach at oracle.com Mon Dec 14 17:14:02 2015 From: steve.drach at oracle.com (Steve Drach) Date: Mon, 14 Dec 2015 09:14:02 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files Message-ID: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> Hi, I sent this out a week ago. Resending to bump it to the top of the stack. Please review the changes and additions to ZipFileSystem to support multi-release jars as described in JEP 238 . The strategy is, if a file system is created over a multi-release jar, a map of root entries to versioned entries for a specific platform release version is constructed. The desired platform release version is configured through the the env parameter given to the FileSystem.newFileSystem method. Then when a root entry is requested, the map is consulted and if a mapping is found, the corresponding versioned entry is returned. Very similar to symbolic links. Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 Change to modules.xml: http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.00/index.html Changes to ZipFileSystem: http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.00/index.html Thanks, Steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From xueming.shen at oracle.com Mon Dec 14 17:46:47 2015 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 14 Dec 2015 09:46:47 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> Message-ID: <566F0087.8040000@oracle.com> Hi Steve, (1) ZipFileSystem.getParent() might no longer need to be changed to "package private"? I don't see it is used by other. (2) For JarFileSystem.isMultiReleaseJar(), the alternative is to use the public api directly try (InputStream is = getInputStream(getBytes("META-INF/MANIFEST.MF"))) { new Manifest(is).getMainAttributes().containsKey(new Attributes.Name("Multi-Release")); } catch (Exception x) {} return false; In this approach, the getInputStream(Entry) can be put back to "private" as well. But this is just a personal preference, your choice:-) The rest looks fine for me Thanks, Sherman On 12/14/15 9:14 AM, Steve Drach wrote: > Hi, > > I sent this out a week ago. Resending to bump it to the top of the > stack. > > Please review the changes and additions to ZipFileSystem to support > multi-release jars as described in JEP 238 > . > > The strategy is, if a file system is created over a multi-release jar, > a map of root entries to versioned entries for a specific platform > release version is constructed. The desired platform release version > is configured through the the env parameter given to the > FileSystem.newFileSystem method. Then when a root entry is requested, > the map is consulted and if a mapping is found, the corresponding > versioned entry is returned. Very similar to symbolic links. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 > > Change to modules.xml: > http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.00/index.html > > > Changes to ZipFileSystem: > http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.00/index.html > > > Thanks, > Steve > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve.drach at oracle.com Mon Dec 14 19:50:35 2015 From: steve.drach at oracle.com (Steve Drach) Date: Mon, 14 Dec 2015 11:50:35 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <566F0087.8040000@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> Message-ID: <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> Hi Sherman, Thanks for looking at this. > On Dec 14, 2015, at 9:46 AM, Xueming Shen wrote: > > Hi Steve, > > (1) ZipFileSystem.getParent() might no longer need to be changed to "package private"? I don't > see it is used by other. Yes, thanks, I missed that. > (2) For JarFileSystem.isMultiReleaseJar(), the alternative is to use the public api directly > > try (InputStream is = getInputStream(getBytes("META-INF/MANIFEST.MF"))) { I can?t seem to find a getInputStream(byte[]). I only see a getInputStream(Entry). Are we looking at the same code? > new Manifest(is).getMainAttributes().containsKey(new Attributes.Name("Multi-Release")); > } catch (Exception x) {} > return false; > > In this approach, the getInputStream(Entry) can be put back to "private" as well. > But this is just a personal preference, your choice:-) > > The rest looks fine for me > > Thanks, > Sherman > > On 12/14/15 9:14 AM, Steve Drach wrote: >> Hi, >> >> I sent this out a week ago. Resending to bump it to the top of the stack. >> >> Please review the changes and additions to ZipFileSystem to support multi-release jars as described in JEP 238 . >> >> The strategy is, if a file system is created over a multi-release jar, a map of root entries to versioned entries for a specific platform release version is constructed. The desired platform release version is configured through the the env parameter given to the FileSystem.newFileSystem method. Then when a root entry is requested, the map is consulted and if a mapping is found, the corresponding versioned entry is returned. Very similar to symbolic links. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 >> >> Change to modules.xml: http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.00/index.html >> >> Changes to ZipFileSystem: http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.00/index.html >> >> Thanks, >> Steve >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xueming.shen at oracle.com Mon Dec 14 20:07:24 2015 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 14 Dec 2015 12:07:24 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> Message-ID: <566F217C.7090000@oracle.com> On 12/14/2015 11:50 AM, Steve Drach wrote: > > >> (2) For JarFileSystem.isMultiReleaseJar(), the alternative is to use the public api directly >> >> try (InputStream is = getInputStream(getBytes("META-INF/MANIFEST.MF"))) { > > I can?t seem to find a getInputStream(byte[]). I only see a getInputStream(Entry). Are we looking at the same code? > My apology. I meant "newInputStream(byte[])" -sherman >> new Manifest(is).getMainAttributes().containsKey(new Attributes.Name("Multi-Release")); >> } catch (Exception x) {} >> return false; >> >> In this approach, the getInputStream(Entry) can be put back to "private" as well. >> But this is just a personal preference, your choice:-) >> >> The rest looks fine for me >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve.drach at oracle.com Mon Dec 14 20:05:55 2015 From: steve.drach at oracle.com (Steve Drach) Date: Mon, 14 Dec 2015 12:05:55 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <566F217C.7090000@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <566F217C.7090000@oracle.com> Message-ID: <089FFBFE-906B-4D31-9318-75DF4561187B@oracle.com> >>> (2) For JarFileSystem.isMultiReleaseJar(), the alternative is to use the public api directly >>> >>> try (InputStream is = getInputStream(getBytes("META-INF/MANIFEST.MF"))) { >> >> I can?t seem to find a getInputStream(byte[]). I only see a getInputStream(Entry). Are we looking at the same code? >> > > My apology. I meant "newInputStream(byte[])? I see it now. Thanks. From steve.drach at oracle.com Tue Dec 15 00:08:22 2015 From: steve.drach at oracle.com (Steve Drach) Date: Mon, 14 Dec 2015 16:08:22 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> Message-ID: Hi, I?ve updated the webrevs to take into account Sherman?s comments. Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 Change to modules.xml: http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.01/index.html Changes to ZipFileSystem: http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.01/index.html Thanks, Steve >> On Dec 14, 2015, at 9:46 AM, Xueming Shen > wrote: >> >> Hi Steve, >> >> (1) ZipFileSystem.getParent() might no longer need to be changed to "package private"? I don't >> see it is used by other. > >> (2) For JarFileSystem.isMultiReleaseJar(), the alternative is to use the public api directly >> >> try (InputStream is = getInputStream(getBytes("META-INF/MANIFEST.MF"))) { > >> new Manifest(is).getMainAttributes().containsKey(new Attributes.Name("Multi-Release")); >> } catch (Exception x) {} >> return false; >> >> In this approach, the getInputStream(Entry) can be put back to "private" as well. >> But this is just a personal preference, your choice:-) >> >> The rest looks fine for me >> >> Thanks, >> Sherman >> >> On 12/14/15 9:14 AM, Steve Drach wrote: >>> Hi, >>> >>> I sent this out a week ago. Resending to bump it to the top of the stack. >>> >>> Please review the changes and additions to ZipFileSystem to support multi-release jars as described in JEP 238 . >>> >>> The strategy is, if a file system is created over a multi-release jar, a map of root entries to versioned entries for a specific platform release version is constructed. The desired platform release version is configured through the the env parameter given to the FileSystem.newFileSystem method. Then when a root entry is requested, the map is consulted and if a mapping is found, the corresponding versioned entry is returned. Very similar to symbolic links. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 >>> >>> Change to modules.xml: http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.00/index.html >>> >>> Changes to ZipFileSystem: http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.00/index.html >>> >>> Thanks, >>> Steve >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fgaliegue at gmail.com Wed Dec 16 13:44:30 2015 From: fgaliegue at gmail.com (Francis Galiegue) Date: Wed, 16 Dec 2015 14:44:30 +0100 Subject: Files.createFile() makes use of newByteChannel(); what if the fs does _not_ support .newByteChannel()? Message-ID: Hello, All is in the question really... The javadoc of .newByteChannel() on the provider says that UnsupportedOperationException will be thrown if one OpenOption is not supported, but since the filesystems I develop cannot return any meaningful form of SeekableByteChannels at all, I always throw UnsupportedOperationException... -- Francis Galiegue, fgaliegue at gmail.com, https://github.com/fge JSON Schema in Java: http://json-schema-validator.herokuapp.com Parsers in pure Java: https://github.com/fge/grappa From xueming.shen at oracle.com Wed Dec 16 22:49:35 2015 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 16 Dec 2015 14:49:35 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> Message-ID: <5671EA7F.9040009@oracle.com> looks good for me. -sherman On 12/14/15, 4:08 PM, Steve Drach wrote: > Hi, > > I?ve updated the webrevs to take into account Sherman?s comments. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 > > Change to modules.xml: > http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.01/index.html > > > Changes to ZipFileSystem: > http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.01/index.html > > > Thanks, > Steve > >>> On Dec 14, 2015, at 9:46 AM, Xueming Shen >> > wrote: >>> >>> Hi Steve, >>> >>> (1) ZipFileSystem.getParent() might no longer need to be changed to >>> "package private"? I don't >>> see it is used by other. >>> (2) For JarFileSystem.isMultiReleaseJar(), the alternative is to use >>> the public api directly >>> >>> try (InputStream is = >>> getInputStream(getBytes("META-INF/MANIFEST.MF"))) { >>> new Manifest(is).getMainAttributes().containsKey(new >>> Attributes.Name("Multi-Release")); >>> } catch (Exception x) {} >>> return false; >>> >>> In this approach, the getInputStream(Entry) can be put back to >>> "private" as well. >>> But this is just a personal preference, your choice:-) >>> >>> The rest looks fine for me >>> >>> Thanks, >>> Sherman >>> >>> On 12/14/15 9:14 AM, Steve Drach wrote: >>>> Hi, >>>> >>>> I sent this out a week ago. Resending to bump it to the top of >>>> the stack. >>>> >>>> Please review the changes and additions to ZipFileSystem to support >>>> multi-release jars as described in JEP 238 >>>> . >>>> >>>> The strategy is, if a file system is created over a multi-release >>>> jar, a map of root entries to versioned entries for a specific >>>> platform release version is constructed. The desired platform >>>> release version is configured through the the env parameter given >>>> to the FileSystem.newFileSystem method. Then when a root entry is >>>> requested, the map is consulted and if a mapping is found, the >>>> corresponding versioned entry is returned. Very similar to >>>> symbolic links. >>>> >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 >>>> >>>> Change to modules.xml: >>>> http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.00/index.html >>>> >>>> >>>> Changes to ZipFileSystem: >>>> http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.00/index.html >>>> >>>> >>>> Thanks, >>>> Steve >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve.drach at oracle.com Fri Dec 18 02:20:05 2015 From: steve.drach at oracle.com (Steve Drach) Date: Thu, 17 Dec 2015 18:20:05 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> Message-ID: <53A15F43-CCA6-4462-9D89-C8DE14D2CFF8@oracle.com> Hi, jprt found a bug in one of my tests when run on windows. The fix is simple as you can see in the patch below. I also took the opportunity to add a comment to a line in the source code. Please let me know if these patches can be added to the changeset. Thanks Steve diff --git a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java @@ -55,7 +55,8 @@ JarFileSystem(ZipFileSystemProvider provider, Path zfpath, Map env) throws IOException { super(provider, zfpath, env); - lookup = path -> path; + lookup = path -> path; // lookup needs to be set before isMultiReleaseJar is called because + // it eventually calls getEntry0 if (isMultiReleaseJar()) { int version; Object o = env.get("multi-release"); diff --git a/test/jdk/nio/zipfs/MultiReleaseJarTest.java b/test/jdk/nio/zipfs/MultiReleaseJarTest.java --- a/test/jdk/nio/zipfs/MultiReleaseJarTest.java +++ b/test/jdk/nio/zipfs/MultiReleaseJarTest.java @@ -65,9 +65,15 @@ creator.buildUnversionedJar(); creator.buildMultiReleaseJar(); creator.buildShortMultiReleaseJar(); - uvuri = new URI("jar:file:" + userdir + "/unversioned.jar"); - mruri = new URI("jar:file:" + userdir + "/multi-release.jar"); - smruri = new URI("jar:file:" + userdir + "/short-multi-release.jar"); + String udir; + if (System.getProperty("os.name").startsWith("Windows")) { + udir = "/" + userdir.replace('\\','/'); + } else { + udir = userdir; + } + uvuri = new URI("jar:file:" + udir + "/unversioned.jar"); + mruri = new URI("jar:file:" + udir + "/multi-release.jar"); + smruri = new URI("jar:file:" + udir + "/short-multi-release.jar"); entryName = className.replace('.', '/') + ".class"; } > Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 > > Change to modules.xml: http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.01/index.html > > Changes to ZipFileSystem: http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.01/index.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Fri Dec 18 08:28:44 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 18 Dec 2015 09:28:44 +0100 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <53A15F43-CCA6-4462-9D89-C8DE14D2CFF8@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <53A15F43-CCA6-4462-9D89-C8DE14D2CFF8@oracle.com> Message-ID: <201ED7E0-6205-4A1F-97EC-56B5BEF0E1F1@oracle.com> HI Steve, I find the use of the system property awkward. Did you try doing: new File(...).toURI() ? I noticed you are already creating Path instances for deletion, so it might be even better to do Path.toURI() e.g.: uvuri = ?jar:? + Paths.get(userdir, "unversioned.jar?).toURI(); rather than: uvuri = ?jar:? + new FIle(userdir, "unversioned.jar?).toURI(); Paul. > On 18 Dec 2015, at 03:20, Steve Drach wrote: > > Hi, > > jprt found a bug in one of my tests when run on windows. The fix is simple as you can see in the patch below. I also took the opportunity to add a comment to a line in the source code. Please let me know if these patches can be added to the changeset. > > Thanks > Steve > > diff --git a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java > --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java > +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java > @@ -55,7 +55,8 @@ > > JarFileSystem(ZipFileSystemProvider provider, Path zfpath, Map env) throws IOException { > super(provider, zfpath, env); > - lookup = path -> path; > + lookup = path -> path; // lookup needs to be set before isMultiReleaseJar is called because > + // it eventually calls getEntry0 > if (isMultiReleaseJar()) { > int version; > Object o = env.get("multi-release"); > diff --git a/test/jdk/nio/zipfs/MultiReleaseJarTest.java b/test/jdk/nio/zipfs/MultiReleaseJarTest.java > --- a/test/jdk/nio/zipfs/MultiReleaseJarTest.java > +++ b/test/jdk/nio/zipfs/MultiReleaseJarTest.java > @@ -65,9 +65,15 @@ > creator.buildUnversionedJar(); > creator.buildMultiReleaseJar(); > creator.buildShortMultiReleaseJar(); > - uvuri = new URI("jar:file:" + userdir + "/unversioned.jar"); > - mruri = new URI("jar:file:" + userdir + "/multi-release.jar"); > - smruri = new URI("jar:file:" + userdir + "/short-multi-release.jar"); > + String udir; > + if (System.getProperty("os.name").startsWith("Windows")) { > + udir = "/" + userdir.replace('\\','/'); > + } else { > + udir = userdir; > + } > + uvuri = new URI("jar:file:" + udir + "/unversioned.jar"); > + mruri = new URI("jar:file:" + udir + "/multi-release.jar"); > + smruri = new URI("jar:file:" + udir + "/short-multi-release.jar"); > entryName = className.replace('.', '/') + ".class"; > } > >> Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 >> >> Change to modules.xml: http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.01/index.html >> >> Changes to ZipFileSystem: http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.01/index.html > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail URL: From Alan.Bateman at oracle.com Fri Dec 18 09:00:25 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 18 Dec 2015 09:00:25 +0000 Subject: Files.createFile() makes use of newByteChannel(); what if the fs does _not_ support .newByteChannel()? In-Reply-To: References: Message-ID: <5673CB29.1010402@oracle.com> On 16/12/2015 13:44, Francis Galiegue wrote: > Hello, > > All is in the question really... > > The javadoc of .newByteChannel() on the provider says that > UnsupportedOperationException will be thrown if one OpenOption is not > supported, but since the filesystems I develop cannot return any > meaningful form of SeekableByteChannels at all, I always throw > UnsupportedOperationException... The original intention was that all providers must implement newByteChannel and the open options listed in the table. In the degnerative case then the underlying file system should at least support reading a file sequentially, or when it's a read-write file system that is supports writing to a file sequentially. Seeking might mean the position(long) method fails of course. Also Files.create is specified to be atomic with respect to other file system operations and that might be challenge too. It would be good if the javadoc could specify the absolute minimum but the original intention was not that newByteChannel always throws UOE. -Alan. From Alan.Bateman at oracle.com Fri Dec 18 13:34:58 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 18 Dec 2015 13:34:58 +0000 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> Message-ID: <56740B82.2030901@oracle.com> On 15/12/2015 00:08, Steve Drach wrote: > Hi, > > I?ve updated the webrevs to take into account Sherman?s comments. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 > > Change to modules.xml: > http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.01/index.html > > > Changes to ZipFileSystem: > http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.01/index.html > > modules.xml looks fine. The changes to ZipFileSystem* looks okay although I wonder if getEntry0 should be renamed to getEntry as it's cleaner when overridden in JarFileSystem. JarFileSystem looks okay and I assume the recursion in walk should never be an issue. A minor nit is that is that it would be good to keep the style consistent with the existing code if you can. Really long source lines are annoying when looking at side-by-side diffs. Also good if the main comment in JarFileSystem could be expanded a bit, also the comments in the methods can use /** instead of /*. The tests looks good. You might want to check the copyright header on MultiReleaseJarTest, the others look okay. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fgaliegue at gmail.com Fri Dec 18 19:43:31 2015 From: fgaliegue at gmail.com (Francis Galiegue) Date: Fri, 18 Dec 2015 20:43:31 +0100 Subject: Files.createFile() makes use of newByteChannel(); what if the fs does _not_ support .newByteChannel()? In-Reply-To: <5673CB29.1010402@oracle.com> References: <5673CB29.1010402@oracle.com> Message-ID: Hello, On Fri, Dec 18, 2015 at 10:00 AM, Alan Bateman wrote: [...] > > The original intention was that all providers must implement newByteChannel > and the open options listed in the table. In the degnerative case then the > underlying file system should at least support reading a file sequentially, > or when it's a read-write file system that is supports writing to a file > sequentially. Seeking might mean the position(long) method fails of course. > Also Files.create is specified to be atomic with respect to other file > system operations and that might be challenge too. It would be good if the > javadoc could specify the absolute minimum but the original intention was > not that newByteChannel always throws UOE. > > -Alan. Hmm, OK. I'm not here yet (I just want to have a first version of all filesystems working under the same API) but I'll keep that in mind... Have fun! Regards, -- Francis Galiegue, fgaliegue at gmail.com, https://github.com/fge JSON Schema in Java: http://json-schema-validator.herokuapp.com Parsers in pure Java: https://github.com/fge/grappa From steve.drach at oracle.com Fri Dec 18 22:42:55 2015 From: steve.drach at oracle.com (Steve Drach) Date: Fri, 18 Dec 2015 14:42:55 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <201ED7E0-6205-4A1F-97EC-56B5BEF0E1F1@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <53A15F43-CCA6-4462-9D89-C8DE14D2CFF8@oracle.com> <201ED7E0-6205-4A1F-97EC-56B5BEF0E1F1@oracle.com> Message-ID: <3851ECF7-7E03-44B1-9489-63B7EA6095B9@oracle.com> Hi, I updated the patch to the changeset to address Paul?s concern with a windows specific solution, as well some of Alan?s concerns. Please let me know if this patch (instead of the previous one) can be applied to the changeset. Thanks Steve diff --git a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java @@ -37,8 +37,12 @@ import java.util.jar.Attributes; import java.util.jar.Manifest; -/* - * Adds aliasing to ZipFileSystem to support multi-release jar files +/** + * Adds aliasing to ZipFileSystem to support multi-release jar files. An alias map + * is created by {@link JarFileSystem#createVersionedLinks(int)}. The map is then + * consulted when an entry is looked up in {@link JarFileSystem#getEntry0(byte[])} + * to determine if the entry has a corresponding versioned entry. If so, the + * versioned entry is returned. * * @author Steve Drach */ @@ -55,7 +59,8 @@ JarFileSystem(ZipFileSystemProvider provider, Path zfpath, Map env) throws IOException { super(provider, zfpath, env); - lookup = path -> path; + lookup = path -> path; // lookup needs to be set before isMultiReleaseJar is called because + // it eventually calls getEntry0 if (isMultiReleaseJar()) { int version; Object o = env.get("multi-release"); @@ -87,7 +92,7 @@ } } - /* + /** * create a map of aliases for versioned entries, for example: * version/PackagePrivate.class -> META-INF/versions/9/version/PackagePrivate.class * version/PackagePrivate.java -> META-INF/versions/9/version/PackagePrivate.java @@ -117,7 +122,7 @@ return path -> aliasMap.get(IndexNode.keyOf(path)); } - /* + /** * create a sorted version map of version -> inode, for inodes <= max version * 9 -> META-INF/versions/9 * 10 -> META-INF/versions/10 @@ -135,7 +140,7 @@ return map; } - /* + /** * extract the integer version number -- META-INF/versions/9 returns 9 */ private Integer getVersion(byte[] name, int offset) { @@ -147,7 +152,7 @@ } } - /* + /** * walk the IndexNode tree processing all leaf nodes */ private void walk(IndexNode inode, Consumer process) { @@ -160,7 +165,7 @@ } } - /* + /** * extract the root name from a versioned entry name * given inode for META-INF/versions/9/foo/bar.class * and prefix META-INF/versions/9/ diff --git a/test/jdk/nio/zipfs/MultiReleaseJarTest.java b/test/jdk/nio/zipfs/MultiReleaseJarTest.java --- a/test/jdk/nio/zipfs/MultiReleaseJarTest.java +++ b/test/jdk/nio/zipfs/MultiReleaseJarTest.java @@ -65,9 +65,12 @@ creator.buildUnversionedJar(); creator.buildMultiReleaseJar(); creator.buildShortMultiReleaseJar(); - uvuri = new URI("jar:file:" + userdir + "/unversioned.jar"); - mruri = new URI("jar:file:" + userdir + "/multi-release.jar"); - smruri = new URI("jar:file:" + userdir + "/short-multi-release.jar"); + String ssp = Paths.get(userdir, "unversioned.jar").toUri().toString(); + uvuri = new URI("jar", ssp , null); + ssp = Paths.get(userdir, "multi-release.jar").toUri().toString(); + mruri = new URI("jar", ssp, null); + ssp = Paths.get(userdir, "short-multi-release.jar").toUri().toString(); + smruri = new URI("jar", ssp, null); entryName = className.replace('.', '/') + ".class"; } > On Dec 18, 2015, at 12:28 AM, Paul Sandoz wrote: > > HI Steve, > > I find the use of the system property awkward. > > Did you try doing: > > new File(...).toURI() > > ? > > I noticed you are already creating Path instances for deletion, so it might be even better to do Path.toURI() e.g.: > > uvuri = ?jar:? + Paths.get(userdir, "unversioned.jar?).toURI(); > > rather than: > > uvuri = ?jar:? + new FIle(userdir, "unversioned.jar?).toURI(); > > Paul. > >> On 18 Dec 2015, at 03:20, Steve Drach wrote: >> >> Hi, >> >> jprt found a bug in one of my tests when run on windows. The fix is simple as you can see in the patch below. I also took the opportunity to add a comment to a line in the source code. Please let me know if these patches can be added to the changeset. >> >> Thanks >> Steve >> >> diff --git a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java >> --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java >> +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java >> @@ -55,7 +55,8 @@ >> >> JarFileSystem(ZipFileSystemProvider provider, Path zfpath, Map env) throws IOException { >> super(provider, zfpath, env); >> - lookup = path -> path; >> + lookup = path -> path; // lookup needs to be set before isMultiReleaseJar is called because >> + // it eventually calls getEntry0 >> if (isMultiReleaseJar()) { >> int version; >> Object o = env.get("multi-release"); >> diff --git a/test/jdk/nio/zipfs/MultiReleaseJarTest.java b/test/jdk/nio/zipfs/MultiReleaseJarTest.java >> --- a/test/jdk/nio/zipfs/MultiReleaseJarTest.java >> +++ b/test/jdk/nio/zipfs/MultiReleaseJarTest.java >> @@ -65,9 +65,15 @@ >> creator.buildUnversionedJar(); >> creator.buildMultiReleaseJar(); >> creator.buildShortMultiReleaseJar(); >> - uvuri = new URI("jar:file:" + userdir + "/unversioned.jar"); >> - mruri = new URI("jar:file:" + userdir + "/multi-release.jar"); >> - smruri = new URI("jar:file:" + userdir + "/short-multi-release.jar"); >> + String udir; >> + if (System.getProperty("os.name").startsWith("Windows")) { >> + udir = "/" + userdir.replace('\\','/'); >> + } else { >> + udir = userdir; >> + } >> + uvuri = new URI("jar:file:" + udir + "/unversioned.jar"); >> + mruri = new URI("jar:file:" + udir + "/multi-release.jar"); >> + smruri = new URI("jar:file:" + udir + "/short-multi-release.jar"); >> entryName = className.replace('.', '/') + ".class"; >> } >> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 >>> >>> Change to modules.xml: http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.01/index.html >>> >>> Changes to ZipFileSystem: http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.01/index.html >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon Dec 21 08:24:29 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 21 Dec 2015 08:24:29 +0000 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <3851ECF7-7E03-44B1-9489-63B7EA6095B9@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <53A15F43-CCA6-4462-9D89-C8DE14D2CFF8@oracle.com> <201ED7E0-6205-4A1F-97EC-56B5BEF0E1F1@oracle.com> <3851ECF7-7E03-44B1-9489-63B7EA6095B9@oracle.com> Message-ID: <5677B73D.4080505@oracle.com> On 18/12/2015 22:42, Steve Drach wrote: > Hi, > > I updated the patch to the changeset to address Paul?s concern with a > windows specific solution, as well some of Alan?s concerns. Please > let me know if this patch (instead of the previous one) can be applied > to the changeset. > > Thanks > Steve Would it be possible to refresh the webrev? The patch you inlined adjusts comments and fixes the issue with the test on Windows but it's hard to see everything (like getting JarFileSystem consistent with ZipFileSystem and I think whether we should rename getEntry0 because it looks strange that this is not private). -Alan From Lari at Hotari.net Mon Dec 21 12:31:51 2015 From: Lari at Hotari.net (Lari Hotari) Date: Mon, 21 Dec 2015 14:31:51 +0200 Subject: serious WatchService bug on Linux when registration requests and events arriving are very close together Message-ID: There is a serious bug in the Linux implementation of WatchService. I found this blog article explaining the problem: "Beware: java.nio.file.WatchService is subtly broken on Linux" http://blog.omega-prime.co.uk/?p=161 This problem occurs when registering watches while simultaneously receiving events. The blog article author explains the bug in LinuxWatchService and how inotify event data gets corrupted in a certain situation. Quoting the blog post: "If data happens to be available both via the pipe and via inotify, then the read from the pipe will corrupt the first few bytes of the inotify event stream!" someone has also created this Hackernews discussion: https://news.ycombinator.com/item?id=10692247 The blog article author has reported the issue to Oracle about 3 months ago with code to reproduce the issue: "I reported this to Oracle about 3 months ago (ref JI-9024492) but haven't heard anything since 2015-09-29, when they contacted me asking for some code to reproduce the issue (which I gave to them on the same day). :-/" What is the OpenJDK bug id for this issue (JI-9024492)? I'd like to know what the status of the fix for this issue is. -Lari -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve.drach at oracle.com Mon Dec 21 16:41:41 2015 From: steve.drach at oracle.com (Steve Drach) Date: Mon, 21 Dec 2015 08:41:41 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <5677B73D.4080505@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <53A15F43-CCA6-4462-9D89-C8DE14D2CFF8@oracle.com> <201ED7E0-6205-4A1F-97EC-56B5BEF0E1F1@oracle.com> <3851ECF7-7E03-44B1-9489-63B7EA6095B9@oracle.com> <5677B73D.4080505@oracle.com> Message-ID: >> Hi, >> >> I updated the patch to the changeset to address Paul?s concern with a windows specific solution, as well some of Alan?s concerns. Please let me know if this patch (instead of the previous one) can be applied to the changeset. >> >> Thanks >> Steve > Would it be possible to refresh the webrev? The patch you inlined adjusts comments and fixes the issue with the test on Windows but it's hard to see everything (like getting JarFileSystem consistent with ZipFileSystem and I think whether we should rename getEntry0 because it looks strange that this is not private). Sure I can refresh the webrev today. I didn?t rename getEntry0 because that name is in ZipFileSystem and I assume it was chosen, instead of getEntry, for a reason. Perhaps Sherman can shed some light on that. The reason that it is not private is because JarFileSystem invokes ZipFileSystem.newInputStream, and that invokes getEntry0. From steve.drach at oracle.com Mon Dec 21 17:55:44 2015 From: steve.drach at oracle.com (Steve Drach) Date: Mon, 21 Dec 2015 09:55:44 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <56740B82.2030901@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <56740B82.2030901@oracle.com> Message-ID: <45A25AF5-665F-44D0-ADC2-6EA99F76B689@oracle.com> Hi, I?ve updated the webrevs to take into account Paul?s and Alan's comments. Issue: https://bugs.openjdk.java.net/browse/JDK-8144355 Change to modules.xml: http://cr.openjdk.java.net/~sdrach/8144355/top/webrev.02/index.html Changes to ZipFileSystem: http://cr.openjdk.java.net/~sdrach/8144355/jdk/webrev.02/index.html I removed the branch on System.property for windows specific functionality in MultiReleaseJarTest as Paul suggested. I?ll address each of Alan?s concerns below: > The changes to ZipFileSystem* looks okay although I wonder if getEntry0 should be renamed to getEntry as it's cleaner when overridden in JarFileSystem. I did not change this. I?d like Sherman to comment on it. I assume getEntry0 was named for a reason. > JarFileSystem looks okay and I assume the recursion in walk should never be an issue. I assume so too since jar files have a finite, relatively small, set of entries. > A minor nit is that is that it would be good to keep the style consistent with the existing code if you can. Really long source lines are annoying when looking at side-by-side diffs. Consistent with ZipFileSystem? As for long lines, the longest line is 109 characters. I always try to keep lines shorter than 120 characters. I think it?s easier to read on a single line. Seems like a matter of style. > Also good if the main comment in JarFileSystem could be expanded a bit, also the comments in the methods can use /** instead of /*. I did that. > > The tests looks good. You might want to check the copyright header on MultiReleaseJarTest, the others look okay. I don?t see what the issue is with the copyright. -------------- next part -------------- An HTML attachment was scrubbed... URL: From xueming.shen at oracle.com Tue Dec 22 01:14:39 2015 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 22 Dec 2015 09:14:39 +0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <53A15F43-CCA6-4462-9D89-C8DE14D2CFF8@oracle.com> <201ED7E0-6205-4A1F-97EC-56B5BEF0E1F1@oracle.com> <3851ECF7-7E03-44B1-9489-63B7EA6095B9@oracle.com> <5677B73D.4080505@oracle.com> Message-ID: <5678A3FF.9010608@oracle.com> On 12/22/15, 12:41 AM, Steve Drach wrote: >>> Hi, >>> >>> I updated the patch to the changeset to address Paul?s concern with a windows specific solution, as well some of Alan?s concerns. Please let me know if this patch (instead of the previous one) can be applied to the changeset. >>> >>> Thanks >>> Steve >> Would it be possible to refresh the webrev? The patch you inlined adjusts comments and fixes the issue with the test on Windows but it's hard to see everything (like getting JarFileSystem consistent with ZipFileSystem and I think whether we should rename getEntry0 because it looks strange that this is not private). > Sure I can refresh the webrev today. I didn?t rename getEntry0 because that name is in ZipFileSystem and I assume it was chosen, instead of getEntry, for a reason. Perhaps Sherman can shed some light on that. The reason that it is not private is because JarFileSystem invokes ZipFileSystem.newInputStream, and that invokes getEntry0. let's rename getEntry0 to getEntry(), given it's no longer a private method now. There was another getEntry() in the early version of ZFS, but it does not matter any more. -sherman From steve.drach at oracle.com Tue Dec 22 18:11:14 2015 From: steve.drach at oracle.com (Steve Drach) Date: Tue, 22 Dec 2015 10:11:14 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <5678A3FF.9010608@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <53A15F43-CCA6-4462-9D89-C8DE14D2CFF8@oracle.com> <201ED7E0-6205-4A1F-97EC-56B5BEF0E1F1@oracle.com> <3851ECF7-7E03-44B1-9489-63B7EA6095B9@oracle.com> <5677B73D.4080505@oracle.com> <5678A3FF.9010608@oracle.com> Message-ID: >>>> I updated the patch to the changeset to address Paul?s concern with a windows specific solution, as well some of Alan?s concerns. Please let me know if this patch (instead of the previous one) can be applied to the changeset. >>>> >>>> Thanks >>>> Steve >>> Would it be possible to refresh the webrev? The patch you inlined adjusts comments and fixes the issue with the test on Windows but it's hard to see everything (like getting JarFileSystem consistent with ZipFileSystem and I think whether we should rename getEntry0 because it looks strange that this is not private). >> Sure I can refresh the webrev today. I didn?t rename getEntry0 because that name is in ZipFileSystem and I assume it was chosen, instead of getEntry, for a reason. Perhaps Sherman can shed some light on that. The reason that it is not private is because JarFileSystem invokes ZipFileSystem.newInputStream, and that invokes getEntry0. > let's rename getEntry0 to getEntry(), given it's no longer a private method now. There > was another getEntry() in the early version of ZFS, but it does not matter any more. Okay, I did that. The patch is attached. It contains only the change from getEntry0 to getEntry. -------------- next part -------------- A non-text attachment was scrubbed... Name: getEntry.patch Type: application/octet-stream Size: 7123 bytes Desc: not available URL: From ej at evanjones.ca Sun Dec 27 20:35:56 2015 From: ej at evanjones.ca (Evan Jones) Date: Sun, 27 Dec 2015 15:35:56 -0500 Subject: sun.nio.ch.Util: Don't cache an unlimited amount of memory Message-ID: Summary: nio Util caches an unlimited amount of memory for temporary direct ByteBuffers, which effectively is a native memory leak. Applications that do large I/Os can inadvertently waste gigabytes of native memory or run out of memory. I suggest it should only cache a "small" amount of memory per-thread (e.g. 1 MB), and maybe have a flag to allow changing the limit for applications where this causes a performance regression. 1. Would JDK committers support this change? 2. If so, any suggestions for the default and/or how to override it with a flag? Tony Printezis (CCed here) added a flag to Twitter's internal JVM/JDK to limit the size of this cache, which we could probably use that as a starting point for a patch. Additional Details: At Twitter, we recently ran into a "native memory leak" caused by sun.nio.ch.Util caching many gigabytes of native memory. Our program makes the mistake of using heap ByteBuffers for I/O, and occasionally will create a very large buffer and write it out a network socket. nio Util caches a temporary direct ByteBuffer that is the maximum size a thread has ever used. Thus, for a long lived thread, this memory usage will only ever increase. This looks like a native memory leak, causing long lived applications to continue to always use more and more native memory, until it hits some limit and gets killed. We aren't the only people who have this this problem. Norman Maurer described running into this at Apple (some notes in this Twitter discussion: https://twitter.com/TonyPrintezis/status/667117354438885376). Ariel Weisberg described running into this as part of Cassandra: https://twitter.com/ArielWeisberg/status/667138444561424384 My quick-and-dirty testing shows that removing the cache completely hurts the performance of applications that use heap ByteBuffers. This is particularly bad for "small" I/O, since the overhead of allocating and deallocating memory is high, relative to the copy and the I/O. My suggestion isl thread cache a maximum of 1 MB of direct ByteBuffer(s). This means small I/Os will be the same as they are now, and large I/Os will have a small amount of overhead. However, I would rather have a performance problem than a native memory leak. Anyone who cares about performance should be using direct ByteBuffers anyway. Tony Printezis discusses this issue at 42:38 in the following video: http://www.youtube.com/watch?v=M9o1LVfGp2A&t=42m38s -- Evan Jones http://evanjones.ca/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon Dec 28 06:53:00 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 28 Dec 2015 06:53:00 +0000 Subject: serious WatchService bug on Linux when registration requests and events arriving are very close together In-Reply-To: References: Message-ID: <5680DC4C.3080104@oracle.com> On 21/12/2015 12:31, Lari Hotari wrote: > : > > What is the OpenJDK bug id for this issue (JI-9024492)? I'd like to > know what the status of the fix for this issue is. Thanks for the bug report. It was moved to the bug database in the last few days as JDK-8145981. -Alan From sebastian.sickelmann at gmx.de Mon Dec 28 14:39:48 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Mon, 28 Dec 2015 15:39:48 +0100 Subject: serious WatchService bug on Linux when registration requests and events arriving are very close together In-Reply-To: <5680DC4C.3080104@oracle.com> References: <5680DC4C.3080104@oracle.com> Message-ID: <568149B4.80805@gmx.de> Hi, I want to fix this one. Is someone working on a solution/test yet? -- Sebastian On 12/28/2015 07:53 AM, Alan Bateman wrote: > > On 21/12/2015 12:31, Lari Hotari wrote: >> : >> >> What is the OpenJDK bug id for this issue (JI-9024492)? I'd like to >> know what the status of the fix for this issue is. > Thanks for the bug report. It was moved to the bug database in the > last few days as JDK-8145981. > > -Alan > From brian.burkhalter at oracle.com Mon Dec 28 16:04:53 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 28 Dec 2015 08:04:53 -0800 Subject: serious WatchService bug on Linux when registration requests and events arriving are very close together In-Reply-To: <568149B4.80805@gmx.de> References: <5680DC4C.3080104@oracle.com> <568149B4.80805@gmx.de> Message-ID: <5476CFFE-3926-4BAA-BF9F-EE80350845F3@oracle.com> No, at present it is yet to be assigned. Brian On Dec 28, 2015, at 6:39 AM, Sebastian Sickelmann wrote: > Hi, > > I want to fix this one. Is someone working on a solution/test yet? > > -- > Sebastian > > On 12/28/2015 07:53 AM, Alan Bateman wrote: >> >> On 21/12/2015 12:31, Lari Hotari wrote: >>> : >>> >>> What is the OpenJDK bug id for this issue (JI-9024492)? I'd like to >>> know what the status of the fix for this issue is. >> Thanks for the bug report. It was moved to the bug database in the >> last few days as JDK-8145981. >> >> -Alan >> > From Alan.Bateman at oracle.com Mon Dec 28 17:15:36 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 28 Dec 2015 17:15:36 +0000 Subject: serious WatchService bug on Linux when registration requests and events arriving are very close together In-Reply-To: <568149B4.80805@gmx.de> References: <5680DC4C.3080104@oracle.com> <568149B4.80805@gmx.de> Message-ID: <56816E38.10308@oracle.com> On 28/12/2015 14:39, Sebastian Sickelmann wrote: > Hi, > > I want to fix this one. Is someone working on a solution/test yet? > No problem if you take it. I did attach an initial patch to the bug after seeing the mails earlier today. There is a reproducer with the (good) bug report but it doesn't reproduce easily on any of the Linux machines that I tried. So one that would be great is to see whether we can tickle the issue quickly. -Alan From sebastian.sickelmann at gmx.de Mon Dec 28 18:51:58 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Mon, 28 Dec 2015 19:51:58 +0100 Subject: RFR: 8145981 (fs) LinuxWatchService can reports events against wrong directory In-Reply-To: <56816E38.10308@oracle.com> References: <5680DC4C.3080104@oracle.com> <568149B4.80805@gmx.de> <56816E38.10308@oracle.com> Message-ID: <568184CE.8040401@gmx.de> Hi Alan, i am not fully sure, but i think your patch does not change anything. But I think I understand what you have written in the comment. I think there is a third possible solution to the two you mentioned. I was not sure, if it was done intentionally (shutdown before processing) so I have keeped the order of processing and shutdown and only resorted the order of the reading in the special case. Your suggested solution may be easier to read and maintain. What do you think? http://cr.openjdk.java.net/~sebastian/8145981/webrev.00/ The backporting to older releases should be straight forward and I will produce changsets for those when we are ready with the review for jdk9. On my box it takes 20-30 seconds to tickle the bug. After the fix i had ran the test for about 10 minutes. Do you have any good idea how to make the test more robust, so that an regression could be detected on all your linux boxes in a short time? -- Sebastian On 12/28/2015 06:15 PM, Alan Bateman wrote: > > On 28/12/2015 14:39, Sebastian Sickelmann wrote: >> Hi, >> >> I want to fix this one. Is someone working on a solution/test yet? >> > No problem if you take it. I did attach an initial patch to the bug > after seeing the mails earlier today. There is a reproducer with the > (good) bug report but it doesn't reproduce easily on any of the Linux > machines that I tried. So one that would be great is to see whether we > can tickle the issue quickly. > > -Alan > From Alan.Bateman at oracle.com Tue Dec 29 08:55:39 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 29 Dec 2015 08:55:39 +0000 Subject: RFR: 8145981 (fs) LinuxWatchService can reports events against wrong directory In-Reply-To: <568184CE.8040401@gmx.de> References: <5680DC4C.3080104@oracle.com> <568149B4.80805@gmx.de> <56816E38.10308@oracle.com> <568184CE.8040401@gmx.de> Message-ID: <56824A8B.3030004@oracle.com> On 28/12/2015 18:51, Sebastian Sickelmann wrote: > Hi Alan, > > i am not fully sure, but i think your patch does not change anything. It moves the handling of requests to after the processing of the inotify data. The ordering doesn't matter and okay to process some events even if there is a queued request to cancel or shutdown. > > But I think I understand what you have written in the comment. I think > there is a third possible solution to the two you mentioned. > > I was not sure, if it was done intentionally (shutdown before > processing) so I have keeped the order of processing and shutdown and > only resorted the order of the reading in the special case. > > Your suggested solution may be easier to read and maintain. What do you > think? I think the simplest is to just move the handling of requests. Your fix will work too but it means that the update/shutdown requests would be handled in two places and so just a little harder to maintain. > > http://cr.openjdk.java.net/~sebastian/8145981/webrev.00/ > > The backporting to older releases should be straight forward and I will > produce changsets for those when we are ready with the review for jdk9. We need to agree the changes for jdk9/dev first, then we can think about jdk8u-dev after the changes have baked in jdk9/dev for a while. In this case then the change should be fine but the test might need test to bed down. The test needs to prove that it is stable on all platforms for example. > > On my box it takes 20-30 seconds to tickle the bug. After the fix i had > ran the test for about 10 minutes. > > Do you have any good idea how to make the test more robust, so that an > regression could be detected on all your linux boxes in a short time? > The test currently takes ~110s and would be good to get it down to <30s to avoid it slowing down the test runs. Your updated tests seems to fail for me in 10-20s on two machines that I tried so we might not have to do too much to do. Sometimes it can be useful to introduce noise or have the test running concurrently in a thread pool to provoke the issue quicker but it might not be needed here. A few other things on the test: 1. It would be good to choose a name for the test that is consistent with the naming in this area. UpdateInterference might work, you might have other ideas. 2. In the jtreg tags then I assume @library is not needed (I can't see it using any the test infrastructure). Also the /timeout=120 is not needed either. 3. It would be good to keep the style consistent if you can. If line lengths are too long then it makes it hard to do side-by-side diffs for example. If you static import StandardWatchEventKinds.* then it should reduce some of the lines. -Alan From Alan.Bateman at oracle.com Tue Dec 29 09:17:21 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 29 Dec 2015 09:17:21 +0000 Subject: sun.nio.ch.Util: Don't cache an unlimited amount of memory In-Reply-To: References: Message-ID: <56824FA1.5050808@oracle.com> On 27/12/2015 20:35, Evan Jones wrote: > Summary: nio Util caches an unlimited amount of memory for temporary > direct ByteBuffers, which effectively is a native memory leak. > Applications that do large I/Os can inadvertently waste gigabytes of > native memory or run out of memory. I suggest it should only cache a > "small" amount of memory per-thread (e.g. 1 MB), and maybe have a flag > to allow changing the limit for applications where this causes a > performance regression. > > 1. Would JDK committers support this change? > 2. If so, any suggestions for the default and/or how to override it > with a flag? > > Tony Printezis (CCed here) added a flag to Twitter's internal JVM/JDK > to limit the size of this cache, which we could probably use that as a > starting point for a patch. Limiting the size of the buffer cache might help in some scenarios, it just means a bit more complexity and yet another tuning option. Do you do scatter/gather I/O? The current implementation will cache up to IOV_MAX buffers per thread but if you aren't doing scatter/gather I/O then caching a maximum of one buffer per thread should reduce the memory usage. It wouldn't be hard to modify the BufferCache implementation to track the number of per-thread buffers in use so that this count is the maximum cached rather than IOV_MAX. I'm curious if you've looked into doing anything along those lines. -Alan From sebastian.sickelmann at gmx.de Tue Dec 29 15:27:53 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Tue, 29 Dec 2015 16:27:53 +0100 Subject: RFR: 8145981 (fs) LinuxWatchService can reports events against wrong directory In-Reply-To: <56824A8B.3030004@oracle.com> References: <5680DC4C.3080104@oracle.com> <568149B4.80805@gmx.de> <56816E38.10308@oracle.com> <568184CE.8040401@gmx.de> <56824A8B.3030004@oracle.com> Message-ID: <5682A679.9010904@gmx.de> On 12/29/2015 09:55 AM, Alan Bateman wrote: > > On 28/12/2015 18:51, Sebastian Sickelmann wrote: >> Hi Alan, >> >> i am not fully sure, but i think your patch does not change anything. > It moves the handling of requests to after the processing of the > inotify data. The ordering doesn't matter and okay to process some > events even if there is a queued request to cancel or shutdown. Yes. That is what i understand from your JBS comment. But maybe i missed the right patch attachment in JBS. > >> >> But I think I understand what you have written in the comment. I think >> there is a third possible solution to the two you mentioned. >> >> I was not sure, if it was done intentionally (shutdown before >> processing) so I have keeped the order of processing and shutdown and >> only resorted the order of the reading in the special case. >> >> Your suggested solution may be easier to read and maintain. What do you >> think? > I think the simplest is to just move the handling of requests. Your > fix will work too but it means that the update/shutdown requests would > be handled in two places and so just a little harder to maintain. I changed it to your solution. You are right the order of processing pending events and new events from inotify do not need any guarantee. So processing pending events (and shutdown) can be done later. >> >> http://cr.openjdk.java.net/~sebastian/8145981/webrev.00/ >> >> The backporting to older releases should be straight forward and I will >> produce changsets for those when we are ready with the review for jdk9. > We need to agree the changes for jdk9/dev first, then we can think > about jdk8u-dev after the changes have baked in jdk9/dev for a while. > In this case then the change should be fine but the test might need > test to bed down. The test needs to prove that it is stable on all > platforms for example. > >> >> On my box it takes 20-30 seconds to tickle the bug. After the fix i had >> ran the test for about 10 minutes. >> >> Do you have any good idea how to make the test more robust, so that an >> regression could be detected on all your linux boxes in a short time? >> > The test currently takes ~110s and would be good to get it down to > <30s to avoid it slowing down the test runs. Your updated tests seems > to fail for me in 10-20s on two machines that I tried so we might not > have to do too much to do. Sometimes it can be useful to introduce > noise or have the test running concurrently in a thread pool to > provoke the issue quicker but it might not be needed here. > > A few other things on the test: > > 1. It would be good to choose a name for the test that is consistent > with the naming in this area. UpdateInterference might work, you might > have other ideas. > > 2. In the jtreg tags then I assume @library is not needed (I can't see > it using any the test infrastructure). Also the /timeout=120 is not > needed either. > > 3. It would be good to keep the style consistent if you can. If line > lengths are too long then it makes it hard to do side-by-side diffs > for example. If you static import StandardWatchEventKinds.* then it > should reduce some of the lines. > > -Alan I updated the webrev and tried to reduce the time in the test by generating more watch-keys. On my machine I am now always below 10 secs and in most cases below 5 sec. http://cr.openjdk.java.net/~sebastian/8145981/webrev.01/ -- Sebastian From ej at evanjones.ca Tue Dec 29 15:59:30 2015 From: ej at evanjones.ca (Evan Jones) Date: Tue, 29 Dec 2015 10:59:30 -0500 Subject: sun.nio.ch.Util: Don't cache an unlimited amount of memory In-Reply-To: <56824FA1.5050808@oracle.com> References: <56824FA1.5050808@oracle.com> Message-ID: No, we don't do scatter/gather I/O. Most Twitter services (Finagle which uses Netty 3) call NIO with variable sized heap ByteBuffers. The "leak" is caused by the fact that each thread caches a single direct ByteBuffer of the maximum size it has ever seen. Hence, if 0.01% of requests eventually cause a ~100 MB chunk to be sent, then each thread ends up with slowly growing native memory usage that never decreases, even if the "typical" size is ~1 MB. I still think it is very surprising and wrong that the JDK caches such enormous amounts of memory in this scenario. I would much rather have a "performance problem" that I can fix by managing my own buffers, than a mysterious native memory leak that is difficult to track down. My (simple) proposal will have a performance impact for applications that do large I/O with heap ByteBuffers. However, I would argue those apps are already slow because of the copy, and probably won't notice :). Would you be interested in a more sophisticated solution that would allow huge cached buffers to eventually expire? For example, if the "recent" usage of the cache has been much smaller than the allocated buffers, free the buffers and re-allocate? For the problematic application that I found, *any* policy for when to expire would effectively solve this problem, since the "peak" usage is significantly worse than the "average" usage. My final suggestion: I would be happy to attempt to revise the javadoc about direct buffers to make it clearer that using heap buffers for I/O will cause copies, and will also cause the JDK to cache native memory. At least then we could argue that this behaviour is "as designed." :) On Tue, Dec 29, 2015 at 4:17 AM, Alan Bateman wrote: > > On 27/12/2015 20:35, Evan Jones wrote: > >> Summary: nio Util caches an unlimited amount of memory for temporary >> direct ByteBuffers, which effectively is a native memory leak. Applications >> that do large I/Os can inadvertently waste gigabytes of native memory or >> run out of memory. I suggest it should only cache a "small" amount of >> memory per-thread (e.g. 1 MB), and maybe have a flag to allow changing the >> limit for applications where this causes a performance regression. >> >> 1. Would JDK committers support this change? >> 2. If so, any suggestions for the default and/or how to override it with >> a flag? >> >> Tony Printezis (CCed here) added a flag to Twitter's internal JVM/JDK to >> limit the size of this cache, which we could probably use that as a >> starting point for a patch. >> > Limiting the size of the buffer cache might help in some scenarios, it > just means a bit more complexity and yet another tuning option. > > Do you do scatter/gather I/O? The current implementation will cache up to > IOV_MAX buffers per thread but if you aren't doing scatter/gather I/O then > caching a maximum of one buffer per thread should reduce the memory usage. > It wouldn't be hard to modify the BufferCache implementation to track the > number of per-thread buffers in use so that this count is the maximum > cached rather than IOV_MAX. I'm curious if you've looked into doing > anything along those lines. > > -Alan > > > -- Evan Jones http://evanjones.ca/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ariel at weisberg.ws Tue Dec 29 16:57:01 2015 From: ariel at weisberg.ws (Ariel Weisberg) Date: Tue, 29 Dec 2015 11:57:01 -0500 Subject: sun.nio.ch.Util: Don't cache an unlimited amount of memory In-Reply-To: References: <56824FA1.5050808@oracle.com> Message-ID: <1451408221.3301219.478528674.0952CE75@webmail.messagingengine.com> Hi, Channel.new[Input | Output]Stream can trigger this as well. It's an odd thing to put in the javadoc there since this is an artifact of a specific Channel implementation and not Channels in general. One solution that would be nice is to break down large writes into several large smaller writes so the per thread buffer size is at least bounded and not set to the size of the largest seen IO. For sockets this is usually not a problem although I can imagine there is some implementation somewhere that will care. For files this is a bit more problematic especially if O_SYNC or O_DIRECT are in use. Filesystems on down also trigger different behaviors based off of read/write size so hiding the fact that something is one big IO can have an impact. My intuition is that something based on breaking down IOs is not going o pass muster, but it exists. Could this be handled better with a different pooling approach that is not per thread over some threshold? There are a bunch of permutations that could trade off allocation/deallocation for concurrency vs. blocking threads waiting for operations to complete freeing up globally pooled buffers to bound peak footprint. Regards, Ariel On Tue, Dec 29, 2015, at 10:59 AM, Evan Jones wrote: > No, we don't do scatter/gather I/O. Most Twitter services (Finagle > which uses Netty 3) call NIO with variable sized heap ByteBuffers. The > "leak" is caused by the fact that each thread caches a single direct > ByteBuffer of the maximum size it has ever seen. Hence, if 0.01% of > requests eventually cause a ~100 MB chunk to be sent, then each thread > ends up with slowly growing native memory usage that never decreases, > even if the "typical" size is ~1 MB. I still think it is very > surprising and wrong that the JDK caches such enormous amounts of > memory in this scenario. I would much rather have a "performance > problem" that I can fix by managing my own buffers, than a mysterious > native memory leak that is difficult to track down. > > My (simple) proposal will have a performance impact for applications > that do large I/O with heap ByteBuffers. However, I would argue > those apps are already slow because of the copy, and probably won't > notice :). > > Would you be interested in a more sophisticated solution that would > allow huge cached buffers to eventually expire? For example, if the > "recent" usage of the cache has been much smaller than the allocated > buffers, free the buffers and re-allocate? For the problematic > application that I found, *any* policy for when to expire would > effectively solve this problem, since the "peak" usage is > significantly worse than the "average" usage. > > My final suggestion: I would be happy to attempt to revise the javadoc > about direct buffers to make it clearer that using heap buffers for > I/O will cause copies, and will also cause the JDK to cache native > memory. At least then we could argue that this behaviour is "as > designed." :) > > > On Tue, Dec 29, 2015 at 4:17 AM, Alan Bateman > wrote: >> >> On 27/12/2015 20:35, Evan Jones wrote: >>> Summary: nio Util caches an unlimited amount of memory for temporary direct ByteBuffers, which effectively is a native memory leak. Applications that do large I/Os can inadvertently waste gigabytes of native memory or run out of memory. I suggest it should only cache a "small" amount of memory per-thread (e.g. 1 MB), and maybe have a flag to allow changing the limit for applications where this causes a performance regression. >>> >>> 1. Would JDK committers support this change? >>> 2. If so, any suggestions for the default and/or how to override it with a flag? >>> >>> Tony Printezis (CCed here) added a flag to Twitter's internal JVM/JDK to limit the size of this cache, which we could probably use that as a starting point for a patch. >> Limiting the size of the buffer cache might help in some scenarios, it just means a bit more complexity and yet another tuning option. >> >> Do you do scatter/gather I/O? The current implementation will cache up to IOV_MAX buffers per thread but if you aren't doing scatter/gather I/O then caching a maximum of one buffer per thread should reduce the memory usage. It wouldn't be hard to modify the BufferCache implementation to track the number of per-thread buffers in use so that this count is the maximum cached rather than IOV_MAX. I'm curious if you've looked into doing anything along those lines. >> >> -Alan >> > > > > -- > Evan Jones http://evanjones.ca/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Dec 29 17:13:45 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 29 Dec 2015 17:13:45 +0000 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <45A25AF5-665F-44D0-ADC2-6EA99F76B689@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <56740B82.2030901@oracle.com> <45A25AF5-665F-44D0-ADC2-6EA99F76B689@oracle.com> Message-ID: <5682BF49.7010408@oracle.com> On 21/12/2015 17:55, Steve Drach wrote: > : > >> The changes to ZipFileSystem* looks okay although I wonder if >> getEntry0 should be renamed to getEntry as it's cleaner when >> overridden in JarFileSystem. > > I did not change this. I?d like Sherman to comment on it. I assume > getEntry0 was named for a reason. It was a private method and originally wrapped with getEntry that has since been removed. > : > > Consistent with ZipFileSystem? As for long lines, the longest line is > 109 characters. I always try to keep lines shorter than 120 > characters. I think it?s easier to read on a single line. Seems like > a matter of style. Best to keep it consistent with ZipFileSystem code. Also it makes it easier when doing side-by-side diffs. > I don?t see what the issue is with the copyright. > Tests should have the GPL header, this one has the Classpath exception so I assume it must have been copied from a file in the src tree. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Dec 29 17:16:26 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 29 Dec 2015 17:16:26 +0000 Subject: RFR: 8145981 (fs) LinuxWatchService can reports events against wrong directory In-Reply-To: <5682A679.9010904@gmx.de> References: <5680DC4C.3080104@oracle.com> <568149B4.80805@gmx.de> <56816E38.10308@oracle.com> <568184CE.8040401@gmx.de> <56824A8B.3030004@oracle.com> <5682A679.9010904@gmx.de> Message-ID: <5682BFEA.5060406@oracle.com> On 29/12/2015 15:27, Sebastian Sickelmann wrote: > : > I changed it to your solution. You are right the order of processing > pending events and new events from inotify do not need any guarantee. So > processing pending events (and shutdown) can be done later. Thanks, I think this is the easiest to maintain. > : > I updated the webrev and tried to reduce the time in the test by > generating more watch-keys. On my machine I am now always below 10 secs > and in most cases below 5 sec. > > http://cr.openjdk.java.net/~sebastian/8145981/webrev.01/ > This looks okay. I'll run it on our build+test system to make sure that the test is passing on all platforms. -Alan From uschindler at apache.org Tue Dec 29 17:47:43 2015 From: uschindler at apache.org (Uwe Schindler) Date: Tue, 29 Dec 2015 18:47:43 +0100 Subject: sun.nio.ch.Util: Don't cache an unlimited amount of memory In-Reply-To: <1451408221.3301219.478528674.0952CE75@webmail.messagingengine.com> References: <56824FA1.5050808@oracle.com> <1451408221.3301219.478528674.0952CE75@webmail.messagingengine.com> Message-ID: <025801d14261$07132090$153961b0$@apache.org> Hi, There is more stuff affected by this: Also the ?good old? RandomAccessFile or FileInput/OutputStream. Although those don?t use DirectBuffers internally, those allocate a native buffer with libc?s malloc() if the byte[] is larger than 8192 bytes. If you transfer a large byte[] using those old APIs it allocates (only for short time, so its better) a malloced buffer. But this buffer is freed after transfer, but its also an overhead as it fragments libc?s heap. If you buffer is < 8192 bytes it uses a buffer on stack, so when using those old APIs prefer to chunk at 8192 bytes. We were affected by both problems in Lucene since the early days. E.g, we sometimes load norms for millions of documents from disk, which can be a Gigabyte of stuff. In early days this was loaded in a single read (using above old APIs, now with NIO.2 or mmap) ? causing OOM because the malloc failed. But also with NIO this is a desaster, because we partly use heap buffers (unless mmap is used for Lucene indexes, which is preferred nowadays on 64 bits). To work around those problems, Lucene?s IO API (wrapped around mmap, nio.2 file or channels) uses chunks automatically. E.g. when writing to an OutputStream, we automatically chunk in 8192 bytes (to not cause any mallocs in non-NIO code or any additional direct buffers). This is done using a wrapper stream (extends FilterOutputStream), e.g., see https://goo.gl/TijUFu for details. Very simple, very effective (if you use streams). Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Ariel Weisberg Sent: Tuesday, December 29, 2015 5:57 PM To: nio-dev at openjdk.java.net Subject: Re: sun.nio.ch.Util: Don't cache an unlimited amount of memory Hi, Channel.new[Input | Output]Stream can trigger this as well. It's an odd thing to put in the javadoc there since this is an artifact of a specific Channel implementation and not Channels in general. One solution that would be nice is to break down large writes into several large smaller writes so the per thread buffer size is at least bounded and not set to the size of the largest seen IO. For sockets this is usually not a problem although I can imagine there is some implementation somewhere that will care. For files this is a bit more problematic especially if O_SYNC or O_DIRECT are in use. Filesystems on down also trigger different behaviors based off of read/write size so hiding the fact that something is one big IO can have an impact. My intuition is that something based on breaking down IOs is not going o pass muster, but it exists. Could this be handled better with a different pooling approach that is not per thread over some threshold? There are a bunch of permutations that could trade off allocation/deallocation for concurrency vs. blocking threads waiting for operations to complete freeing up globally pooled buffers to bound peak footprint. Regards, Ariel On Tue, Dec 29, 2015, at 10:59 AM, Evan Jones wrote: No, we don't do scatter/gather I/O. Most Twitter services (Finagle which uses Netty 3) call NIO with variable sized heap ByteBuffers. The "leak" is caused by the fact that each thread caches a single direct ByteBuffer of the maximum size it has ever seen. Hence, if 0.01% of requests eventually cause a ~100 MB chunk to be sent, then each thread ends up with slowly growing native memory usage that never decreases, even if the "typical" size is ~1 MB. I still think it is very surprising and wrong that the JDK caches such enormous amounts of memory in this scenario. I would much rather have a "performance problem" that I can fix by managing my own buffers, than a mysterious native memory leak that is difficult to track down. My (simple) proposal will have a performance impact for applications that do large I/O with heap ByteBuffers. However, I would argue those apps are already slow because of the copy, and probably won't notice :). Would you be interested in a more sophisticated solution that would allow huge cached buffers to eventually expire? For example, if the "recent" usage of the cache has been much smaller than the allocated buffers, free the buffers and re-allocate? For the problematic application that I found, *any* policy for when to expire would effectively solve this problem, since the "peak" usage is significantly worse than the "average" usage. My final suggestion: I would be happy to attempt to revise the javadoc about direct buffers to make it clearer that using heap buffers for I/O will cause copies, and will also cause the JDK to cache native memory. At least then we could argue that this behaviour is "as designed." :) On Tue, Dec 29, 2015 at 4:17 AM, Alan Bateman < Alan.Bateman at oracle.com> wrote: On 27/12/2015 20:35, Evan Jones wrote: Summary: nio Util caches an unlimited amount of memory for temporary direct ByteBuffers, which effectively is a native memory leak. Applications that do large I/Os can inadvertently waste gigabytes of native memory or run out of memory. I suggest it should only cache a "small" amount of memory per-thread (e.g. 1 MB), and maybe have a flag to allow changing the limit for applications where this causes a performance regression. 1. Would JDK committers support this change? 2. If so, any suggestions for the default and/or how to override it with a flag? Tony Printezis (CCed here) added a flag to Twitter's internal JVM/JDK to limit the size of this cache, which we could probably use that as a starting point for a patch. Limiting the size of the buffer cache might help in some scenarios, it just means a bit more complexity and yet another tuning option. Do you do scatter/gather I/O? The current implementation will cache up to IOV_MAX buffers per thread but if you aren't doing scatter/gather I/O then caching a maximum of one buffer per thread should reduce the memory usage. It wouldn't be hard to modify the BufferCache implementation to track the number of per-thread buffers in use so that this count is the maximum cached rather than IOV_MAX. I'm curious if you've looked into doing anything along those lines. -Alan -- Evan Jones http://evanjones.ca/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian.sickelmann at gmx.de Tue Dec 29 18:44:55 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Tue, 29 Dec 2015 19:44:55 +0100 Subject: RFR: 8145981 (fs) LinuxWatchService can reports events against wrong directory In-Reply-To: <5682BFEA.5060406@oracle.com> References: <5680DC4C.3080104@oracle.com> <568149B4.80805@gmx.de> <56816E38.10308@oracle.com> <568184CE.8040401@gmx.de> <56824A8B.3030004@oracle.com> <5682A679.9010904@gmx.de> <5682BFEA.5060406@oracle.com> Message-ID: <5682D4A7.1000405@gmx.de> On 12/29/2015 06:16 PM, Alan Bateman wrote: > > > On 29/12/2015 15:27, Sebastian Sickelmann wrote: >> : >> I changed it to your solution. You are right the order of processing >> pending events and new events from inotify do not need any guarantee. So >> processing pending events (and shutdown) can be done later. > Thanks, I think this is the easiest to maintain. > > >> : >> I updated the webrev and tried to reduce the time in the test by >> generating more watch-keys. On my machine I am now always below 10 secs >> and in most cases below 5 sec. >> >> http://cr.openjdk.java.net/~sebastian/8145981/webrev.01/ >> > This looks okay. I'll run it on our build+test system to make sure > that the test is passing on all platforms. > > -Alan How does it work with a jdk8 backport? I think the issue-number 8145981 would be burned with this patch. Should we create a new subtask of the issue for this jdk9 patch? Here is a changeset for the case the original issue-number should be used and everything goes ok with build and test. http://cr.openjdk.java.net/~sebastian/8145981/webrev.02/jdk.changeset -- Sebastian From steve.drach at oracle.com Tue Dec 29 19:03:22 2015 From: steve.drach at oracle.com (Steve Drach) Date: Tue, 29 Dec 2015 11:03:22 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <5682BF49.7010408@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <56740B82.2030901@oracle.com> <45A25AF5-665F-44D0-ADC2-6EA99F76B689@oracle.com> <5682BF49.7010408@oracle.com> Message-ID: <7E10E5DD-2563-4F92-B217-F843256C91EA@oracle.com> >>> The changes to ZipFileSystem* looks okay although I wonder if getEntry0 should be renamed to getEntry as it's cleaner when overridden in JarFileSystem. >> >> I did not change this. I?d like Sherman to comment on it. I assume getEntry0 was named for a reason. > It was a private method and originally wrapped with getEntry that has since been removed. Yes, I changed it to getEntry after Sherman agreed that it should be done. >> Consistent with ZipFileSystem? As for long lines, the longest line is 109 characters. I always try to keep lines shorter than 120 characters. I think it?s easier to read on a single line. Seems like a matter of style. > Best to keep it consistent with ZipFileSystem code. Also it makes it easier when doing side-by-side diffs. I?ll look at it again, but it does just seem like a personal taste kind of thing. > > >> I don?t see what the issue is with the copyright. >> > Tests should have the GPL header, this one has the Classpath exception so I assume it must have been copied from a file in the src tree. Oh. Okay, I?ll fix that. From Alan.Bateman at oracle.com Tue Dec 29 19:18:01 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 29 Dec 2015 19:18:01 +0000 Subject: RFR: 8145981 (fs) LinuxWatchService can reports events against wrong directory In-Reply-To: <5682D4A7.1000405@gmx.de> References: <5680DC4C.3080104@oracle.com> <568149B4.80805@gmx.de> <56816E38.10308@oracle.com> <568184CE.8040401@gmx.de> <56824A8B.3030004@oracle.com> <5682A679.9010904@gmx.de> <5682BFEA.5060406@oracle.com> <5682D4A7.1000405@gmx.de> Message-ID: <5682DC69.1000704@oracle.com> On 29/12/2015 18:44, Sebastian Sickelmann wrote: > : > How does it work with a jdk8 backport? I think the issue-number 8145981 > would be burned with this patch. > Should we create a new subtask of the issue for this jdk9 patch? > There is an issue type "backport" when changes are back-ported. The backport issue will be created automatically when it is pushed to jdk8u-dev. It can be created in advance too. The change-set comment will have the original issue number. Best to give the fix a bit of time to bed down in jdk9/dev before requesting approval to back-port. The main thing here will being confident that the test is stable on all platforms. BTW: In the test then do you mind I change "service" to "watcher" to make it clearer and also consistent with the existing tests? -Alan From sebastian.sickelmann at gmx.de Tue Dec 29 19:38:50 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Tue, 29 Dec 2015 20:38:50 +0100 Subject: RFR: 8145981 (fs) LinuxWatchService can reports events against wrong directory In-Reply-To: <5682DC69.1000704@oracle.com> References: <5680DC4C.3080104@oracle.com> <568149B4.80805@gmx.de> <56816E38.10308@oracle.com> <568184CE.8040401@gmx.de> <56824A8B.3030004@oracle.com> <5682A679.9010904@gmx.de> <5682BFEA.5060406@oracle.com> <5682D4A7.1000405@gmx.de> <5682DC69.1000704@oracle.com> Message-ID: <5682E14A.3050000@gmx.de> On 12/29/2015 08:18 PM, Alan Bateman wrote: > > > On 29/12/2015 18:44, Sebastian Sickelmann wrote: >> : >> How does it work with a jdk8 backport? I think the issue-number 8145981 >> would be burned with this patch. >> Should we create a new subtask of the issue for this jdk9 patch? >> > There is an issue type "backport" when changes are back-ported. The > backport issue will be created automatically when it is pushed to > jdk8u-dev. It can be created in advance too. The change-set comment > will have the original issue number. Best to give the fix a bit of > time to bed down in jdk9/dev before requesting approval to back-port. > The main thing here will being confident that the test is stable on > all platforms. > > BTW: In the test then do you mind I change "service" to "watcher" to > make it clearer and also consistent with the existing tests? I am fine with this. But i have an additional legal question. The test is directly derived from the bug-reporter reproduction-source. Is this ok in the licensing-question? Or do we need a copyright agreement for it? -- Sebastian > > -Alan > > From steve.drach at oracle.com Wed Dec 30 02:28:42 2015 From: steve.drach at oracle.com (Steve Drach) Date: Tue, 29 Dec 2015 18:28:42 -0800 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <7E10E5DD-2563-4F92-B217-F843256C91EA@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <56740B82.2030901@oracle.com> <45A25AF5-665F-44D0-ADC2-6EA99F76B689@oracle.com> <5682BF49.7010408@oracle.com> <7E10E5DD-2563-4F92-B217-F843256C91EA@oracle.com> Message-ID: <4FD89CC1-29C3-48D8-B750-0FB4BB8E080B@oracle.com> >>>> The changes to ZipFileSystem* looks okay although I wonder if getEntry0 should be renamed to getEntry as it's cleaner when overridden in JarFileSystem. >>> >>> I did not change this. I?d like Sherman to comment on it. I assume getEntry0 was named for a reason. >> It was a private method and originally wrapped with getEntry that has since been removed. > > Yes, I changed it to getEntry after Sherman agreed that it should be done. > >>> Consistent with ZipFileSystem? As for long lines, the longest line is 109 characters. I always try to keep lines shorter than 120 characters. I think it?s easier to read on a single line. Seems like a matter of style. >> Best to keep it consistent with ZipFileSystem code. Also it makes it easier when doing side-by-side diffs. > > I?ll look at it again, but it does just seem like a personal taste kind of thing. I changed the maximum line length to 96 to be compatible with ZipFileSystem. I left two long ?fixme? comments because I intend to remove them, and fix the issues as soon as the JEP 223 Version class is available. > >> >> >>> I don?t see what the issue is with the copyright. >>> >> Tests should have the GPL header, this one has the Classpath exception so I assume it must have been copied from a file in the src tree. > > Oh. Okay, I?ll fix that. Fixed. The diff is attached: -------------- next part -------------- A non-text attachment was scrubbed... Name: minor.patch Type: application/octet-stream Size: 4055 bytes Desc: not available URL: -------------- next part -------------- From Alan.Bateman at oracle.com Wed Dec 30 08:16:39 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 30 Dec 2015 08:16:39 +0000 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <4FD89CC1-29C3-48D8-B750-0FB4BB8E080B@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <56740B82.2030901@oracle.com> <45A25AF5-665F-44D0-ADC2-6EA99F76B689@oracle.com> <5682BF49.7010408@oracle.com> <7E10E5DD-2563-4F92-B217-F843256C91EA@oracle.com> <4FD89CC1-29C3-48D8-B750-0FB4BB8E080B@oracle.com> Message-ID: <568392E7.8090000@oracle.com> On 30/12/2015 02:28, Steve Drach wrote: > I changed the maximum line length to 96 to be compatible with ZipFileSystem. I left two long ?fixme? comments because I intend to remove them, > and fix the issues as soon as the JEP 223 Version class is available. > Okay, I can will get this pushed for you this week. -Alan. From ej at evanjones.ca Wed Dec 30 14:41:18 2015 From: ej at evanjones.ca (Evan Jones) Date: Wed, 30 Dec 2015 09:41:18 -0500 Subject: sun.nio.ch.Util: Don't cache an unlimited amount of memory In-Reply-To: <025801d14261$07132090$153961b0$@apache.org> References: <56824FA1.5050808@oracle.com> <1451408221.3301219.478528674.0952CE75@webmail.messagingengine.com> <025801d14261$07132090$153961b0$@apache.org> Message-ID: On Tue, Dec 29, 2015 at 12:47 PM, Uwe Schindler wrote: > > There is more stuff affected by this: Also the ?good old? RandomAccessFile > or FileInput/OutputStream. Although those don?t use DirectBuffers > internally, those allocate a native buffer with libc?s malloc() if the > byte[] is larger than 8192 bytes. If you transfer a large byte[] using > those old APIs it allocates (only for short time, so its better) a malloced > buffer. But this buffer is freed after transfer, but its also an overhead > as it fragments libc?s heap. If you buffer is < 8192 bytes it uses a buffer > on stack, so when using those old APIs prefer to chunk at 8192 bytes. > This is another argument that my naive proposal of not caching more buffers than some limited size (e.g. 1 MB) may not be totally crazy: it wouldn't be worse than traditional Java I/O, that executes malloc and free for each I/O. -- Evan Jones http://evanjones.ca/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ej at evanjones.ca Wed Dec 30 14:49:19 2015 From: ej at evanjones.ca (Evan Jones) Date: Wed, 30 Dec 2015 09:49:19 -0500 Subject: sun.nio.ch.Util: Don't cache an unlimited amount of memory In-Reply-To: <1451408221.3301219.478528674.0952CE75@webmail.messagingengine.com> References: <56824FA1.5050808@oracle.com> <1451408221.3301219.478528674.0952CE75@webmail.messagingengine.com> Message-ID: On Tue, Dec 29, 2015 at 11:57 AM, Ariel Weisberg wrote: > My intuition is that something based on breaking down IOs is not going o > pass muster, but it exists. > This would probably be ideal, but I'd be worried about changing the application semantics. This would definitely be a more invasive change, so I'm not about to attempt that. Could this be handled better with a different pooling approach that is not > per thread over some threshold? There are a bunch of permutations that > could trade off allocation/deallocation for concurrency vs. blocking > threads waiting for operations to complete freeing up globally pooled > buffers to bound peak footprint. > This is starting to feel like implementing malloc() :) Seriously though: this is another reasonable proposal that would achieve my goal of preventing others from being surprised by enormous native memory usage due to this issue. I have a bit of time available to attempt to write a patch. If an OpenJDK committer is interested in helping figure out a fix, I'd be happy to help. Current proposals: 1. Naive per-thread limit (I think this is a better trade-off than the current version, but will have a small CPU cost for I/Os larger than the limit) 2. Estimate recent "max usage" and shrink to fit: Probably a superior solution, but will be more complex. 3. Combination of per-thread cache, plus possibly a global cache for big buffers or something similar? I'm not 100% sure what parameters would make sense here. -- Evan Jones http://evanjones.ca/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Dec 30 14:53:23 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 30 Dec 2015 14:53:23 +0000 Subject: sun.nio.ch.Util: Don't cache an unlimited amount of memory In-Reply-To: References: <56824FA1.5050808@oracle.com> Message-ID: <5683EFE3.9080501@oracle.com> On 29/12/2015 15:59, Evan Jones wrote: > : > > My (simple) proposal will have a performance impact for applications > that do large I/O with heap ByteBuffers. However, I would argue those > apps are already slow because of the copy, and probably won't notice :). > > Would you be interested in a more sophisticated solution that would > allow huge cached buffers to eventually expire? For example, if the > "recent" usage of the cache has been much smaller than the allocated > buffers, free the buffers and re-allocate? For the problematic > application that I found, *any* policy for when to expire would > effectively solve this problem, since the "peak" usage is > significantly worse than the "average" usage. A more sophisticated solution makes it harder to explain when examining the memory usage so something simple might not be too bad. I would suggest go ahead and propose something here, it sounds like you might already have -Djdk.nio.maxCachedBufferSize=1m like solution already. > > My final suggestion: I would be happy to attempt to revise the javadoc > about direct buffers to make it clearer that using heap buffers for > I/O will cause copies, and will also cause the JDK to cache native > memory. At least then we could argue that this behaviour is "as > designed." :) An @implNote for SocketChannel and FileChannel (mostly these two) might be helpful. -Alan From sebastian.sickelmann at gmx.de Wed Dec 30 18:37:24 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Wed, 30 Dec 2015 19:37:24 +0100 Subject: RFR: 8145981 (fs) LinuxWatchService can reports events against wrong directory In-Reply-To: <5682DC69.1000704@oracle.com> References: <5680DC4C.3080104@oracle.com> <568149B4.80805@gmx.de> <56816E38.10308@oracle.com> <568184CE.8040401@gmx.de> <56824A8B.3030004@oracle.com> <5682A679.9010904@gmx.de> <5682BFEA.5060406@oracle.com> <5682D4A7.1000405@gmx.de> <5682DC69.1000704@oracle.com> Message-ID: <56842464.7050606@gmx.de> On 12/29/2015 08:18 PM, Alan Bateman wrote: > > > On 29/12/2015 18:44, Sebastian Sickelmann wrote: >> : >> How does it work with a jdk8 backport? I think the issue-number 8145981 >> would be burned with this patch. >> Should we create a new subtask of the issue for this jdk9 patch? >> > There is an issue type "backport" when changes are back-ported. The > backport issue will be created automatically when it is pushed to > jdk8u-dev. It can be created in advance too. The change-set comment > will have the original issue number. Best to give the fix a bit of > time to bed down in jdk9/dev before requesting approval to back-port. > The main thing here will being confident that the test is stable on > all platforms. > > BTW: In the test then do you mind I change "service" to "watcher" to > make it clearer and also consistent with the existing tests? I updated the changeset, if you like you can use it. http://cr.openjdk.java.net/~sebastian/8145981/webrev.03/ -- Sebastian > > -Alan > > > From Alan.Bateman at oracle.com Thu Dec 31 07:46:28 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 31 Dec 2015 07:46:28 +0000 Subject: RFR 8144355: JDK 9 changes to ZipFileSystem to support multi-release jar files In-Reply-To: <4FD89CC1-29C3-48D8-B750-0FB4BB8E080B@oracle.com> References: <367E2B35-46DA-45A7-8973-5D0049BCAF1A@oracle.com> <566F0087.8040000@oracle.com> <51070231-9486-4D31-9AF3-03D0CCA62AF5@oracle.com> <56740B82.2030901@oracle.com> <45A25AF5-665F-44D0-ADC2-6EA99F76B689@oracle.com> <5682BF49.7010408@oracle.com> <7E10E5DD-2563-4F92-B217-F843256C91EA@oracle.com> <4FD89CC1-29C3-48D8-B750-0FB4BB8E080B@oracle.com> Message-ID: <5684DD54.5010305@oracle.com> On 30/12/2015 02:28, Steve Drach wrote: > : > Fixed. > > The diff is attached: > Thanks, this is now pushed to jdk9/dev. -Alan. From Alan.Bateman at oracle.com Thu Dec 31 07:47:05 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 31 Dec 2015 07:47:05 +0000 Subject: RFR: 8145981 (fs) LinuxWatchService can reports events against wrong directory In-Reply-To: <56842464.7050606@gmx.de> References: <5680DC4C.3080104@oracle.com> <568149B4.80805@gmx.de> <56816E38.10308@oracle.com> <568184CE.8040401@gmx.de> <56824A8B.3030004@oracle.com> <5682A679.9010904@gmx.de> <5682BFEA.5060406@oracle.com> <5682D4A7.1000405@gmx.de> <5682DC69.1000704@oracle.com> <56842464.7050606@gmx.de> Message-ID: <5684DD79.1010302@oracle.com> On 30/12/2015 18:37, Sebastian Sickelmann wrote: > On 12/29/2015 08:18 PM, Alan Bateman wrote: >> : >> >> >> BTW: In the test then do you mind I change "service" to "watcher" to >> make it clearer and also consistent with the existing tests? > I updated the changeset, if you like you can use it. > > http://cr.openjdk.java.net/~sebastian/8145981/webrev.03/ > > Thanks, I think this is good so I've pushed your change-set to jdk9/dev. -Alan.