From brian.burkhalter at oracle.com Fri Jul 1 19:27:02 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 1 Jul 2016 12:27:02 -0700 Subject: JDK 9 RFR of 8160018: (fs) Remove GioFileTypeDetector on Solaris In-Reply-To: <9656D819-BD9A-4A88-9FFE-67715D0196DC@oracle.com> References: <9656D819-BD9A-4A88-9FFE-67715D0196DC@oracle.com> Message-ID: <2F22A496-146D-43B5-B20D-7F694ED210AC@oracle.com> An updated patch is viewable at http://cr.openjdk.java.net/~bpb/8160018/webrev.01/ It includes a patch to remove a reference to previously excised sun/nio/fs/GnomeFileTypeDetector.java from the top level CompileJavaModules.gmk. I thought that this is sufficiently related to be included here. A regression run with these patches applies passed. Thanks, Brian On Jun 30, 2016, at 3:06 PM, Brian Burkhalter wrote: > Hi Roger, > > Good point. Now that you mention it I think the GioFileTypeDetector.java should be moved from under ?unix? to under ?linux? as now it is only used on Linux. I?ll create another webrev. > > Thanks, > > Brian > > On Jun 30, 2016, at 2:28 PM, Roger Riggs wrote: > >> Hi Brian, >> >> Should the GioFileTypeDetector.java also be excluded from the Solaris build to save a bit of space and avoid confusion? >> >> The rest looks ok. >> >> Roger >> >> >> On 6/30/2016 4:19 PM, Brian Burkhalter wrote: >>> Please review at your convenience. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8160018 >>> Patch: http://cr.openjdk.java.net/~bpb/8160018/ >>> >>> The Solaris builds succeed and the core NIO regression tests pass with this change applied. >>> >>> Thanks, >>> >>> Brian >> > From brian.burkhalter at oracle.com Fri Jul 1 19:32:31 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 1 Jul 2016 12:32:31 -0700 Subject: JDK 9 RFR of 8160687: (fs) Cannot tell which WatchService test is not deleting temp directories "work*" Message-ID: <2B59AA23-5FFD-4658-88E3-6170B64F662A@oracle.com> Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8160687 Diff: included below Regression test run succeeded. Thanks, Brian --- a/test/java/nio/file/WatchService/DeleteInterference.java +++ b/test/java/nio/file/WatchService/DeleteInterference.java @@ -49,7 +49,7 @@ * directory. */ public static void main(String[] args) throws Exception { - Path dir = Files.createTempDirectory("work"); + Path dir = Files.createTempDirectory(?DeleteInterference"); ExecutorService pool = Executors.newCachedThreadPool(); try { Future task1 = pool.submit(() -> openAndCloseWatcher(dir)); --- a/test/java/nio/file/WatchService/LotsOfCancels.java +++ b/test/java/nio/file/WatchService/LotsOfCancels.java @@ -50,7 +50,7 @@ // one to bash on cancel, the other to poll the events ExecutorService pool = Executors.newCachedThreadPool(); try { - Path top = Files.createTempDirectory("work"); + Path top = Files.createTempDirectory(?LotsOfCancels"); top.toFile().deleteOnExit(); for (int i=1; i<=16; i++) { Path dir = Files.createDirectory(top.resolve("dir-" + i)); -------------- next part -------------- An HTML attachment was scrubbed... URL: From Roger.Riggs at Oracle.com Fri Jul 1 19:43:30 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 1 Jul 2016 15:43:30 -0400 Subject: JDK 9 RFR of 8160018: (fs) Remove GioFileTypeDetector on Solaris In-Reply-To: <2F22A496-146D-43B5-B20D-7F694ED210AC@oracle.com> References: <9656D819-BD9A-4A88-9FFE-67715D0196DC@oracle.com> <2F22A496-146D-43B5-B20D-7F694ED210AC@oracle.com> Message-ID: <09c7112d-078d-e072-73b9-8dcde450783d@Oracle.com> Hi Brian, Looks fine, thanks for taking another look and doing the cleanup. Roger On 7/1/2016 3:27 PM, Brian Burkhalter wrote: > An updated patch is viewable at > > http://cr.openjdk.java.net/~bpb/8160018/webrev.01/ > > It includes a patch to remove a reference to previously excised sun/nio/fs/GnomeFileTypeDetector.java from the top level CompileJavaModules.gmk. I thought that this is sufficiently related to be included here. > > A regression run with these patches applies passed. > > Thanks, > > Brian > > On Jun 30, 2016, at 3:06 PM, Brian Burkhalter wrote: > >> Hi Roger, >> >> Good point. Now that you mention it I think the GioFileTypeDetector.java should be moved from under ?unix? to under ?linux? as now it is only used on Linux. I?ll create another webrev. >> >> Thanks, >> >> Brian >> >> On Jun 30, 2016, at 2:28 PM, Roger Riggs wrote: >> >>> Hi Brian, >>> >>> Should the GioFileTypeDetector.java also be excluded from the Solaris build to save a bit of space and avoid confusion? >>> >>> The rest looks ok. >>> >>> Roger >>> >>> >>> On 6/30/2016 4:19 PM, Brian Burkhalter wrote: >>>> Please review at your convenience. >>>> >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8160018 >>>> Patch: http://cr.openjdk.java.net/~bpb/8160018/ >>>> >>>> The Solaris builds succeed and the core NIO regression tests pass with this change applied. >>>> >>>> Thanks, >>>> >>>> Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Roger.Riggs at Oracle.com Fri Jul 1 19:45:22 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 1 Jul 2016 15:45:22 -0400 Subject: JDK 9 RFR of 8160687: (fs) Cannot tell which WatchService test is not deleting temp directories "work*" In-Reply-To: <2B59AA23-5FFD-4658-88E3-6170B64F662A@oracle.com> References: <2B59AA23-5FFD-4658-88E3-6170B64F662A@oracle.com> Message-ID: <966d7652-46b1-8836-307b-ff8b28d2ef99@Oracle.com> Looks fine for an added diagnostic. Roger On 7/1/2016 3:32 PM, Brian Burkhalter wrote: > Please review at your convenience. > > Issue:https://bugs.openjdk.java.net/browse/JDK-8160687 > Diff:included below > > Regression test run succeeded. > > Thanks, > > Brian > > --- a/test/java/nio/file/WatchService/DeleteInterference.java > +++ b/test/java/nio/file/WatchService/DeleteInterference.java > @@ -49,7 +49,7 @@ > * directory. > */ > public static void main(String[] args) throws Exception { > - Path dir = Files.createTempDirectory("work"); > + Path dir = Files.createTempDirectory(?DeleteInterference"); > ExecutorService pool = Executors.newCachedThreadPool(); > try { > Future task1 = pool.submit(() -> openAndCloseWatcher(dir)); > > --- a/test/java/nio/file/WatchService/LotsOfCancels.java > +++ b/test/java/nio/file/WatchService/LotsOfCancels.java > @@ -50,7 +50,7 @@ > // one to bash on cancel, the other to poll the events > ExecutorService pool = Executors.newCachedThreadPool(); > try { > - Path top = Files.createTempDirectory("work"); > + Path top = Files.createTempDirectory(?LotsOfCancels"); > top.toFile().deleteOnExit(); > for (int i=1; i<=16; i++) { > Path dir = Files.createDirectory(top.resolve("dir-" + i)); From Alan.Bateman at oracle.com Tue Jul 5 12:55:49 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 5 Jul 2016 13:55:49 +0100 Subject: JDK 9 RFR of 8160687: (fs) Cannot tell which WatchService test is not deleting temp directories "work*" In-Reply-To: <2B59AA23-5FFD-4658-88E3-6170B64F662A@oracle.com> References: <2B59AA23-5FFD-4658-88E3-6170B64F662A@oracle.com> Message-ID: On 01/07/2016 20:32, Brian Burkhalter wrote: > Please review at your convenience. > > Issue:https://bugs.openjdk.java.net/browse/JDK-8160687 > Diff:included below > > The simplest might be to just change it to use a sub-directory in the current directory (meaning the jtreg work directory). That will ensure that it gets cleaned up when the work directory is blown away. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Jul 5 12:57:15 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 5 Jul 2016 13:57:15 +0100 Subject: JDK 9 RFR of 8160018: (fs) Remove GioFileTypeDetector on Solaris In-Reply-To: References: Message-ID: <16abee48-a50a-2d1c-5301-7a607032edc3@oracle.com> On 30/06/2016 21:19, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8160018 > Patch: http://cr.openjdk.java.net/~bpb/8160018/ > > The Solaris builds succeed and the core NIO regression tests pass with this change applied. > This FileTypeDetector isn't Linux specific so probably best not to move it. That said, maybe it's time to drop it from Linux too. -Alan From brian.burkhalter at oracle.com Wed Jul 6 00:30:42 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 5 Jul 2016 17:30:42 -0700 Subject: JDK 9 RFR of 8160687: (fs) Cannot tell which WatchService test is not deleting temp directories "work*" In-Reply-To: References: <2B59AA23-5FFD-4658-88E3-6170B64F662A@oracle.com> Message-ID: On Jul 5, 2016, at 5:55 AM, Alan Bateman wrote: > On 01/07/2016 20:32, Brian Burkhalter wrote: >> Please review at your convenience. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8160687 >> Diff: included below >> >> > The simplest might be to just change it to use a sub-directory in the current directory (meaning the jtreg work directory). That will ensure that it gets cleaned up when the work directory is blown away. This change was already pushed last week. Should a follow up issue to do the above be filed? Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jul 6 00:31:13 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 5 Jul 2016 17:31:13 -0700 Subject: JDK 9 RFR of 8160018: (fs) Remove GioFileTypeDetector on Solaris In-Reply-To: <16abee48-a50a-2d1c-5301-7a607032edc3@oracle.com> References: <16abee48-a50a-2d1c-5301-7a607032edc3@oracle.com> Message-ID: <62984E86-A8C9-4B57-B63A-821E713A30C4@oracle.com> On Jul 5, 2016, at 5:57 AM, Alan Bateman wrote: > On 30/06/2016 21:19, Brian Burkhalter wrote: >> Please review at your convenience. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8160018 >> Patch: http://cr.openjdk.java.net/~bpb/8160018/ >> >> The Solaris builds succeed and the core NIO regression tests pass with this change applied. >> > This FileTypeDetector isn't Linux specific so probably best not to move it. That said, maybe it's time to drop it from Linux too. The patch was pushed last week so it has already been moved. Do you think that another issue should be filed to move the GioFileTypeDetector back under the unix source tree? While in concept it could be used by other Unix variants, in the JDK it is now only being used by the FileSystemProvider for Linux. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Jul 6 06:02:11 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Jul 2016 07:02:11 +0100 Subject: JDK 9 RFR of 8160018: (fs) Remove GioFileTypeDetector on Solaris In-Reply-To: <62984E86-A8C9-4B57-B63A-821E713A30C4@oracle.com> References: <16abee48-a50a-2d1c-5301-7a607032edc3@oracle.com> <62984E86-A8C9-4B57-B63A-821E713A30C4@oracle.com> Message-ID: <5d4a67cb-5d9e-6a2a-a1c6-b689b215c2bb@oracle.com> On 06/07/2016 01:31, Brian Burkhalter wrote: > > The patch was pushed last week so it has already been moved. Do you > think that another issue should be filed to move the > GioFileTypeDetector back under the unix source tree? While in concept > it could be used by other Unix variants, in the JDK it is now only > being used by the FileSystemProvider for Linux. > It would be nice if we could remove it completely but if we have to keep it then I assume the unix tree is the right place for it. -Alan From Alan.Bateman at oracle.com Wed Jul 6 06:29:00 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Jul 2016 07:29:00 +0100 Subject: JDK 9 RFR of 8160687: (fs) Cannot tell which WatchService test is not deleting temp directories "work*" In-Reply-To: References: <2B59AA23-5FFD-4658-88E3-6170B64F662A@oracle.com> Message-ID: <3f4dc386-d6f3-6529-5762-352afda96ea2@oracle.com> On 06/07/2016 01:30, Brian Burkhalter wrote: > On Jul 5, 2016, at 5:55 AM, Alan Bateman > wrote: > >> On 01/07/2016 20:32, Brian Burkhalter wrote: >>> Please review at your convenience. >>> >>> Issue:https://bugs.openjdk.java.net/browse/JDK-8160687 >>> Diff:included below >>> >>> >> The simplest might be to just change it to use a sub-directory in the >> current directory (meaning the jtreg work directory). That will >> ensure that it gets cleaned up when the work directory is blown away. > > This change was already pushed last week. Should a follow up issue to > do the above be filed? > I assume that as part of the changes for JDK-8160220 that the tests will be changed to not use the tmp file system. So I think that will cover it. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jul 6 17:31:33 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 6 Jul 2016 10:31:33 -0700 Subject: JDK 9 RFR of 8160018: (fs) Remove GioFileTypeDetector on Solaris In-Reply-To: <5d4a67cb-5d9e-6a2a-a1c6-b689b215c2bb@oracle.com> References: <16abee48-a50a-2d1c-5301-7a607032edc3@oracle.com> <62984E86-A8C9-4B57-B63A-821E713A30C4@oracle.com> <5d4a67cb-5d9e-6a2a-a1c6-b689b215c2bb@oracle.com> Message-ID: <1C6986D7-3689-499B-8916-1D43B08D859D@oracle.com> On Jul 5, 2016, at 11:02 PM, Alan Bateman wrote: >> The patch was pushed last week so it has already been moved. Do you think that another issue should be filed to move the GioFileTypeDetector back under the unix source tree? While in concept it could be used by other Unix variants, in the JDK it is now only being used by the FileSystemProvider for Linux. >> > It would be nice if we could remove it completely but if we have to keep it then I assume the unix tree is the right place for it. OK I filed https://bugs.openjdk.java.net/browse/JDK-8160903 to move it back under the Unix source tree. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jul 6 17:32:39 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 6 Jul 2016 10:32:39 -0700 Subject: JDK 9 RFR of 8160687: (fs) Cannot tell which WatchService test is not deleting temp directories "work*" In-Reply-To: <3f4dc386-d6f3-6529-5762-352afda96ea2@oracle.com> References: <2B59AA23-5FFD-4658-88E3-6170B64F662A@oracle.com> <3f4dc386-d6f3-6529-5762-352afda96ea2@oracle.com> Message-ID: On Jul 5, 2016, at 11:29 PM, Alan Bateman wrote: >> This change was already pushed last week. Should a follow up issue to do the above be filed? >> > I assume that as part of the changes for JDK-8160220 that the tests will be changed to not use the tmp file system. So I think that will cover it. Yes it should. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Jul 8 21:02:50 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 8 Jul 2016 14:02:50 -0700 Subject: JDK 9 RFR of 8157287: java/nio/file/FileStore/Basic.java failed with "java.nio.file.AccessDeniedException : /zones/zoneone/root " Message-ID: Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8157287 When testing whether space attributes are available, skip the store if an AccessDeniedException is encountered. Thanks, Brian --- a/test/java/nio/file/FileStore/Basic.java +++ b/test/java/nio/file/FileStore/Basic.java @@ -114,21 +114,27 @@ for (FileStore store: FileSystems.getDefault().getFileStores()) { System.out.format("%s (name=%s type=%s)\n", store, store.name(), store.type()); // check space attributes are accessible try { store.getTotalSpace(); store.getUnallocatedSpace(); store.getUsableSpace(); } catch (NoSuchFileException nsfe) { // ignore exception as the store could have been // deleted since the iterator was instantiated System.err.format("%s was not found\n", store); + } catch (AccessDeniedException ade) { + // ignore exception as the lack of ability to access the + // store due to lack of file permission or similar does not + // reflect whether the space attributes would be accessible + // were access to be permitted + System.err.format("%s is inaccessible\n", store); } // two distinct FileStores should not be equal assertTrue(!store.equals(prev)); prev = store; } -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Jul 8 21:10:17 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 8 Jul 2016 22:10:17 +0100 Subject: JDK 9 RFR of 8157287: java/nio/file/FileStore/Basic.java failed with "java.nio.file.AccessDeniedException : /zones/zoneone/root " In-Reply-To: References: Message-ID: <2fbf623d-df93-94fd-faec-fcbd0bf0385b@oracle.com> On 08/07/2016 22:02, Brian Burkhalter wrote: > Please review at your convenience. > > Issue:https://bugs.openjdk.java.net/browse/JDK-8157287 > > When testing whether space attributes are available, skip the store if > an AccessDeniedException is encountered. > This looks okay. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Sat Jul 9 00:22:41 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 8 Jul 2016 17:22:41 -0700 Subject: JDK 9 RFR of 8160221: jdk/test/java/nio/channels/FileChannel/Transfers.java leaving files behind Message-ID: <4B501CCA-5201-4C9F-A595-83D949E4BB3F@oracle.com> Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8160221 Diff: --- a/test/java/nio/channels/FileChannel/Transfers.java +++ b/test/java/nio/channels/FileChannel/Transfers.java @@ -489,12 +489,14 @@ debug = verbose = true; } - sourceFile = File.createTempFile("xfer.src.", ""); + File testDir = new File(System.getProperty("test.dir", ".")); + + sourceFile = File.createTempFile("xfer.src.", "", testDir); sourceFile.deleteOnExit(); - targetFile = File.createTempFile("xfer.tgt.", ""); + targetFile = File.createTempFile("xfer.tgt.", "", testDir); targetFile.deleteOnExit(); - File fn = File.createTempFile("xfer.fch.", ""); + File fn = File.createTempFile("xfer.fch.", "", testDir); fn.deleteOnExit(); FileChannel fc = new RandomAccessFile(fn, "rw").getChannel(); Summary: Create temporary files in the test scratch directory instead of in the default temporary file directory. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon Jul 11 07:56:52 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Jul 2016 08:56:52 +0100 Subject: JDK 9 RFR of 8160221: jdk/test/java/nio/channels/FileChannel/Transfers.java leaving files behind In-Reply-To: <4B501CCA-5201-4C9F-A595-83D949E4BB3F@oracle.com> References: <4B501CCA-5201-4C9F-A595-83D949E4BB3F@oracle.com> Message-ID: <1c0014e2-60bc-705e-c641-adaa9a56f977@oracle.com> Looks okay to me. On 09/07/2016 01:22, Brian Burkhalter wrote: > Please review at your convenience. > > Issue:https://bugs.openjdk.java.net/browse/JDK-8160221 > > Diff: > > --- a/test/java/nio/channels/FileChannel/Transfers.java > +++ b/test/java/nio/channels/FileChannel/Transfers.java > @@ -489,12 +489,14 @@ > debug = verbose = true; > } > > > - sourceFile = File.createTempFile("xfer.src.", ""); > + File testDir = new File(System.getProperty("test.dir", ".")); > + > + sourceFile = File.createTempFile("xfer.src.", "", testDir); > sourceFile.deleteOnExit(); > - targetFile = File.createTempFile("xfer.tgt.", ""); > + targetFile = File.createTempFile("xfer.tgt.", "", testDir); > targetFile.deleteOnExit(); > > > - File fn = File.createTempFile("xfer.fch.", ""); > + File fn = File.createTempFile("xfer.fch.", "", testDir); > fn.deleteOnExit(); > FileChannel fc = new RandomAccessFile(fn, "rw").getChannel(); > > > Summary: Create temporary files in the test scratch directory instead > of in the default temporary file directory. > > Thanks, > > Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Mon Jul 11 20:30:22 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 11 Jul 2016 13:30:22 -0700 Subject: JDK 9 RFR of 8160831: GioFileTypeDetector always calls deprecated g_type_init Message-ID: <6C514BD0-A190-4B8A-BAD5-C37947185E58@oracle.com> Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8160831 Patch: http://cr.openjdk.java.net/~bpb/8160831/webrev.00/ Verify that g_type_init is non-null before dereferencing it. Thanks, Brian From Alan.Bateman at oracle.com Mon Jul 11 21:24:29 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Jul 2016 22:24:29 +0100 Subject: JDK 9 RFR of 8160831: GioFileTypeDetector always calls deprecated g_type_init In-Reply-To: <6C514BD0-A190-4B8A-BAD5-C37947185E58@oracle.com> References: <6C514BD0-A190-4B8A-BAD5-C37947185E58@oracle.com> Message-ID: <03e4ad0b-d4a4-41b5-8623-198073038b2d@oracle.com> On 11/07/2016 21:30, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8160831 > Patch: http://cr.openjdk.java.net/~bpb/8160831/webrev.00/ > > Verify that g_type_init is non-null before dereferencing it. > Is there an alternative init function to use when g_type_init is not present? I can't quite tell this from the bug. In any case I wonder if it's time to just remove this file type detector. -Alan From brian.burkhalter at oracle.com Mon Jul 11 21:26:08 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 11 Jul 2016 14:26:08 -0700 Subject: JDK 9 RFR of 8160831: GioFileTypeDetector always calls deprecated g_type_init In-Reply-To: <03e4ad0b-d4a4-41b5-8623-198073038b2d@oracle.com> References: <6C514BD0-A190-4B8A-BAD5-C37947185E58@oracle.com> <03e4ad0b-d4a4-41b5-8623-198073038b2d@oracle.com> Message-ID: On Jul 11, 2016, at 2:24 PM, Alan Bateman wrote: > On 11/07/2016 21:30, Brian Burkhalter wrote: > >> Please review at your convenience. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8160831 >> Patch: http://cr.openjdk.java.net/~bpb/8160831/webrev.00/ >> >> Verify that g_type_init is non-null before dereferencing it. >> > Is there an alternative init function to use when g_type_init is not present? No. The GLib documentation states that from 2.36 onward the type initialization is automatic. > I can't quite tell this from the bug. In any case I wonder if it's time to just remove this file type detector. Perhaps, but it might nevertheless not be a bad idea to correct this first. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Mon Jul 11 21:27:03 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 11 Jul 2016 14:27:03 -0700 Subject: JDK 9 RFR of 8160831: GioFileTypeDetector always calls deprecated g_type_init In-Reply-To: References: <6C514BD0-A190-4B8A-BAD5-C37947185E58@oracle.com> <03e4ad0b-d4a4-41b5-8623-198073038b2d@oracle.com> Message-ID: <0E8872D2-BD21-4906-A600-4B438EF6DB67@oracle.com> On Jul 11, 2016, at 2:26 PM, Brian Burkhalter wrote: >>> Verify that g_type_init is non-null before dereferencing it. >>> >> Is there an alternative init function to use when g_type_init is not present? > > No. The GLib documentation states that from 2.36 onward the type initialization is automatic. [1] [1] https://developer.gnome.org/gobject/unstable/gobject-Type-Information.html#g-type-init Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Mon Jul 11 21:29:39 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 11 Jul 2016 14:29:39 -0700 Subject: JDK 9 RFR of 8160221: jdk/test/java/nio/channels/FileChannel/Transfers.java leaving files behind In-Reply-To: <1c0014e2-60bc-705e-c641-adaa9a56f977@oracle.com> References: <4B501CCA-5201-4C9F-A595-83D949E4BB3F@oracle.com> <1c0014e2-60bc-705e-c641-adaa9a56f977@oracle.com> Message-ID: <63D9CB4E-BBF9-4E1C-B8D7-9CF43A82B61F@oracle.com> For some as yet unknown reason a regression test run strangely had an error for this change on Windows only. Need to run that down before pushing this. Brian On Jul 11, 2016, at 12:56 AM, Alan Bateman wrote: > Looks okay to me. > > On 09/07/2016 01:22, Brian Burkhalter wrote: >> Please review at your convenience. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8160221 >> Diff: >> >> --- a/test/java/nio/channels/FileChannel/Transfers.java >> +++ b/test/java/nio/channels/FileChannel/Transfers.java >> @@ -489,12 +489,14 @@ >> debug = verbose = true; >> } >> >> - sourceFile = File.createTempFile("xfer.src.", ""); >> + File testDir = new File(System.getProperty("test.dir", ".")); >> + >> + sourceFile = File.createTempFile("xfer.src.", "", testDir); >> sourceFile.deleteOnExit(); >> - targetFile = File.createTempFile("xfer.tgt.", ""); >> + targetFile = File.createTempFile("xfer.tgt.", "", testDir); >> targetFile.deleteOnExit(); >> >> - File fn = File.createTempFile("xfer.fch.", ""); >> + File fn = File.createTempFile("xfer.fch.", "", testDir); >> fn.deleteOnExit(); >> FileChannel fc = new RandomAccessFile(fn, "rw").getChannel(); >> >> >> Summary: Create temporary files in the test scratch directory instead of in the default temporary file directory. >> >> Thanks, >> >> Brian > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon Jul 11 21:39:46 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Jul 2016 22:39:46 +0100 Subject: JDK 9 RFR of 8160831: GioFileTypeDetector always calls deprecated g_type_init In-Reply-To: References: <6C514BD0-A190-4B8A-BAD5-C37947185E58@oracle.com> <03e4ad0b-d4a4-41b5-8623-198073038b2d@oracle.com> Message-ID: On 11/07/2016 22:26, Brian Burkhalter wrote: > > On Jul 11, 2016, at 2:24 PM, Alan Bateman > wrote: > >> On 11/07/2016 21:30, Brian Burkhalter wrote: >> >>> Please review at your convenience. >>> >>> Issue:https://bugs.openjdk.java.net/browse/JDK-8160831 >>> Patch:http://cr.openjdk.java.net/~bpb/8160831/webrev.00/ >>> >>> >>> Verify that g_type_init is non-null before dereferencing it. >>> >> Is there an alternative init function to use when g_type_init is not >> present? > > No. The GLib documentation states that from 2.36 onward the type > initialization is automatic. > Okay, thanks for checking. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Mon Jul 11 21:40:55 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 11 Jul 2016 14:40:55 -0700 Subject: JDK 9 RFR of 8160831: GioFileTypeDetector always calls deprecated g_type_init In-Reply-To: References: <6C514BD0-A190-4B8A-BAD5-C37947185E58@oracle.com> <03e4ad0b-d4a4-41b5-8623-198073038b2d@oracle.com> Message-ID: <6B1F1948-0B4E-461A-9814-C5BED6C386E7@oracle.com> On Jul 11, 2016, at 2:39 PM, Alan Bateman wrote: >>> Is there an alternative init function to use when g_type_init is not present? >> >> No. The GLib documentation states that from 2.36 onward the type initialization is automatic. >> > Okay, thanks for checking. I should have put it in the original post. I assume this is a +1. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Jul 12 00:21:32 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 11 Jul 2016 17:21:32 -0700 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind Message-ID: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8160220 Patch: http://cr.openjdk.java.net/~bpb/8160220/webrev.00/ Summary: 1) DeleteInterference & LotsOfCancels Create temporary files in a directory specifiable via the ?test.dir? property with default ?.?. 2) LotsOfCancels Add (probably redundant) deletion of temporary file tree. Thanks, Brian From Alan.Bateman at oracle.com Tue Jul 12 06:28:29 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 12 Jul 2016 07:28:29 +0100 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> Message-ID: On 12/07/2016 01:21, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8160220 > Patch: http://cr.openjdk.java.net/~bpb/8160220/webrev.00/ > > Summary: > > 1) DeleteInterference & LotsOfCancels > > Create temporary files in a directory specifiable via the ?test.dir? property with default ?.?. This looks okay although I assume you don't need createTempDirectory, it can be just a fixed sub-directory of the current directory (up to you). > > 2) LotsOfCancels > > Add (probably redundant) deletion of temporary file tree. > The tests in this area use TestUtil.removeAll so you don't need to add deleteFileTree. Also if you expand the scope of the try/finally then you can get rid of deleteOnExit. -Alan From brian.burkhalter at oracle.com Tue Jul 12 15:01:38 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 12 Jul 2016 08:01:38 -0700 Subject: JDK 9 RFR of 8160221: jdk/test/java/nio/channels/FileChannel/Transfers.java leaving files behind In-Reply-To: <63D9CB4E-BBF9-4E1C-B8D7-9CF43A82B61F@oracle.com> References: <4B501CCA-5201-4C9F-A595-83D949E4BB3F@oracle.com> <1c0014e2-60bc-705e-c641-adaa9a56f977@oracle.com> <63D9CB4E-BBF9-4E1C-B8D7-9CF43A82B61F@oracle.com> Message-ID: <0DFCCD8C-60C6-433F-9E4B-36F031914105@oracle.com> A failure was still being observed on Windows during jtreg cleanup due to a file which could not be deleted as a FileChannel still held it open. This patch http://cr.openjdk.java.net/~bpb/8160221/webrev.01/ fixes that problem at line 561 (verified). Thanks, Brian On Jul 11, 2016, at 2:29 PM, Brian Burkhalter wrote: > For some as yet unknown reason a regression test run strangely had an error for this change on Windows only. Need to run that down before pushing this. > > Brian > > On Jul 11, 2016, at 12:56 AM, Alan Bateman wrote: > >> Looks okay to me. >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Jul 12 16:40:05 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 12 Jul 2016 09:40:05 -0700 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> Message-ID: On Jul 11, 2016, at 11:28 PM, Alan Bateman wrote: >> 2) LotsOfCancels >> >> Add (probably redundant) deletion of temporary file tree. >> > The tests in this area use TestUtil.removeAll so you don't need to add deleteFileTree. Also if you expand the scope of the try/finally then you can get rid of deleteOnExit. The change for https://bugs.openjdk.java.net/browse/JDK-8160687 showed that LotsOfCancels was leaving files around before this change so there must be something else going on which needs to be identified. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Jul 12 16:57:00 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 12 Jul 2016 17:57:00 +0100 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> Message-ID: <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> On 12/07/2016 17:40, Brian Burkhalter wrote: > On Jul 11, 2016, at 11:28 PM, Alan Bateman > wrote: > >>> 2) LotsOfCancels >>> >>> Add (probably redundant) deletion of temporary file tree. >>> >> The tests in this area use TestUtil.removeAll so you don't need to >> add deleteFileTree. Also if you expand the scope of the try/finally >> then you can get rid of deleteOnExit. > > The change for https://bugs.openjdk.java.net/browse/JDK-8160687 showed > that LotsOfCancels was leaving files around before this change so > there must be something else going on which needs to be identified. > Yes, I suspect there was a VM crash at some point in the VM that happen to have run this test. The main issue with agentvm mode is that it runs many tests before it exists and so the delete-on-exit list builds up. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Jul 12 17:51:15 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 12 Jul 2016 10:51:15 -0700 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> Message-ID: On Jul 12, 2016, at 9:57 AM, Alan Bateman wrote: >>> The tests in this area use TestUtil.removeAll so you don't need to add deleteFileTree. Also if you expand the scope of the try/finally then you can get rid of deleteOnExit. >> >> The change for https://bugs.openjdk.java.net/browse/JDK-8160687showed that LotsOfCancels was leaving files around before this change so there must be something else going on which needs to be identified. >> > Yes, I suspect there was a VM crash at some point in the VM that happen to have run this test. The main issue with agentvm mode is that it runs many tests before it exists and so the delete-on-exit list builds up. It does not sound as if it is the best idea to rely on deleteOnExit(). Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Tue Jul 12 19:17:50 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 12 Jul 2016 20:17:50 +0100 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> Message-ID: <729C0F2B-14DD-46DF-852B-3338080B7DA3@oracle.com> > On 12 Jul 2016, at 18:51, Brian Burkhalter wrote: > > On Jul 12, 2016, at 9:57 AM, Alan Bateman wrote: > >>>> The tests in this area use TestUtil.removeAll so you don't need to add deleteFileTree. Also if you expand the scope of the try/finally then you can get rid of deleteOnExit. >>> >>> The change for https://bugs.openjdk.java.net/browse/JDK-8160687showed that LotsOfCancels was leaving files around before this change so there must be something else going on which needs to be identified. >>> >> Yes, I suspect there was a VM crash at some point in the VM that happen to have run this test. The main issue with agentvm mode is that it runs many tests before it exists and so the delete-on-exit list builds up. > > It does not sound as if it is the best idea to rely on deleteOnExit(). I cannot find it now, but I added a comment to the same affect, to a recent similar issue. If you are going to explicitly delete the file / dir, please use one of the appropriate utilities in jdk.testlibrary.FileUtils. -Chris. From Alan.Bateman at oracle.com Tue Jul 12 19:41:22 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 12 Jul 2016 20:41:22 +0100 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> Message-ID: <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> On 12/07/2016 18:51, Brian Burkhalter wrote: > > It does not sound as if it is the best idea to rely on deleteOnExit(). > It was okay in the past when the test execution was othervm. Switching it to TestUtils.removeAll will make it consistent with the other tests in this area. -Alan From Alan.Bateman at oracle.com Tue Jul 12 19:54:00 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 12 Jul 2016 20:54:00 +0100 Subject: JDK 9 RFR of 8160221: jdk/test/java/nio/channels/FileChannel/Transfers.java leaving files behind In-Reply-To: <0DFCCD8C-60C6-433F-9E4B-36F031914105@oracle.com> References: <4B501CCA-5201-4C9F-A595-83D949E4BB3F@oracle.com> <1c0014e2-60bc-705e-c641-adaa9a56f977@oracle.com> <63D9CB4E-BBF9-4E1C-B8D7-9CF43A82B61F@oracle.com> <0DFCCD8C-60C6-433F-9E4B-36F031914105@oracle.com> Message-ID: <8fe64492-3024-1df2-6b0a-fe80633d5973@oracle.com> On 12/07/2016 16:01, Brian Burkhalter wrote: > A failure was still being observed on Windows during jtreg cleanup due > to a file which could not be deleted as a FileChannel still held it > open. This patch > > http://cr.openjdk.java.net/~bpb/8160221/webrev.01/ > > > fixes that problem at line 561 (verified). > > This looks okay, just wondering if you can use try-with-reosurces instead. -Alan From brian.burkhalter at oracle.com Tue Jul 12 20:36:38 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 12 Jul 2016 13:36:38 -0700 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> Message-ID: <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> On Jul 12, 2016, at 12:41 PM, Alan Bateman wrote: > On 12/07/2016 18:51, Brian Burkhalter wrote: >> >> It does not sound as if it is the best idea to rely on deleteOnExit(). >> > It was okay in the past when the test execution was othervm. Switching it to TestUtils.removeAll will make it consistent with the other tests in this area. Here?s an updated version which uses FileUtils.deleteFileTreeWithRetry() and does not use deleteOnExit(): http://cr.openjdk.java.net/~bpb/8160220/webrev.01/ Thanks, Brian From brian.burkhalter at oracle.com Wed Jul 13 00:22:25 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 12 Jul 2016 17:22:25 -0700 Subject: JDK 9 RFR of 8160221: jdk/test/java/nio/channels/FileChannel/Transfers.java leaving files behind In-Reply-To: <8fe64492-3024-1df2-6b0a-fe80633d5973@oracle.com> References: <4B501CCA-5201-4C9F-A595-83D949E4BB3F@oracle.com> <1c0014e2-60bc-705e-c641-adaa9a56f977@oracle.com> <63D9CB4E-BBF9-4E1C-B8D7-9CF43A82B61F@oracle.com> <0DFCCD8C-60C6-433F-9E4B-36F031914105@oracle.com> <8fe64492-3024-1df2-6b0a-fe80633d5973@oracle.com> Message-ID: On Jul 12, 2016, at 12:54 PM, Alan Bateman wrote: > On 12/07/2016 16:01, Brian Burkhalter wrote: >> A failure was still being observed on Windows during jtreg cleanup due to a file which could not be deleted as a FileChannel still held it open. This patch >> >> http://cr.openjdk.java.net/~bpb/8160221/webrev.01/ >> >> fixes that problem at line 561 (verified). >> >> > This looks okay, just wondering if you can use try-with-reosurces instead. Updated to use try-with-resources to auto-close the FileChannel: http://cr.openjdk.java.net/~bpb/8160221/webrev.02/ Regression test job succeeded. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Jul 13 05:36:33 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 13 Jul 2016 06:36:33 +0100 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> Message-ID: On 12/07/2016 21:36, Brian Burkhalter wrote: > : > Here?s an updated version which uses FileUtils.deleteFileTreeWithRetry() and does not use deleteOnExit(): > > http://cr.openjdk.java.net/~bpb/8160220/webrev.01/ > Thinking about, this isn't needed. Since you've moved to the work directory then jtreg will do the clean-up (including retry). So the change should be to just move from /tmp to the work directory and you should be all set. -Alan From Alan.Bateman at oracle.com Wed Jul 13 06:35:08 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 13 Jul 2016 07:35:08 +0100 Subject: JDK 9 RFR of 8160221: jdk/test/java/nio/channels/FileChannel/Transfers.java leaving files behind In-Reply-To: References: <4B501CCA-5201-4C9F-A595-83D949E4BB3F@oracle.com> <1c0014e2-60bc-705e-c641-adaa9a56f977@oracle.com> <63D9CB4E-BBF9-4E1C-B8D7-9CF43A82B61F@oracle.com> <0DFCCD8C-60C6-433F-9E4B-36F031914105@oracle.com> <8fe64492-3024-1df2-6b0a-fe80633d5973@oracle.com> Message-ID: On 13/07/2016 01:22, Brian Burkhalter wrote: > > > Updated to use try-with-resources to auto-close the FileChannel: > > http://cr.openjdk.java.net/~bpb/8160221/webrev.02/ > > > Regression test job succeeded. > Looks good. From brian.burkhalter at oracle.com Wed Jul 13 14:57:48 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 13 Jul 2016 07:57:48 -0700 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> Message-ID: <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> On Jul 12, 2016, at 10:36 PM, Alan Bateman wrote: > On 12/07/2016 21:36, Brian Burkhalter wrote: >> : >> Here?s an updated version which uses FileUtils.deleteFileTreeWithRetry() and does not use deleteOnExit(): >> >> http://cr.openjdk.java.net/~bpb/8160220/webrev.01/ >> > Thinking about, this isn't needed. Since you've moved to the work directory then jtreg will do the clean-up (including retry). So the change should be to just move from /tmp to the work directory and you should be all set. So I should be able to remove FileUtils.deleteFileTreeWithRetry() and the jdk.testlibrary references in *both* of these tests as well as the original deleteFileTree() in DeleteInterference. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Wed Jul 13 15:03:36 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 13 Jul 2016 16:03:36 +0100 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> Message-ID: <57865848.8090301@oracle.com> On 13/07/16 15:57, Brian Burkhalter wrote: > > On Jul 12, 2016, at 10:36 PM, Alan Bateman > wrote: > >> On 12/07/2016 21:36, Brian Burkhalter wrote: >>> : >>> Here?s an updated version which uses >>> FileUtils.deleteFileTreeWithRetry() and does not use deleteOnExit(): >>> >>> http://cr.openjdk.java.net/~bpb/8160220/webrev.01/ >>> >> Thinking about, this isn't needed. Since you've moved to the work >> directory then jtreg will do the clean-up (including retry). So the >> change should be to just move from /tmp to the work directory and you >> should be all set. > > So I should be able to remove FileUtils.deleteFileTreeWithRetry() and > the jdk.testlibrary references in *both* of these tests as well as the > original deleteFileTree() in DeleteInterference. If the tests are using the current working directory, then yes jtreg will clean up afterwards. The only reason to retain the explicit delete is if you intend to run the tests manually ( outside of jtreg ), then of course it will need to remove the files itself ( so as to not clutter up your directory ). Either way should be fine, just ensure, if you intend to delete the files explicitly, that there are enough bread crumbs left in the working directory if the test fails ( so you can duplicate/debug the issue ). -Chris. From christoph.langer at sap.com Wed Jul 13 15:29:07 2016 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 13 Jul 2016 15:29:07 +0000 Subject: Help needed: Java Socket and close detection Message-ID: <3c8a676e2c4f4760a7aaf07cc6c771f7@DEWDFE13DE11.global.corp.sap> Hi folks, I have a question to the experts - regarding an issue that was reported to me by a customer. In the customer scenario they are running a Servlet engine and the Servlet is constantly sending data to a browser client. When the browser client is closed, the server does not get a notification of the other end having been terminated and is constantly sending out data and blocking an application thread. I'm under the assumption that the server should get an RST packet from the network upon writing/flushing data to the OutputStream as soon as the client is gone and hence an Exception should pop up but this isn't happening. There is a load balancer and maybe other network infrastructure involved in between the Servlet JVM and the browser client. We did some TCPDUMP tracing at the load balancer and we could not see an RST packet coming in from the client side. But when I'm running the scenario without all this network infrastructure involved, e.g. between servers and clients in the same network, I would always observe an RST packet once I close the browser. A FIN packet is received, too, but this does not lead to an Exception and to all my knowledge this can't be detected, not from the java Socket API and even less from the Servlet API which is just dealing with Streams. So my question to the experts is most of all: Would you agree that an RST packet should be generated in the network and received by the server? Or is it a normal behavior that servers must deal with not receiving RSTs and hence needing to wait for a timeout until e.g. the load balancer generates an RST? Also, is there any way to detect a FIN in the JVM and react on it? Thanks in advance and best regards Christoph -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jul 13 15:51:55 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 13 Jul 2016 08:51:55 -0700 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <57865848.8090301@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> <57865848.8090301@oracle.com> Message-ID: On Jul 13, 2016, at 8:03 AM, Chris Hegarty wrote: > On 13/07/16 15:57, Brian Burkhalter wrote: >> >> On Jul 12, 2016, at 10:36 PM, Alan Bateman > > wrote: >> >>> On 12/07/2016 21:36, Brian Burkhalter wrote: >>>> : >>>> Here?s an updated version which uses >>>> FileUtils.deleteFileTreeWithRetry() and does not use deleteOnExit(): >>>> >>>> http://cr.openjdk.java.net/~bpb/8160220/webrev.01/ >>>> >>> Thinking about, this isn't needed. Since you've moved to the work >>> directory then jtreg will do the clean-up (including retry). So the >>> change should be to just move from /tmp to the work directory and you >>> should be all set. >> >> So I should be able to remove FileUtils.deleteFileTreeWithRetry() and >> the jdk.testlibrary references in *both* of these tests as well as the >> original deleteFileTree() in DeleteInterference. > > If the tests are using the current working directory, then yes jtreg > will clean up afterwards. The only reason to retain the explicit > delete is if you intend to run the tests manually ( outside of jtreg ), > then of course it will need to remove the files itself ( so as to not > clutter up your directory ). Either way should be fine, just ensure, > if you intend to delete the files explicitly, that there are enough > bread crumbs left in the working directory if the test fails ( so > you can duplicate/debug the issue ). This rather sounds as if the tests should use the current working directly and only do an explicit delete if the test fails outside of jtreg, e.g., the test.src property is undefined. If that is the conclusion then I don?t believe we have consistency among the tests at this time on this point. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Wed Jul 13 15:57:53 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 13 Jul 2016 16:57:53 +0100 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <57865848.8090301@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> <57865848.8090301@oracle.com> Message-ID: On 13/07/16 16:03, Chris Hegarty wrote: > Either way should be fine, just ensure, > if you intend to delete the files explicitly, that there are enough > bread crumbs left in the working directory if the test fails ( so > you can duplicate/debug the issue ). I usually rely on traces to do that - as in my experience the sneaky tests often fail in environments where you don't have access to the tests working directory anyway... cheers, -- daniel From Alan.Bateman at oracle.com Wed Jul 13 16:33:49 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 13 Jul 2016 17:33:49 +0100 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> Message-ID: <3171dc29-444f-f0c0-dc0c-601c8d426a0a@oracle.com> On 13/07/2016 15:57, Brian Burkhalter wrote: > > On Jul 12, 2016, at 10:36 PM, Alan Bateman > wrote: > >> On 12/07/2016 21:36, Brian Burkhalter wrote: >>> : >>> Here?s an updated version which uses >>> FileUtils.deleteFileTreeWithRetry() and does not use deleteOnExit(): >>> >>> http://cr.openjdk.java.net/~bpb/8160220/webrev.01/ >>> >>> >> Thinking about, this isn't needed. Since you've moved to the work >> directory then jtreg will do the clean-up (including retry). So the >> change should be to just move from /tmp to the work directory and you >> should be all set. > > So I should be able to remove FileUtils.deleteFileTreeWithRetry() and > the jdk.testlibrary references in *both* of these tests as well as the > original deleteFileTree() in DeleteInterference. > Yes, I think so as it will be cleaned up by the test harness. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jul 13 18:45:39 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 13 Jul 2016 11:45:39 -0700 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <3171dc29-444f-f0c0-dc0c-601c8d426a0a@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> <3171dc29-444f-f0c0-dc0c-601c8d426a0a@oracle.com> Message-ID: <424698EE-5FD5-4504-A798-681438F70114@oracle.com> On Jul 13, 2016, at 9:33 AM, Alan Bateman wrote: >>> Thinking about, this isn't needed. Since you've moved to the work directory then jtreg will do the clean-up (including retry). So the change should be to just move from /tmp to the work directory and you should be all set. >> >> So I should be able to remove FileUtils.deleteFileTreeWithRetry() and the jdk.testlibrary references in *both* of these tests as well as the original deleteFileTree() in DeleteInterference. >> > Yes, I think so as it will be cleaned up by the test harness. One more revision: http://cr.openjdk.java.net/~bpb/8160220/webrev.02/ Changes with respect to current repository (not with respect to prior revisions): 1. Temporary files are created in a subdirectory of ?test.dir? with cleanup left to the test harness. 2. Explicit deletion of temporary files is remove from main() in DeleteInterference. 3. deleteFileTree() is removed from DeleteInterference and its use in deleteAndRecreateDirectory() is replaced with FileUtils.deleteFileTreeWithRetry(). If this looks reasonable I will run one more regression job before pushing. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.lloyd at redhat.com Wed Jul 13 19:27:18 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 13 Jul 2016 14:27:18 -0500 Subject: Help needed: Java Socket and close detection In-Reply-To: <3c8a676e2c4f4760a7aaf07cc6c771f7@DEWDFE13DE11.global.corp.sap> References: <3c8a676e2c4f4760a7aaf07cc6c771f7@DEWDFE13DE11.global.corp.sap> Message-ID: <57869616.6080202@redhat.com> I don't think this really makes sense as a place to ask these questions, but I can probably answer them to an extent at least. When a TCP connection is closed normally, the side (we'll call it A) which initiates the close (usually by shutting down the write half of the socket) sends a FIN and enters into the FIN_WAIT_1 state, and the other side (which we'll call B) receives the FIN, entering into the CLOSE_WAIT state. When B receives the FIN, it does two things: it (invisibly) sends an ACK to A, and it reports an end-of-file condition on the socket (resulting in -1 being read in Java, assuming any other unread data is read). When A receives that ACK, it usually transitions from FIN_WAIT_1 to FIN_WAIT_2 (we'll see an exception to that next). Typically once a -1 is read by B, B will shut down the read half and will finish up writing any last bits of data on the socket, and then initiate its own close, shutting down the rest of the socket, causing B to transition from CLOSE_WAIT to LAST_ACK. This causes B to send a FIN back to A, who then receives it and transitions to TIME_WAIT. Note that if all this happens quickly enough, A might get a FIN+ACK before it gets the ACK, causing a direct transition from FIN_WAIT_1 to TIME_WAIT. If A gets the FIN first, it will transition from FIN_WAIT_1 to CLOSING, where it will remain until the final ACK comes, at which point it will move to TIME_WAIT. Any transition to CLOSING or TIME_WAIT will cause the "last ACK" to be sent, which upon receive will cause B to move to the CLOSED state, and allow B to free up the resources associated with the socket. A will remain in TIME_WAIT until a linger timeout expires, at which point the socket will be CLOSED and its resources released. Note that nowhere in this process does an RST come into play. Now when you close a socket "abruptly", for example by closing the read side while the peer is still sending an RST could be generated. This is usually considered a "soft" error condition by the way; you normally shouldn't shut things down that way, but it's probably tolerable most of the time. The exception message in this case is generally "Connection reset by peer", rather than receiving a graceful -1 as you would in the case of receiving a FIN. There are other ways to force RST to be sent, but most applications don't and shouldn't do it. The tricky bit comes in when you have a connection which is idle on both sides. If both sides are then "closed", that is, both read and write halves are shut down at the same time, everything can still happen tidily: a FIN is sent, a FIN is received, and we begin the walk towards the CLOSED state. The problem is that if one side doing a whole-channel close is not aware that the other side is still sending, an RST will result. So to answer your questions in order then: first, an RST can be generated by the network stack for a variety of reasons, but I would not expect well-behaved Java applications to do it normally, unless it's a case of application termination or something along those lines. Second, yes it is normal behavior for RST to not be sent. And third, the JVM does detect FIN packets by reporting an EOF to the read side of the socket (after the read buffer was fully drained); this represents normal application behavior. PS, if you Google for "TCP state diagram", you will find several ways to visualize what I've described above. On 07/13/2016 10:29 AM, Langer, Christoph wrote: > Hi folks, > > I have a question to the experts - regarding an issue that was reported > to me by a customer. > > In the customer scenario they are running a Servlet engine and the > Servlet is constantly sending data to a browser client. When the browser > client is closed, the server does not get a notification of the other > end having been terminated and is constantly sending out data and > blocking an application thread. I?m under the assumption that the server > should get an RST packet from the network upon writing/flushing data to > the OutputStream as soon as the client is gone and hence an Exception > should pop up but this isn?t happening. > > There is a load balancer and maybe other network infrastructure involved > in between the Servlet JVM and the browser client. We did some TCPDUMP > tracing at the load balancer and we could not see an RST packet coming > in from the client side. But when I?m running the scenario without all > this network infrastructure involved, e.g. between servers and clients > in the same network, I would always observe an RST packet once I close > the browser. A FIN packet is received, too, but this does not lead to an > Exception and to all my knowledge this can?t be detected, not from the > java Socket API and even less from the Servlet API which is just dealing > with Streams. > > So my question to the experts is most of all: Would you agree that an > RST packet should be generated in the network and received by the > server? Or is it a normal behavior that servers must deal with not > receiving RSTs and hence needing to wait for a timeout until e.g. the > load balancer generates an RST? Also, is there any way to detect a FIN > in the JVM and react on it? > > Thanks in advance and best regards > > Christoph > -- - DML From chris.hegarty at oracle.com Thu Jul 14 10:30:32 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 14 Jul 2016 11:30:32 +0100 Subject: Help needed: Java Socket and close detection In-Reply-To: <330d3165-8827-ea82-2c55-4964a59cbaed@oracle.com> References: <3c8a676e2c4f4760a7aaf07cc6c771f7@DEWDFE13DE11.global.corp.sap> <330d3165-8827-ea82-2c55-4964a59cbaed@oracle.com> Message-ID: Some further reading? https://docs.oracle.com/javase/8/docs/technotes/guides/net/articles/connection_release.html -Chris. > On 13 Jul 2016, at 21:37, Dmitry Samersoff wrote: > > Christoph, > > My $0.2 > > Typically you see RST packet when the data come to a *closed* socket. > You shouldn't get RST if client/server communication shutdown properly. > > Also balancer may take a care about connection shutdown (it needs to > update internal tables) so it's possible that you never get RST after > balancer. > > Moreover, there are RST based attack and scanning methods so firewall > might be set to reject RST packets and doesn't pass it to inside network. > > I.e. IMHO, customer server should be updated to live without RST. > > -Dmitry > > On 2016-07-13 18:29, Langer, Christoph wrote: >> Hi folks, >> >> >> >> I have a question to the experts - regarding an issue that was reported >> to me by a customer. >> >> >> >> In the customer scenario they are running a Servlet engine and the >> Servlet is constantly sending data to a browser client. When the browser >> client is closed, the server does not get a notification of the other >> end having been terminated and is constantly sending out data and >> blocking an application thread. I?m under the assumption that the server >> should get an RST packet from the network upon writing/flushing data to >> the OutputStream as soon as the client is gone and hence an Exception >> should pop up but this isn?t happening. >> >> >> >> There is a load balancer and maybe other network infrastructure involved >> in between the Servlet JVM and the browser client. We did some TCPDUMP >> tracing at the load balancer and we could not see an RST packet coming >> in from the client side. But when I?m running the scenario without all >> this network infrastructure involved, e.g. between servers and clients >> in the same network, I would always observe an RST packet once I close >> the browser. A FIN packet is received, too, but this does not lead to an >> Exception and to all my knowledge this can?t be detected, not from the >> java Socket API and even less from the Servlet API which is just dealing >> with Streams. >> >> >> >> So my question to the experts is most of all: Would you agree that an >> RST packet should be generated in the network and received by the >> server? Or is it a normal behavior that servers must deal with not >> receiving RSTs and hence needing to wait for a timeout until e.g. the >> load balancer generates an RST? Also, is there any way to detect a FIN >> in the JVM and react on it? >> >> >> >> Thanks in advance and best regards >> >> Christoph >> >> >> > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From christoph.langer at sap.com Fri Jul 15 11:34:52 2016 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 15 Jul 2016 11:34:52 +0000 Subject: Help needed: Java Socket and close detection In-Reply-To: References: <3c8a676e2c4f4760a7aaf07cc6c771f7@DEWDFE13DE11.global.corp.sap> <330d3165-8827-ea82-2c55-4964a59cbaed@oracle.com> Message-ID: <0aff055dcc024e70bc7e30be95f4c5b4@DEWDFE13DE11.global.corp.sap> Chris, David, Dmitry thanks for your valuable input. This helps me a lot. Best regards Christoph > -----Original Message----- > From: Chris Hegarty [mailto:chris.hegarty at oracle.com] > Sent: Donnerstag, 14. Juli 2016 12:31 > To: Langer, Christoph > Cc: net-dev at openjdk.java.net; nio-dev at openjdk.java.net > Subject: Re: Help needed: Java Socket and close detection > > Some further reading... > > https://docs.oracle.com/javase/8/docs/technotes/guides/net/articles/connectio > n_release.html > > -Chris. > > > On 13 Jul 2016, at 21:37, Dmitry Samersoff > wrote: > > > > Christoph, > > > > My $0.2 > > > > Typically you see RST packet when the data come to a *closed* socket. > > You shouldn't get RST if client/server communication shutdown properly. > > > > Also balancer may take a care about connection shutdown (it needs to > > update internal tables) so it's possible that you never get RST after > > balancer. > > > > Moreover, there are RST based attack and scanning methods so firewall > > might be set to reject RST packets and doesn't pass it to inside network. > > > > I.e. IMHO, customer server should be updated to live without RST. > > > > -Dmitry > > > > On 2016-07-13 18:29, Langer, Christoph wrote: > >> Hi folks, > >> > >> > >> > >> I have a question to the experts - regarding an issue that was reported > >> to me by a customer. > >> > >> > >> > >> In the customer scenario they are running a Servlet engine and the > >> Servlet is constantly sending data to a browser client. When the browser > >> client is closed, the server does not get a notification of the other > >> end having been terminated and is constantly sending out data and > >> blocking an application thread. I'm under the assumption that the server > >> should get an RST packet from the network upon writing/flushing data to > >> the OutputStream as soon as the client is gone and hence an Exception > >> should pop up but this isn't happening. > >> > >> > >> > >> There is a load balancer and maybe other network infrastructure involved > >> in between the Servlet JVM and the browser client. We did some TCPDUMP > >> tracing at the load balancer and we could not see an RST packet coming > >> in from the client side. But when I'm running the scenario without all > >> this network infrastructure involved, e.g. between servers and clients > >> in the same network, I would always observe an RST packet once I close > >> the browser. A FIN packet is received, too, but this does not lead to an > >> Exception and to all my knowledge this can't be detected, not from the > >> java Socket API and even less from the Servlet API which is just dealing > >> with Streams. > >> > >> > >> > >> So my question to the experts is most of all: Would you agree that an > >> RST packet should be generated in the network and received by the > >> server? Or is it a normal behavior that servers must deal with not > >> receiving RSTs and hence needing to wait for a timeout until e.g. the > >> load balancer generates an RST? Also, is there any way to detect a FIN > >> in the JVM and react on it? > >> > >> > >> > >> Thanks in advance and best regards > >> > >> Christoph > >> > >> > >> > > > > > > -- > > Dmitry Samersoff > > Oracle Java development team, Saint Petersburg, Russia > > * I would love to change the world, but they won't give me the sources. From brian.burkhalter at oracle.com Fri Jul 15 14:35:53 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 15 Jul 2016 07:35:53 -0700 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <424698EE-5FD5-4504-A798-681438F70114@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> <3171dc29-444f-f0c0-dc0c-601c8d426a0a@oracle.com> <424698EE-5FD5-4504-A798-681438F70114@oracle.com> Message-ID: <15FF7718-134D-4A8A-BAB7-AD89A397836A@oracle.com> Ping. On Jul 13, 2016, at 11:45 AM, Brian Burkhalter wrote: > On Jul 13, 2016, at 9:33 AM, Alan Bateman wrote: > >>>> Thinking about, this isn't needed. Since you've moved to the work directory then jtreg will do the clean-up (including retry). So the change should be to just move from /tmp to the work directory and you should be all set. >>> >>> So I should be able to remove FileUtils.deleteFileTreeWithRetry() and the jdk.testlibrary references in *both* of these tests as well as the original deleteFileTree() in DeleteInterference. >>> >> Yes, I think so as it will be cleaned up by the test harness. > > One more revision: > > http://cr.openjdk.java.net/~bpb/8160220/webrev.02/ > > Changes with respect to current repository (not with respect to prior revisions): > > 1. Temporary files are created in a subdirectory of ?test.dir? with cleanup left to the test harness. > 2. Explicit deletion of temporary files is remove from main() in DeleteInterference. > 3. deleteFileTree() is removed from DeleteInterference and its use in deleteAndRecreateDirectory() is replaced with FileUtils.deleteFileTreeWithRetry(). > > If this looks reasonable I will run one more regression job before pushing. > > Thanks, > > Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Jul 15 14:41:28 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 15 Jul 2016 15:41:28 +0100 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <15FF7718-134D-4A8A-BAB7-AD89A397836A@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> <3171dc29-444f-f0c0-dc0c-601c8d426a0a@oracle.com> <424698EE-5FD5-4504-A798-681438F70114@oracle.com> <15FF7718-134D-4A8A-BAB7-AD89A397836A@oracle.com> Message-ID: On 15/07/2016 15:35, Brian Burkhalter wrote: > Ping. The changes to main are fine. I don't think there is any need to use FileUtils.deleteFileTreeWithRetry here. Remember, if the test leaves anything behind then jtreg will cleanup. Also using FileUtils makes it harder to run standalone. -Alan From dmitry.samersoff at oracle.com Wed Jul 13 20:37:34 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 13 Jul 2016 23:37:34 +0300 Subject: Help needed: Java Socket and close detection In-Reply-To: <3c8a676e2c4f4760a7aaf07cc6c771f7@DEWDFE13DE11.global.corp.sap> References: <3c8a676e2c4f4760a7aaf07cc6c771f7@DEWDFE13DE11.global.corp.sap> Message-ID: <330d3165-8827-ea82-2c55-4964a59cbaed@oracle.com> Christoph, My $0.2 Typically you see RST packet when the data come to a *closed* socket. You shouldn't get RST if client/server communication shutdown properly. Also balancer may take a care about connection shutdown (it needs to update internal tables) so it's possible that you never get RST after balancer. Moreover, there are RST based attack and scanning methods so firewall might be set to reject RST packets and doesn't pass it to inside network. I.e. IMHO, customer server should be updated to live without RST. -Dmitry On 2016-07-13 18:29, Langer, Christoph wrote: > Hi folks, > > > > I have a question to the experts - regarding an issue that was reported > to me by a customer. > > > > In the customer scenario they are running a Servlet engine and the > Servlet is constantly sending data to a browser client. When the browser > client is closed, the server does not get a notification of the other > end having been terminated and is constantly sending out data and > blocking an application thread. I?m under the assumption that the server > should get an RST packet from the network upon writing/flushing data to > the OutputStream as soon as the client is gone and hence an Exception > should pop up but this isn?t happening. > > > > There is a load balancer and maybe other network infrastructure involved > in between the Servlet JVM and the browser client. We did some TCPDUMP > tracing at the load balancer and we could not see an RST packet coming > in from the client side. But when I?m running the scenario without all > this network infrastructure involved, e.g. between servers and clients > in the same network, I would always observe an RST packet once I close > the browser. A FIN packet is received, too, but this does not lead to an > Exception and to all my knowledge this can?t be detected, not from the > java Socket API and even less from the Servlet API which is just dealing > with Streams. > > > > So my question to the experts is most of all: Would you agree that an > RST packet should be generated in the network and received by the > server? Or is it a normal behavior that servers must deal with not > receiving RSTs and hence needing to wait for a timeout until e.g. the > load balancer generates an RST? Also, is there any way to detect a FIN > in the JVM and react on it? > > > > Thanks in advance and best regards > > Christoph > > > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From brian.burkhalter at oracle.com Fri Jul 15 16:42:04 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 15 Jul 2016 09:42:04 -0700 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> <3171dc29-444f-f0c0-dc0c-601c8d426a0a@oracle.com> <424698EE-5FD5-4504-A798-681438F70114@oracle.com> <15FF7718-134D-4A8A-BAB7-AD89A397836A@oracle.com> Message-ID: <5E455339-57E2-43FC-9E59-7A0592D1FD82@oracle.com> On Jul 15, 2016, at 7:41 AM, Alan Bateman wrote: > On 15/07/2016 15:35, Brian Burkhalter wrote: >> Ping. > The changes to main are fine. I don't think there is any need to use FileUtils.deleteFileTreeWithRetry here. Remember, if the test leaves anything behind then jtreg will cleanup. Also using FileUtils makes it harder to run standalone. OK so I will reinstate the deleteFileTree() method and remove the FileUtils references. Will that do it? Thanks, Brian From brian.burkhalter at oracle.com Fri Jul 15 17:01:11 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 15 Jul 2016 10:01:11 -0700 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <5E455339-57E2-43FC-9E59-7A0592D1FD82@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> <3171dc29-444f-f0c0-dc0c-601c8d426a0a@oracle.com> <424698EE-5FD5-4504-A798-681438F70114@oracle.com> <15FF7718-134D-4A8A-BAB7-AD89A397836A@oracle.com> <5E455339-57E2-43FC-9E59-7A0592D1FD82@oracle.com> Message-ID: <49404AA2-F31F-4CB6-A8B7-47DE19B30888@oracle.com> On Jul 15, 2016, at 9:42 AM, Brian Burkhalter wrote: > On Jul 15, 2016, at 7:41 AM, Alan Bateman wrote: > >> On 15/07/2016 15:35, Brian Burkhalter wrote: >>> Ping. >> The changes to main are fine. I don't think there is any need to use FileUtils.deleteFileTreeWithRetry here. Remember, if the test leaves anything behind then jtreg will cleanup. Also using FileUtils makes it harder to run standalone. > > OK so I will reinstate the deleteFileTree() method and remove the FileUtils references. Will that do it? Here is one more revision with the foregoing change: http://cr.openjdk.java.net/~bpb/8160220/webrev.03/ Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Jul 15 19:29:05 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 15 Jul 2016 12:29:05 -0700 Subject: [doc-only] JDK 9 RFR of 8161455: Missing word in API documentation Message-ID: Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8161455 Patch: [1] Thanks, Brian [1] diff --- a/src/java.base/share/classes/java/nio/file/Files.java +++ b/src/java.base/share/classes/java/nio/file/Files.java @@ -3290,8 +3290,8 @@ * a size of {@code 0}. All bytes in the byte array are written to the file. * The method ensures that the file is closed when all bytes have been * written (or an I/O error or other runtime exception is thrown). If an I/O - * error occurs then it may do so after the file has created or truncated, - * or after some bytes have been written to the file. + * error occurs then it may do so after the file has been created or + * truncated, or after some bytes have been written to the file. * *

Usage example: By default the method creates a new file or * overwrites an existing file. Suppose you instead want to append bytes @@ -3360,7 +3360,8 @@ * a size of {@code 0}. The method ensures that the file is closed when all * lines have been written (or an I/O error or other runtime exception is * thrown). If an I/O error occurs then it may do so after the file has - * created or truncated, or after some bytes have been written to the file. + * been created or truncated, or after some bytes have been written to the + * file. * * @param path * the path to the file -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Jul 15 19:32:56 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 15 Jul 2016 20:32:56 +0100 Subject: [doc-only] JDK 9 RFR of 8161455: Missing word in API documentation In-Reply-To: References: Message-ID: <9d95fd70-63c4-a731-ca66-f73bd56a3153@oracle.com> On 15/07/2016 20:29, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8161455 > Patch: [1] > > This looks fine. -Alan From Alan.Bateman at oracle.com Fri Jul 15 19:33:28 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 15 Jul 2016 20:33:28 +0100 Subject: JDK 9 RFR of 8160220: (fs) Tests in jdk/test/java/nio/file/WatchService leave directory trees behind In-Reply-To: <49404AA2-F31F-4CB6-A8B7-47DE19B30888@oracle.com> References: <41ADC9BE-1946-4F96-BD37-2F0F5FB10536@oracle.com> <32b6ec04-d3d8-d4f3-c001-a5fce6b8efde@oracle.com> <915857ec-16cb-90bb-6e19-0bc90017d21c@oracle.com> <16175168-CED4-43F5-844A-D8FE826319DD@oracle.com> <0AE41ECB-942D-42CB-A1EC-54B229F55A34@oracle.com> <3171dc29-444f-f0c0-dc0c-601c8d426a0a@oracle.com> <424698EE-5FD5-4504-A798-681438F70114@oracle.com> <15FF7718-134D-4A8A-BAB7-AD89A397836A@oracle.com> <5E455339-57E2-43FC-9E59-7A0592D1FD82@oracle.com> <49404AA2-F31F-4CB6-A8B7-47DE19B30888@oracle.com> Message-ID: On 15/07/2016 18:01, Brian Burkhalter wrote: > > Here is one more revision with the foregoing change: > > http://cr.openjdk.java.net/~bpb/8160220/webrev.03/ > > Looks good. From rcmuir at gmail.com Mon Jul 25 22:03:16 2016 From: rcmuir at gmail.com (Robert Muir) Date: Mon, 25 Jul 2016 18:03:16 -0400 Subject: FileStore overflow for 8 exabytes filesystem Message-ID: Hello, We had a user report this interesting one with amazon EFS: https://discuss.elastic.co/t/elasticsearch-with-amazon-elastic-file-system/55867/8 FileStore.java overflows here, all the methods look like this: @Override public long getUsableSpace() throws IOException { UnixFileStoreAttributes attrs = readAttributes(); return attrs.blockSize() * attrs.availableBlocks(); } the original blockSize and various block counts are unsigned 64 bit values, but they are treated as signed longs, then multiplied, and then it causes negative numbers reported back via FileStore. What can be done? It seems FileStore API is hopelessly broken since it returns long, and returns bytes. treating it as unsigned long wouldn't be that much better... Should I open a bug? From fgaliegue at gmail.com Mon Jul 25 23:06:31 2016 From: fgaliegue at gmail.com (Francis Galiegue) Date: Tue, 26 Jul 2016 01:06:31 +0200 Subject: FileStore overflow for 8 exabytes filesystem In-Reply-To: References: Message-ID: In my view, and given the documentation, this is clearly a bug. While the documentation doesn't mention any specific behavior, I have developed filesystems of my own; and AFAIR, the recommendation when the underlying FileStore was unable to return a usable value was to return Long.MAX_VALUE. Here, you seem to be using a mount on a Unix filesystem hierarchy, which only makes things worse. For any metadata, you have two indirection: from EFS to someUnix and from someUnix to the Java API. That does not help matters. Ideally, you want a direct JSR 203 bind to EFS. This will likely solve all your problems in one go. On Tue, Jul 26, 2016 at 12:03 AM, Robert Muir wrote: > Hello, > > We had a user report this interesting one with amazon EFS: > > https://discuss.elastic.co/t/elasticsearch-with-amazon-elastic-file-system/55867/8 > > FileStore.java overflows here, all the methods look like this: > > @Override > public long getUsableSpace() throws IOException { > UnixFileStoreAttributes attrs = readAttributes(); > return attrs.blockSize() * attrs.availableBlocks(); > } > > the original blockSize and various block counts are unsigned 64 bit > values, but they are treated as signed longs, then multiplied, and > then it causes negative numbers reported back via FileStore. > > What can be done? It seems FileStore API is hopelessly broken since it > returns long, and returns bytes. treating it as unsigned long wouldn't > be that much better... > > Should I open a bug? -- Francis Galiegue, fgaliegue at gmail.com, https://github.com/fge JSON Schema in Java: http://json-schema-validator.herokuapp.com Parsers in pure Java: https://github.com/fge/grappa From Alan.Bateman at oracle.com Tue Jul 26 06:56:01 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 26 Jul 2016 07:56:01 +0100 Subject: FileStore overflow for 8 exabytes filesystem In-Reply-To: References: Message-ID: On 25/07/2016 23:03, Robert Muir wrote: > : > > Should I open a bug? Yes please as it will require new methods to space larger than > Long.MAX_VALUE, assuming it's reported in bytes. -Alan From rcmuir at gmail.com Tue Jul 26 07:48:01 2016 From: rcmuir at gmail.com (Robert Muir) Date: Tue, 26 Jul 2016 03:48:01 -0400 Subject: FileStore overflow for 8 exabytes filesystem In-Reply-To: References: Message-ID: Thanks, I opened this bug: JI-9042198 On Tue, Jul 26, 2016 at 2:56 AM, Alan Bateman wrote: > On 25/07/2016 23:03, Robert Muir wrote: > >> : >> >> Should I open a bug? > > Yes please as it will require new methods to space larger than > > Long.MAX_VALUE, assuming it's reported in bytes. > > -Alan From brian.burkhalter at oracle.com Wed Jul 27 16:36:07 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 27 Jul 2016 09:36:07 -0700 Subject: JDK 9 RFR of 8162624: (fs) Remove FileTypeDetectors based on libgio and libmagic Message-ID: <87E2568C-37F2-4F1D-AA99-969B96DB978A@oracle.com> Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8162624 Patch: http://cr.openjdk.java.net/~bpb/8162624/webrev.00/ Remove GioFileTypeDetector and MagicFileTypeDetector as these seem not to provide significant value, introduce unpredictability into the result of file probing, and are inconsistent with the other principle server side platform Solaris. Thanks, Brian From Alan.Bateman at oracle.com Wed Jul 27 16:47:10 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 27 Jul 2016 17:47:10 +0100 Subject: JDK 9 RFR of 8162624: (fs) Remove FileTypeDetectors based on libgio and libmagic In-Reply-To: <87E2568C-37F2-4F1D-AA99-969B96DB978A@oracle.com> References: <87E2568C-37F2-4F1D-AA99-969B96DB978A@oracle.com> Message-ID: <4c0fb50e-52ce-0cb0-1b46-53e0b1fe16ed@oracle.com> On 27/07/2016 17:36, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8162624 > Patch: http://cr.openjdk.java.net/~bpb/8162624/webrev.00/ > > Remove GioFileTypeDetector and MagicFileTypeDetector as these seem not to provide significant value, introduce unpredictability into the result of file probing, and are inconsistent with the other principle server side platform Solaris. > > This looks okay to me, particular the GIO based one as that has been very problematic. -Alan From paul.sandoz at oracle.com Thu Jul 28 10:33:19 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 28 Jul 2016 12:33:19 +0200 Subject: RFR 8162458 Buffer view implementations use incorrect offset for Unsafe access In-Reply-To: <2F815E1E-1DA6-4010-9B39-5D7BBB2DF069@oracle.com> References: <2F815E1E-1DA6-4010-9B39-5D7BBB2DF069@oracle.com> Message-ID: <90A0B797-2324-42B5-BAF9-E28AAB48B3E5@oracle.com> Hooking in nio dev. I think this issue is important to review/push soon (i.e. this week) so as it does not leak out beyond the hs repo. ? Incidentally this patch also fixes what might have been a long standing bug in the compact method of buffer views. Here is ByteBufferAsIntBufferB?s compact method: public IntBuffer compact() { int pos = position(); int lim = limit(); assert (pos <= lim); int rem = (pos <= lim ? lim - pos : 0); ByteBuffer db = bb.duplicate(); db.limit(ix(lim)); db.position(ix(0)); ByteBuffer sb = db.slice(); sb.position(pos << 2); sb.compact(); position(rem); limit(capacity()); discardMark(); return this; } For view heap buffers the ix method used to return an offset relative to the underlying byte array, not relative to the buffer. this this can result values that are beyond the capacity of the buffer. The following code: ByteBuffer bb0 = ByteBuffer.allocate(16); ByteBuffer bb4 = bb0.position(4).slice(); ByteBuffer bb8 = bb4.position(4).slice(); IntBuffer ib = bb8.asIntBuffer().compact(); results in: Exception in thread "main" java.lang.IllegalArgumentException: newLimit > capacity: (16 > 8) Paul. > On 27 Jul 2016, at 13:47, Paul Sandoz wrote: > > Hi, > > I made an embarrassing mistake in the fix for > > https://bugs.openjdk.java.net/browse/JDK-8151163 > All Buffer implementations should leverage Unsafe unaligned accessors > > The offset calculation for Unsafe access was incorrect, it?s easy to get confused because for heap buffers the offset is relative to the array, and for direct buffers the address (which can update for slices/duplicates). Disturbingly all existing tests were passing both for core and hotspot when i pushed to hs. > > As a penance i wrote a combinatorial test for buffer views to navigate the twisty maze of heap/direct, aligned/unaligned, little/big endian for accessing binary data and views from the source buffer. > > Please review: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8162458-byte-buffer-view-offset-access-incorrect/webrev/ > > (This may be a duplicate of [1]). > > Test has been verified to fail with the existing code. Focused JPRT runs pass, but i will kick off core/hotspot runs later on today. > > I will push to hs since that is where JDK-8151163 and it has not been integrated into jdk9/dev. > > Paul. > > [1] https://bugs.openjdk.java.net/browse/JDK-8159257 > unsafe.cpp: assert(byte_offset < p_size) failed: Unsafe access: offset 32767 > object's size 16 > > For the test runtime/Unsafe/RangeCheck.java I can reproduce a crash in jdk9/dev which does not have JDK-8151163, and i can reproduce on jdk9/hs with this fix for JDK-8162458. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail URL: From daniel.daugherty at oracle.com Thu Jul 28 14:29:47 2016 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 28 Jul 2016 08:29:47 -0600 Subject: RFR 8162458 Buffer view implementations use incorrect offset for Unsafe access In-Reply-To: <90A0B797-2324-42B5-BAF9-E28AAB48B3E5@oracle.com> References: <2F815E1E-1DA6-4010-9B39-5D7BBB2DF069@oracle.com> <90A0B797-2324-42B5-BAF9-E28AAB48B3E5@oracle.com> Message-ID: <130ce829-b790-54b3-21ad-20da02623377@oracle.com> Paul, I have marked this bug (JDK-8162458) as an 'integration_blocker' in order to keep JDK-8151163 from leaving JDK9-hs in its current state. Dan On 7/28/16 4:33 AM, Paul Sandoz wrote: > Hooking in nio dev. > > I think this issue is important to review/push soon (i.e. this week) so as it does not leak out beyond the hs repo. > > ? > > Incidentally this patch also fixes what might have been a long standing bug in the compact method of buffer views. > > Here is ByteBufferAsIntBufferB?s compact method: > > public IntBuffer compact() { > > int pos = position(); > int lim = limit(); > assert (pos <= lim); > int rem = (pos <= lim ? lim - pos : 0); > > ByteBuffer db = bb.duplicate(); > db.limit(ix(lim)); > db.position(ix(0)); > ByteBuffer sb = db.slice(); > sb.position(pos << 2); > sb.compact(); > position(rem); > limit(capacity()); > discardMark(); > return this; > } > > > For view heap buffers the ix method used to return an offset relative to the underlying byte array, not relative to the buffer. this this can result values that are beyond the capacity of the buffer. > > The following code: > > ByteBuffer bb0 = ByteBuffer.allocate(16); > ByteBuffer bb4 = bb0.position(4).slice(); > ByteBuffer bb8 = bb4.position(4).slice(); > IntBuffer ib = bb8.asIntBuffer().compact(); > > results in: > > Exception in thread "main" java.lang.IllegalArgumentException: newLimit > capacity: (16 > 8) > > Paul. > > >> On 27 Jul 2016, at 13:47, Paul Sandoz wrote: >> >> Hi, >> >> I made an embarrassing mistake in the fix for >> >> https://bugs.openjdk.java.net/browse/JDK-8151163 >> All Buffer implementations should leverage Unsafe unaligned accessors >> >> The offset calculation for Unsafe access was incorrect, it?s easy to get confused because for heap buffers the offset is relative to the array, and for direct buffers the address (which can update for slices/duplicates). Disturbingly all existing tests were passing both for core and hotspot when i pushed to hs. >> >> As a penance i wrote a combinatorial test for buffer views to navigate the twisty maze of heap/direct, aligned/unaligned, little/big endian for accessing binary data and views from the source buffer. >> >> Please review: >> >> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8162458-byte-buffer-view-offset-access-incorrect/webrev/ >> >> (This may be a duplicate of [1]). >> >> Test has been verified to fail with the existing code. Focused JPRT runs pass, but i will kick off core/hotspot runs later on today. >> >> I will push to hs since that is where JDK-8151163 and it has not been integrated into jdk9/dev. >> >> Paul. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8159257 >> unsafe.cpp: assert(byte_offset < p_size) failed: Unsafe access: offset 32767 > object's size 16 >> >> For the test runtime/Unsafe/RangeCheck.java I can reproduce a crash in jdk9/dev which does not have JDK-8151163, and i can reproduce on jdk9/hs with this fix for JDK-8162458. > From brian.burkhalter at oracle.com Thu Jul 28 23:55:09 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 28 Jul 2016 16:55:09 -0700 Subject: JDK 9 RFR of 8146215: (fs) java/nio/file/Files/probeContentType/Basic.java failed frequently on Solaris-sparc with Unexpected type: text/plain Message-ID: <607BC614-DFBE-49B2-AA6D-5A01EFF21D56@oracle.com> Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8146215 Patch: http://cr.openjdk.java.net/~bpb/8146215/webrev.00/ On Solaris only, append to the FileTypeDetectors chain a FileTypeDetector which uses the java.net.FileNameMap returned by URLConnection.getFileNameMap() to probe for the MIME type. Also clean up probeContentType/Basic a little. Note that this FileNameMap-based detector is not really platform-specific, so if desired it could as well be used on other platforms perhaps as a fallback detector, i.e., the last one in each of the respective chains. On Solaris there is also the possibility of using a MimeTypesFileTypeDetector with the Apache MIME types file /etc/apache2/2.2/mime.types which is available by default on Solaris 11 [1]. There does not however appear to be a straightforward programmatic way of locating this file, so if for example the version number were to change to 2.3, the file would not be found if the name were hard-coded. This suggests that if it were desirable to use the Apache MIME types file, some provision for dynamically locating it in the file system would be needed. Thanks, Brian [1] http://docs.oracle.com/cd/E36784_01/html/E36871/apache2-1m.html From Alan.Bateman at oracle.com Fri Jul 29 16:57:08 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 29 Jul 2016 17:57:08 +0100 Subject: JDK 9 RFR of 8146215: (fs) java/nio/file/Files/probeContentType/Basic.java failed frequently on Solaris-sparc with Unexpected type: text/plain In-Reply-To: <607BC614-DFBE-49B2-AA6D-5A01EFF21D56@oracle.com> References: <607BC614-DFBE-49B2-AA6D-5A01EFF21D56@oracle.com> Message-ID: On 29/07/2016 00:55, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8146215 > Patch: http://cr.openjdk.java.net/~bpb/8146215/webrev.00/ > > On Solaris only, append to the FileTypeDetectors chain a FileTypeDetector which uses the java.net.FileNameMap returned by URLConnection.getFileNameMap() to probe for the MIME type. Also clean up probeContentType/Basic a little. > > Note that this FileNameMap-based detector is not really platform-specific, so if desired it could as well be used on other platforms perhaps as a fallback detector, i.e., the last one in each of the respective chains. A concern with this increases the potential behavior across platforms again. If do we use this one then I would think it has to be Linux too. On the implementation then fileNameMap should be volatile. You can introduce a local `map` when reading it. You should also make fileNameMap static as you only need to obtain the FileNameMap once, > On Solaris there is also the possibility of using a MimeTypesFileTypeDetector with the Apache MIME types file /etc/apache2/2.2/mime.types which is available by default on Solaris 11 [1]. There does not however appear to be a straightforward programmatic way of locating this file, so if for example the version number were to change to 2.3, the file would not be found if the name were hard-coded. This suggests that if it were desirable to use the Apache MIME types file, some provision for dynamically locating it in the file system would be needed. > I don't know if this Apache bundling is documented or not but I think would just make things very unpredictable. -Alan From brian.burkhalter at oracle.com Fri Jul 29 17:46:12 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 29 Jul 2016 10:46:12 -0700 Subject: JDK 9 RFR of 8146215: (fs) java/nio/file/Files/probeContentType/Basic.java failed frequently on Solaris-sparc with Unexpected type: text/plain In-Reply-To: References: <607BC614-DFBE-49B2-AA6D-5A01EFF21D56@oracle.com> Message-ID: <1C814958-7FA4-4F7A-BA69-E698EBE7AAD2@oracle.com> On Jul 29, 2016, at 9:57 AM, Alan Bateman wrote: > On 29/07/2016 00:55, Brian Burkhalter wrote: >> Please review at your convenience. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8146215 >> Patch: http://cr.openjdk.java.net/~bpb/8146215/webrev.00/ >> >> On Solaris only, append to the FileTypeDetectors chain a FileTypeDetector which uses the java.net.FileNameMap returned by URLConnection.getFileNameMap() to probe for the MIME type. Also clean up probeContentType/Basic a little. >> >> Note that this FileNameMap-based detector is not really platform-specific, so if desired it could as well be used on other platforms perhaps as a fallback detector, i.e., the last one in each of the respective chains. > A concern with this increases the potential [inconsistent] behavior across platforms again. It depends where it is placed in the hierarchy. If it were second, after ~/.mime.types, then it could increase predictability across platforms. > If do we use this one then I would think it has to be Linux too. Agreed. > On the implementation then fileNameMap should be volatile. You can introduce a local `map` when reading it. You should also make fileNameMap static as you only need to obtain the FileNameMap once, OK. >> On Solaris there is also the possibility of using a MimeTypesFileTypeDetector with the Apache MIME types file /etc/apache2/2.2/mime.types which is available by default on Solaris 11 [1]. There does not however appear to be a straightforward programmatic way of locating this file, so if for example the version number were to change to 2.3, the file would not be found if the name were hard-coded. This suggests that if it were desirable to use the Apache MIME types file, some provision for dynamically locating it in the file system would be needed. >> > I don't know if this Apache bundling is documented or not but I think would just make things very unpredictable. It looks like it from [1] and it was also present in the base installation on our Solaris test systems. Another across-the-board solution which would make things more predictable would be to bundle the Apache MIME types file, which is in the public domain [2], put it in ?java.home?/mime.types, and add a MimeTypesFileTypeDetector using it to all chains. Thanks, Brian [1] http://docs.oracle.com/cd/E36784_01/html/E36871/apache2-1m.html [2] http://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/conf/mime.types -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Jul 29 21:13:42 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 29 Jul 2016 22:13:42 +0100 Subject: JDK 9 RFR of 8146215: (fs) java/nio/file/Files/probeContentType/Basic.java failed frequently on Solaris-sparc with Unexpected type: text/plain In-Reply-To: <1C814958-7FA4-4F7A-BA69-E698EBE7AAD2@oracle.com> References: <607BC614-DFBE-49B2-AA6D-5A01EFF21D56@oracle.com> <1C814958-7FA4-4F7A-BA69-E698EBE7AAD2@oracle.com> Message-ID: On 29/07/2016 18:46, Brian Burkhalter wrote: > > : > > It depends where it is placed in the hierarchy. If it were second, > after ~/.mime.types, then it could increase predictability across > platforms. Yes, having ~/.mine.types first is useful as it allows the mapping to be easily created or overridden if needed. -Alan From brian.burkhalter at oracle.com Fri Jul 29 21:51:13 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 29 Jul 2016 14:51:13 -0700 Subject: JDK 9 RFR of 8146215: (fs) java/nio/file/Files/probeContentType/Basic.java failed frequently on Solaris-sparc with Unexpected type: text/plain In-Reply-To: References: <607BC614-DFBE-49B2-AA6D-5A01EFF21D56@oracle.com> <1C814958-7FA4-4F7A-BA69-E698EBE7AAD2@oracle.com> Message-ID: <07BA34F1-F454-4D2C-91CE-7698A65B0902@oracle.com> On Jul 29, 2016, at 2:13 PM, Alan Bateman wrote: > On 29/07/2016 18:46, Brian Burkhalter wrote: >> >> It depends where it is placed in the hierarchy. If it were second, after ~/.mime.types, then it could increase predictability across platforms. > Yes, having ~/.mine.types first is useful as it allows the mapping to be easily created or overridden if needed. An updated version of the patch is here: http://cr.openjdk.java.net/~bpb/8146215/webrev.01/ Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Sat Jul 30 09:31:45 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 30 Jul 2016 10:31:45 +0100 Subject: JDK 9 RFR of 8146215: (fs) java/nio/file/Files/probeContentType/Basic.java failed frequently on Solaris-sparc with Unexpected type: text/plain In-Reply-To: <07BA34F1-F454-4D2C-91CE-7698A65B0902@oracle.com> References: <607BC614-DFBE-49B2-AA6D-5A01EFF21D56@oracle.com> <1C814958-7FA4-4F7A-BA69-E698EBE7AAD2@oracle.com> <07BA34F1-F454-4D2C-91CE-7698A65B0902@oracle.com> Message-ID: <96fab9bc-0afc-59a2-6da1-0a238bcd815a@oracle.com> On 29/07/2016 22:51, Brian Burkhalter wrote: > > An updated version of the patch is here: > > http://cr.openjdk.java.net/~bpb/8146215/webrev.01/ > > > Thanks, > > Brian The synchronization doesn't look right in FileNameMapFileTypeDetector. As fileNameMap is static then synchronized (this) { ... } doesn't do what you intend. It's not an issue if URLConnection.getFileNameMap() is invoked more than once so I think you can reduce this down to: FileNameMap map = fileNameMap; if (map == null) { fileNameMap = map = URLConnection.getFileNameMap(); } return map.getContentTypeFor(...); Alternatively just call URLConnection.getFileNameMap() each time. As regards ordering what what you have it okay. I assume it will be rare to set the property content.types.user.table and so the JDK's content-types.properties will be a fallback. -Alan