From brian.burkhalter at oracle.com Tue Oct 1 14:36:17 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 1 Oct 2019 07:36:17 -0700 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> Message-ID: <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> > On Sep 30, 2019, at 11:43 AM, Brian Burkhalter wrote: > >> On Sep 28, 2019, at 9:07 AM, Alan Bateman > wrote: >> >> On 27/09/2019 22:02, Brian Burkhalter wrote: >>> : >>> >>> 1. Return the number of blocks of each type >>> >>> long getTotalBlocks(); >>> long getUnallocatedBlocks(); >>> long getUsableBlocks(); >>> >>> The respective sizes in whichever units can then be computed by the caller using the value returned by getBlockSize(). >> This option does fit with the recently added getBlockSize, I just wonder if 2^63 blocks is enough. The native APIs seem to allow up to 2^64 (unsigned). > > I?m not sure what can be done aside from using an alternate representation instead of a long. The product of getTotalBlocks() and getBlockSize() would give a very large range of values. Here is a stab [1] at fixing [2] using the above approach. Thanks, Brian [1] http://cr.openjdk.java.net/~bpb/8162520/webrev.00/ [2] https://bugs.openjdk.java.net/browse/JDK-8162520 -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Oct 1 14:48:51 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 1 Oct 2019 15:48:51 +0100 Subject: 8231603: (se) Selector implementations do not need to use cancelledKeys Message-ID: This is a change to the Selector and SelectionKey::cancel implementations to avoid needing the cancelled-key set. Although specified, the cancelled-set key is not exposed in the API (except in implementations of AbstractSelector where the implementation has access to its own cancelled-key set) so the change should not be observable. The change has no impact on alternative implementations of AbstractSelector.? While in the area, I replaced the use of AtomicBoolean for the open state with a VarHandle.? I also inverted the valid flag in SelectionKeyImpl to that its initial value is false and so doesn't need to be initialized to true with a volatile write. http://cr.openjdk.java.net/~alanb/8231603/webrev/index.html -Alan From Alan.Bateman at oracle.com Tue Oct 1 14:59:40 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 1 Oct 2019 15:59:40 +0100 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> Message-ID: On 01/10/2019 15:36, Brian Burkhalter wrote: > Here is a stab [1] at fixing [2] using the above approach. > If we go with this approach then the new methods will need default implementations that throw UOE (and UOE will need to be specified too). This is because there are several FileSystemProvider implementations in the wild. It's good to get the jrt and zip providers updated of course. I see you've changed the spec for the existing methods to return a negative value when the size is great than Long.MAX_VALUE. The alternative is to return MAX_VALUE, I'm not sure which is better without studying some of the existing usages of the API. -Alan From brian.burkhalter at oracle.com Tue Oct 1 15:04:54 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 1 Oct 2019 08:04:54 -0700 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> Message-ID: <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> > On Oct 1, 2019, at 7:59 AM, Alan Bateman wrote: > > On 01/10/2019 15:36, Brian Burkhalter wrote: >> Here is a stab [1] at fixing [2] using the above approach. >> > If we go with this approach then the new methods will need default implementations that throw UOE (and UOE will need to be specified too). This is because there are several FileSystemProvider implementations in the wild. It's good to get the jrt and zip providers updated of course. I actually first wrote a version with default implementations instead of abstract methods but changed my mind. I?ll revert it and clean it up. > I see you've changed the spec for the existing methods to return a negative value when the size is great than Long.MAX_VALUE. The alternative is to return MAX_VALUE, I'm not sure which is better without studying some of the existing usages of the API. My thinking was that MAX_VALUE could be a legitimate size, albeit unlikely being an odd number. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Oct 1 16:07:22 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 1 Oct 2019 09:07:22 -0700 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> Message-ID: <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> > On Oct 1, 2019, at 8:04 AM, Brian Burkhalter wrote: > >> On Oct 1, 2019, at 7:59 AM, Alan Bateman > wrote: >> >> On 01/10/2019 15:36, Brian Burkhalter wrote: >>> Here is a stab [1] at fixing [2] using the above approach. >>> >> If we go with this approach then the new methods will need default implementations that throw UOE (and UOE will need to be specified too). This is because there are several FileSystemProvider implementations in the wild. It's good to get the jrt and zip providers updated of course. > > I actually first wrote a version with default implementations instead of abstract methods but changed my mind. I?ll revert it and clean it up. Here is an updated version with default instead of abstract methods: http://cr.openjdk.java.net/~bpb/8162520/webrev.01/ Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Tue Oct 1 16:19:29 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 1 Oct 2019 17:19:29 +0100 Subject: 8231603: (se) Selector implementations do not need to use cancelledKeys In-Reply-To: References: Message-ID: <7588ED88-1D15-424F-A985-965599FA2DF9@oracle.com> > On 1 Oct 2019, at 15:48, Alan Bateman wrote: > > This is a change to the Selector and SelectionKey::cancel implementations to avoid needing the cancelled-key set. Although specified, the cancelled-set key is not exposed in the API (except in implementations of AbstractSelector where the implementation has access to its own cancelled-key set) so the change should not be observable. The change has no impact on alternative implementations of AbstractSelector. While in the area, I replaced the use of AtomicBoolean for the open state with a VarHandle. I also inverted the valid flag in SelectionKeyImpl to that its initial value is false and so doesn't need to be initialized to true with a volatile write. > > http://cr.openjdk.java.net/~alanb/8231603/webrev/index.html Seems fine to me. ( I agree that the change should not really be observable ) I assume that the kind of access operations that the selector does, add keys & iterate over all keys, are more suited to a deque, and therefore faster ( than HashSet ). The updates from j.u.c.atomic to VarHandle look fine too. -Chris. From brian.burkhalter at oracle.com Tue Oct 1 16:50:59 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 1 Oct 2019 09:50:59 -0700 Subject: 8231603: (se) Selector implementations do not need to use cancelledKeys In-Reply-To: <7588ED88-1D15-424F-A985-965599FA2DF9@oracle.com> References: <7588ED88-1D15-424F-A985-965599FA2DF9@oracle.com> Message-ID: > On Oct 1, 2019, at 9:19 AM, Chris Hegarty wrote: > >> On 1 Oct 2019, at 15:48, Alan Bateman > wrote: >> >> This is a change to the Selector and SelectionKey::cancel implementations to avoid needing the cancelled-key set. Although specified, the cancelled-set key is not exposed in the API (except in implementations of AbstractSelector where the implementation has access to its own cancelled-key set) so the change should not be observable. The change has no impact on alternative implementations of AbstractSelector. While in the area, I replaced the use of AtomicBoolean for the open state with a VarHandle. I also inverted the valid flag in SelectionKeyImpl to that its initial value is false and so doesn't need to be initialized to true with a volatile write. >> >> http://cr.openjdk.java.net/~alanb/8231603/webrev/index.html > > Seems fine to me. ( I agree that the change should not really be observable ) > > I assume that the kind of access operations that the selector does, add keys & iterate over all keys, are more suited to a deque, and therefore faster ( than HashSet ). > > The updates from j.u.c.atomic to VarHandle look fine too. Agreed, this looks fine. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From patrick.concannon at oracle.com Wed Oct 2 09:49:18 2019 From: patrick.concannon at oracle.com (Patrick Concannon) Date: Wed, 2 Oct 2019 10:49:18 +0100 Subject: RFR[8230946]: Clarify security manager behaviour of a connected DatagramSocket and DatagramChannel In-Reply-To: References: <806a8cef-225e-ba5e-e2d5-74c6610321d0@oracle.com> <8dc97f45-be76-eb51-6073-b835479790f7@oracle.com> Message-ID: Hi, I've updated the fix for JDK-8230946 with the feedback received here and from the CSR (JDK-8231305 ). The new webrev can be found below: http://cr.openjdk.java.net/~pconcannon/8230946/webrevs/webrev.04/ Kind regards, Patrick On 30/09/2019 16:45, Chris Hegarty wrote: > > On 30/09/2019 15:27, Alan Bateman wrote: >> On 30/09/2019 15:10, Patrick Concannon wrote: >>> Hi Alan, >>> >>> >>> A new issue has been created as requested, and can be found here: >>> https://bugs.openjdk.java.net/browse/JDK-8231570 >> In JDK-8231570, I think Chris is concerned that a custom SM will >> observe, and may prevent, send/receive from binding where as an >> implicit bind by connect is not observed by the SM. It will not be >> wrong for the binding in connect to go through the SM too. > > One could reasonably write a test that connects an unbound channel, > and expect that SM::checkListen is invoked. > > My intention was to use 8231570, in part, to examine the` @throws > SecurityException` for receive. ( as well as what was mentioned above ) > > >> The related issue that we touched on here is that >> DatagramChannel.receive does not specify that it discards datagrams >> when denied by the SM. I think we will need to adjust the wording for >> the SecurityException. > > DatagramChannel::receive has the following @throws: > > ???? * @throws? SecurityException > ???? *????????? If a security manager has been installed > ???? *????????? and it does not permit datagrams to be accepted > ???? *????????? from the datagram's sender > > This is incorrect. > > The specification should say that denied datagram packets will be > discarded **. > > The only possible "fix" here, ignoring implicit bind since that will > be handled by 8231570, is to remove the @throws ( as well as adding > some spec verbiage, which is already implicit through the link to > DatagramSocket ). But it might be prudent to not remove @throws yet, > given 8231570 is still unresolved. 8231570 can be used to "fix" the > @throws, to either 1) remove it, or 2) modify it to say that the bind > may throw. > > I don't have a strong opinion either way which JIRA issue is used for > which parts of the change, just that the changes happen without > removing and re-adding an @throws > > -Chris. > > ** If registered with a selector for OP_READ, after waking up that > selector then discarding the datagram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Wed Oct 2 09:54:29 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 2 Oct 2019 10:54:29 +0100 Subject: RFR[8230946]: Clarify security manager behaviour of a connected DatagramSocket and DatagramChannel In-Reply-To: References: <806a8cef-225e-ba5e-e2d5-74c6610321d0@oracle.com> <8dc97f45-be76-eb51-6073-b835479790f7@oracle.com> Message-ID: <946D58E5-19FC-4247-BEE1-57EB76044BB9@oracle.com> > On 2 Oct 2019, at 10:49, Patrick Concannon wrote: > Hi, > > I've updated the fix for JDK-8230946 with the feedback received here and from the CSR (JDK-8231305). The new webrev can be found below: > http://cr.openjdk.java.net/~pconcannon/8230946/webrevs/webrev.04/ > Thanks Patrick. Looks good to me. -Chris. From daniel.fuchs at oracle.com Wed Oct 2 10:05:34 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 2 Oct 2019 11:05:34 +0100 Subject: RFR[8230946]: Clarify security manager behaviour of a connected DatagramSocket and DatagramChannel In-Reply-To: References: <806a8cef-225e-ba5e-e2d5-74c6610321d0@oracle.com> <8dc97f45-be76-eb51-6073-b835479790f7@oracle.com> Message-ID: Hi Patrick: DatagramSocket.java: typo 712 * If there is a security manager, and th esocket is not currently Otherwise looks good to me too! best regards, -- daniel On 02/10/2019 10:49, Patrick Concannon wrote: > Hi, > > > I've updated the fix for JDK-8230946 > with the feedback > received here and from the CSR (JDK-8231305 > ). The new webrev can > be found below: > http://cr.openjdk.java.net/~pconcannon/8230946/webrevs/webrev.04/ > > > Kind regards, From patrick.concannon at oracle.com Wed Oct 2 10:16:57 2019 From: patrick.concannon at oracle.com (Patrick Concannon) Date: Wed, 2 Oct 2019 11:16:57 +0100 Subject: RFR[8230946]: Clarify security manager behaviour of a connected DatagramSocket and DatagramChannel In-Reply-To: References: <806a8cef-225e-ba5e-e2d5-74c6610321d0@oracle.com> <8dc97f45-be76-eb51-6073-b835479790f7@oracle.com> Message-ID: <4055d018-93d0-511c-7976-851843e67c99@oracle.com> Hi Daniel, Well spotted. I've fixed that now, and updated the webrev. Kind regards, Patrick On 02/10/2019 11:05, Daniel Fuchs wrote: > Hi Patrick: > > DatagramSocket.java: typo > > 712????? * If there is a security manager, and th esocket is not > currently > > Otherwise looks good to me too! > > best regards, > > -- daniel > > On 02/10/2019 10:49, Patrick Concannon wrote: >> Hi, >> >> >> I've updated the fix for JDK-8230946 >> with the feedback >> received here and from the CSR (JDK-8231305 >> ). The new webrev >> can be found below: >> http://cr.openjdk.java.net/~pconcannon/8230946/webrevs/webrev.04/ >> >> >> Kind regards, From Alan.Bateman at oracle.com Wed Oct 2 12:44:19 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 2 Oct 2019 13:44:19 +0100 Subject: RFR[8230946]: Clarify security manager behaviour of a connected DatagramSocket and DatagramChannel In-Reply-To: <4055d018-93d0-511c-7976-851843e67c99@oracle.com> References: <806a8cef-225e-ba5e-e2d5-74c6610321d0@oracle.com> <8dc97f45-be76-eb51-6073-b835479790f7@oracle.com> <4055d018-93d0-511c-7976-851843e67c99@oracle.com> Message-ID: <1dafb5f1-e879-e351-9a88-4aeb01c03408@oracle.com> On 02/10/2019 11:16, Patrick Concannon wrote: > Hi Daniel, > > > Well spotted. I've fixed that now, and updated the webrev. Looks good. From lance.andersen at oracle.com Wed Oct 2 16:45:32 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 2 Oct 2019 12:45:32 -0400 Subject: RFR 8223771: (zipfs) FileSystemProvider.newFileSystem(Path, Map) should throw IOException when called with a file that cannot be open Message-ID: <1120CB01-6DE5-41FD-BC9E-B0915958E9FA@oracle.com> Hi all, Please review a proposed fix to address that Zip FS does not throw an IOException if the file cannot be opened and is not being created. Zip FS has always thrown FileSystemAlreadyExistsException. As part of the proposed fix, I added a flag, jdk.nio.zipfs.useFileSystemNotFoundException, to allow for FileSystemAlreadyExistsException to still be thrown if needed. The bug can be found: https://bugs.openjdk.java.net/browse/JDK-8223771 The CSR which is currently in draft form can be found at: https://bugs.openjdk.java.net/browse/JDK-8231267 The webrev can be found at: http://cr.openjdk.java.net/~lancea/8223771/webrev.00/index.html Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From lance.andersen at oracle.com Wed Oct 2 16:45:26 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 2 Oct 2019 12:45:26 -0400 Subject: RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file Message-ID: Hi all, This is a request to review the proposed fix for 8231766 which allows for Files.copy/move to correctly use the specified compression method when copying/moving a file within the same Zip file. The webrev can be found at: http://cr.openjdk.java.net/~lancea/8231766/webrev.00/ Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From Alan.Bateman at oracle.com Thu Oct 3 13:03:16 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 3 Oct 2019 14:03:16 +0100 Subject: RFR 8223771: (zipfs) FileSystemProvider.newFileSystem(Path, Map) should throw IOException when called with a file that cannot be open In-Reply-To: <1120CB01-6DE5-41FD-BC9E-B0915958E9FA@oracle.com> References: <1120CB01-6DE5-41FD-BC9E-B0915958E9FA@oracle.com> Message-ID: On 02/10/2019 17:45, Lance Andersen wrote: > Hi all, > > Please review a proposed fix to address that Zip FS does not throw an > IOException if the file cannot be opened and is not being created. > ?Zip FS has always thrown FileSystemAlreadyExistsException. > > As part of the proposed fix, I added a flag, > jdk.nio.zipfs.useFileSystemNotFoundException, to allow for > FileSystemAlreadyExistsException to still be thrown if needed. > > The bug can be found: https://bugs.openjdk.java.net/browse/JDK-8223771 You can throw the more specific NoSuchFileException here as this code path is for the case that the file has been confirmed not to exist. The change also bring up the question on whether the exception message includes the full file path or just the name (I can't tell if this is the reason to reduce the message to just the file name). It's hard to say if this bug fix needs a system property to restore old behavior or not. I can't imagine there is a lot of code depending on the FileSystemAlreadyExistsException so I'm wondering if it might be simpler to leave this out initially (it can easily be added if needed). If a property is included then the name will need discussion as "useFileSystemNotFoundException" might not be the best name for this. Separately, we should probably clean up all the places in this code that use AccessController.doPrivileged as the cast to PrivilegedAction is really ugly. The code would be a lot more readable if it creates the PrivilegedAction and then invokes AccessController.doPrivileged(pa). That voids the cast. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Thu Oct 3 13:44:04 2019 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Thu, 3 Oct 2019 09:44:04 -0400 Subject: RFR 8223771: (zipfs) FileSystemProvider.newFileSystem(Path, Map) should throw IOException when called with a file that cannot be open In-Reply-To: References: <1120CB01-6DE5-41FD-BC9E-B0915958E9FA@oracle.com> Message-ID: <27DC6F93-EB95-430C-87F2-F6CE58734B4C@oracle.com> Thank you for the feedback Alan > On Oct 3, 2019, at 9:03 AM, Alan Bateman wrote: > >> On 02/10/2019 17:45, Lance Andersen wrote: >> Hi all, >> >> Please review a proposed fix to address that Zip FS does not throw an IOException if the file cannot be opened and is not being created. Zip FS has always thrown FileSystemAlreadyExistsException. >> >> As part of the proposed fix, I added a flag, jdk.nio.zipfs.useFileSystemNotFoundException, to allow for FileSystemAlreadyExistsException to still be thrown if needed. >> >> The bug can be found: https://bugs.openjdk.java.net/browse/JDK-8223771 > You can throw the more specific NoSuchFileException here as this code path is for the case that the file has been confirmed not to exist. The change also bring up the question on whether the exception message includes the full file path or just the name (I can't tell if this is the reason to reduce the message to just the file name). Ok good point, I will change it to throw NoSuchFileException As far as the file name I am happy to provide the full path but thought it might be best to not include the full path as have had to address similar issues where there was concern of leaking the path, which do you suggest? > > It's hard to say if this bug fix needs a system property to restore old behavior or not. I can't imagine there is a lot of code depending on the FileSystemAlreadyExistsException so I'm wondering if it might be simpler to leave this out initially (it can easily be added if needed). If a property is included then the name will need discussion as "useFileSystemNotFoundException" might not be the best name for this. Ok I can remove it for now and we can revisit I just thought given the exception had always been thrown, that it would be best to add this now but undocumented. > > Separately, we should probably clean up all the places in this code that use AccessController.doPrivileged as the cast to PrivilegedAction is really ugly. The code would be a lot more readable if it creates the PrivilegedAction and then invokes AccessController.doPrivileged(pa). That voids the cast. Will log a bug and address this next > > -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Oct 3 16:17:34 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 3 Oct 2019 17:17:34 +0100 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> Message-ID: On 01/10/2019 17:07, Brian Burkhalter wrote: > : > > Here is an updated version with default instead of abstract methods: > > http://cr.openjdk.java.net/~bpb/8162520/webrev.01/ The default methods look right. I'm not sure about the change to the existing methods to specify a negative value. That isn't going to work with code that calls say getUnallocatedSpace() to get an idea on whether there is enough space to create a large file. You'll probably end up picking the best of a bad bunch so finding at least some usages of the existing methods may help. -Alan From brian.burkhalter at oracle.com Thu Oct 3 16:49:05 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 3 Oct 2019 09:49:05 -0700 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> Message-ID: <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> > On Oct 3, 2019, at 9:17 AM, Alan Bateman wrote: > > On 01/10/2019 17:07, Brian Burkhalter wrote: >> : >> Here is an updated version with default instead of abstract methods: >> >> http://cr.openjdk.java.net/~bpb/8162520/webrev.01/ > The default methods look right. I'm not sure about the change to the existing methods to specify a negative value. That isn't going to work with code that calls say getUnallocatedSpace() to get an idea on whether there is enough space to create a large file. You'll probably end up picking the best of a bad bunch so finding at least some usages of the existing methods may help. So I think you are suggesting that returning Long.MAX_VALUE would signify that the result is actually >= Long.MAX_VALUE. That seems OK to me but I think would need to be documented. Also, I think some changes in the native layer would be needed to trap the cases where the unsigned values would overflow a signed long so that min(unsignedSize,9223372036854775807L) is returned instead of an overflowed long. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Thu Oct 3 18:01:02 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 3 Oct 2019 14:01:02 -0400 Subject: RFR 8223771: (zipfs) FileSystemProvider.newFileSystem(Path, Map) should throw IOException when called with a file that cannot be open In-Reply-To: <27DC6F93-EB95-430C-87F2-F6CE58734B4C@oracle.com> References: <1120CB01-6DE5-41FD-BC9E-B0915958E9FA@oracle.com> <27DC6F93-EB95-430C-87F2-F6CE58734B4C@oracle.com> Message-ID: <816DB904-021C-4018-A5FF-3EB03AE6F7BD@oracle.com> I updated the patch: http://cr.openjdk.java.net/~lancea/8223771/webrev.01/ as well as the CSR to reflect the change and lack of a property. I reverted the change which just returned the filename when the Exception is thrown. My reasoning for just returning the filename was based on https://www.oracle.com/technetwork/java/seccodeguide-139067.html#2 . Thank you again for the review! > On Oct 3, 2019, at 9:44 AM, Lance @ Oracle wrote: > > > Thank you for the feedback Alan > > > On Oct 3, 2019, at 9:03 AM, Alan Bateman > wrote: > >> On 02/10/2019 17:45, Lance Andersen wrote: >>> Hi all, >>> >>> Please review a proposed fix to address that Zip FS does not throw an IOException if the file cannot be opened and is not being created. Zip FS has always thrown FileSystemAlreadyExistsException. >>> >>> As part of the proposed fix, I added a flag, jdk.nio.zipfs.useFileSystemNotFoundException, to allow for FileSystemAlreadyExistsException to still be thrown if needed. >>> >>> The bug can be found: https://bugs.openjdk.java.net/browse/JDK-8223771 You can throw the more specific NoSuchFileException here as this code path is for the case that the file has been confirmed not to exist. The change also bring up the question on whether the exception message includes the full file path or just the name (I can't tell if this is the reason to reduce the message to just the file name). > > Ok good point, I will change it to throw NoSuchFileException > > As far as the file name I am happy to provide the full path but thought it might be best to not include the full path as have had to address similar issues where there was concern of leaking the path, which do you suggest? >> >> It's hard to say if this bug fix needs a system property to restore old behavior or not. I can't imagine there is a lot of code depending on the FileSystemAlreadyExistsException so I'm wondering if it might be simpler to leave this out initially (it can easily be added if needed). If a property is included then the name will need discussion as "useFileSystemNotFoundException" might not be the best name for this. > > Ok I can remove it for now and we can revisit I just thought given the exception had always been thrown, that it would be best to add this now but undocumented. >> >> Separately, we should probably clean up all the places in this code that use AccessController.doPrivileged as the cast to PrivilegedAction is really ugly. The code would be a lot more readable if it creates the PrivilegedAction and then invokes AccessController.doPrivileged(pa). That voids the cast. > > > Will log a bug and address this next >> >> -Alan Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From Alan.Bateman at oracle.com Thu Oct 3 18:43:19 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 3 Oct 2019 19:43:19 +0100 Subject: RFR 8223771: (zipfs) FileSystemProvider.newFileSystem(Path, Map) should throw IOException when called with a file that cannot be open In-Reply-To: <816DB904-021C-4018-A5FF-3EB03AE6F7BD@oracle.com> References: <1120CB01-6DE5-41FD-BC9E-B0915958E9FA@oracle.com> <27DC6F93-EB95-430C-87F2-F6CE58734B4C@oracle.com> <816DB904-021C-4018-A5FF-3EB03AE6F7BD@oracle.com> Message-ID: <0582efa5-2f7f-39e3-aa98-1a589e2de578@oracle.com> On 03/10/2019 19:01, Lance Andersen wrote: > I updated the patch: > http://cr.openjdk.java.net/~lancea/8223771/webrev.01/?as well as the > CSR to reflect the change and lack of a property. Looks good. I tweaked a few things in the CSR and added myself as a Reviewer. > > I reverted the change ?which just returned the filename when the > Exception is thrown. ?My reasoning for just returning the filename was > based on > https://www.oracle.com/technetwork/java/seccodeguide-139067.html#2?. > Right, there's a discussion topic there as exception messages can leak interesting file paths. -Alan From lance.andersen at oracle.com Thu Oct 3 18:52:01 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 3 Oct 2019 14:52:01 -0400 Subject: RFR 8223771: (zipfs) FileSystemProvider.newFileSystem(Path, Map) should throw IOException when called with a file that cannot be open In-Reply-To: <0582efa5-2f7f-39e3-aa98-1a589e2de578@oracle.com> References: <1120CB01-6DE5-41FD-BC9E-B0915958E9FA@oracle.com> <27DC6F93-EB95-430C-87F2-F6CE58734B4C@oracle.com> <816DB904-021C-4018-A5FF-3EB03AE6F7BD@oracle.com> <0582efa5-2f7f-39e3-aa98-1a589e2de578@oracle.com> Message-ID: <0FA78DE1-32BB-44A6-9A31-97CCF3EFDFF8@oracle.com> Thank you Alan. > On Oct 3, 2019, at 2:43 PM, Alan Bateman wrote: > > On 03/10/2019 19:01, Lance Andersen wrote: >> I updated the patch: http://cr.openjdk.java.net/~lancea/8223771/webrev.01/ as well as the CSR to reflect the change and lack of a property. > Looks good. I tweaked a few things in the CSR and added myself as a Reviewer. I finalized the CSR, thank you for adding yourself as a reviewer. > >> >> I reverted the change which just returned the filename when the Exception is thrown. My reasoning for just returning the filename was based on https://www.oracle.com/technetwork/java/seccodeguide-139067.html#2 . >> > Right, there's a discussion topic there as exception messages can leak interesting file paths. A discussion for a different day :-) Best Lance > > -Alan Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From daniel.fuchs at oracle.com Fri Oct 4 13:34:45 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 4 Oct 2019 14:34:45 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) Message-ID: Hi, Please find below a fix for: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) https://bugs.openjdk.java.net/browse/JDK-8231260 The nature of the fix introduces an unlucky (though unlikely) case where an exception might get thrown which leaves the underlying native socket in an unspecified state. An API note is added to the specification of `DatagramChannel::disconnect` to recommend closing the channel in that case: CSR to review: CSR: https://bugs.openjdk.java.net/browse/JDK-8231880 webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.00/ best regards, -- daniel From Alan.Bateman at oracle.com Fri Oct 4 13:55:44 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 4 Oct 2019 14:55:44 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: References: Message-ID: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> On 04/10/2019 14:34, Daniel Fuchs wrote: > : > > webrev: > http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.00/ > The apiNote looks okay. DatagramChannelImpl::disconnect looks okay but I assume "might not preserve" should be "does not preserve" (if you make that change then the "This is the expected ..." line isn't needed). One suggestion is to rename the test to AddressesAfterDisconnect to make it a bit clearer that it tests the local/remote addresses after disconnect. I also assume we can change this to a TestNG test to avoid the need for the assertXXX methods. -Alan. From daniel.fuchs at oracle.com Fri Oct 4 14:25:46 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 4 Oct 2019 15:25:46 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> References: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> Message-ID: On 04/10/2019 14:55, Alan Bateman wrote: > On 04/10/2019 14:34, Daniel Fuchs wrote: >> : >> >> webrev: >> http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.00/ >> > The apiNote looks okay. DatagramChannelImpl::disconnect looks okay but I > assume "might not preserve" should be "does not preserve" (if you make > that change then the "This is the expected ..." line isn't needed). Thanks Alan. The current wording was to make it clear that this behavior is not considered as a bug in the Linux kernel. But maybe it doesn't come out as I intended. > One suggestion is to rename the test to AddressesAfterDisconnect to make > it a bit clearer that it tests the local/remote addresses after > disconnect. I also assume we can change this to a TestNG test to avoid > the need for the assertXXX methods. Good idea. I will work on that. best regards, -- daniel From fweimer at redhat.com Fri Oct 4 15:25:07 2019 From: fweimer at redhat.com (Florian Weimer) Date: Fri, 04 Oct 2019 17:25:07 +0200 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: (Daniel Fuchs's message of "Fri, 4 Oct 2019 14:34:45 +0100") References: Message-ID: <87v9t4ednw.fsf@oldenburg2.str.redhat.com> * Daniel Fuchs: > Hi, > > Please find below a fix for: > > 8231260: (dc) DatagramChannel::disconnect changes the port of the > local address to 0 (lnx) > https://bugs.openjdk.java.net/browse/JDK-8231260 I remember looking at this a couple of years ago. I was mostly concerned about the legacy datagram socket back then, where the rebinding can fail as well. Not sure if a bug was field about the behavioral different between the two socket implementations at the time. I'm a bit worried about the first hunk: @@ -873,10 +873,15 @@ synchronized (stateLock) { ensureOpen(); if (state == ST_CONNECTED) throw new AlreadyConnectedException(); + // ensure that the socket is bound + if (localAddress == null) { + bindInternal(null); + } + int n = Net.connect(family, fd, isa.getAddress(), isa.getPort()); if (n <= 0) Doesn't it introduce a race condition where the socket has a port and queue packets, but the kernel will not perform source address checking because it is unconnected? The legacy datagram socket implementation also has logic to deal with packets that arrive after a disconnect (CVE-2014-6512, the explicitFilter stuff). I think DatagramChannelImpl would need something comparable if we want to paper over the kernel behavior. Some background: Thanks, Florian From Alan.Bateman at oracle.com Fri Oct 4 15:50:26 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 4 Oct 2019 16:50:26 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <87v9t4ednw.fsf@oldenburg2.str.redhat.com> References: <87v9t4ednw.fsf@oldenburg2.str.redhat.com> Message-ID: <7b26f142-47a6-d240-b623-2248d289af98@oracle.com> On 04/10/2019 16:25, Florian Weimer wrote: > : > @@ -873,10 +873,15 @@ > synchronized (stateLock) { > ensureOpen(); > if (state == ST_CONNECTED) > throw new AlreadyConnectedException(); > > + // ensure that the socket is bound > + if (localAddress == null) { > + bindInternal(null); > + } > + > int n = Net.connect(family, > fd, > isa.getAddress(), > isa.getPort()); > if (n <= 0) > > > Doesn't it introduce a race condition where the socket has a port and > queue packets, but the kernel will not perform source address checking > because it is unconnected? > The socket is unbound, this change to the connect method in Daniel's patch just binds it before connecting. The kernel will do this anyway so it's not strictly needed here (at least for the case that the connect succeeds_. There's a completely unrelated issue with the security manager (SM) checkListen being consulted when binding but that is a topic for another thread. -Alan From fweimer at redhat.com Fri Oct 4 16:03:24 2019 From: fweimer at redhat.com (Florian Weimer) Date: Fri, 04 Oct 2019 18:03:24 +0200 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <7b26f142-47a6-d240-b623-2248d289af98@oracle.com> (Alan Bateman's message of "Fri, 4 Oct 2019 16:50:26 +0100") References: <87v9t4ednw.fsf@oldenburg2.str.redhat.com> <7b26f142-47a6-d240-b623-2248d289af98@oracle.com> Message-ID: <87eezsebw3.fsf@oldenburg2.str.redhat.com> * Alan Bateman: > On 04/10/2019 16:25, Florian Weimer wrote: >> : >> @@ -873,10 +873,15 @@ >> synchronized (stateLock) { >> ensureOpen(); >> if (state == ST_CONNECTED) >> throw new AlreadyConnectedException(); >> + // ensure that the socket is bound >> + if (localAddress == null) { >> + bindInternal(null); >> + } >> + >> int n = Net.connect(family, >> fd, >> isa.getAddress(), >> isa.getPort()); >> if (n <= 0) >> >> >> Doesn't it introduce a race condition where the socket has a port and >> queue packets, but the kernel will not perform source address checking >> because it is unconnected? >> > The socket is unbound, this change to the connect method in Daniel's > patch just binds it before connecting. The kernel will do this anyway > so it's not strictly needed here (at least for the case that the > connect succeeds_. The kernel will do it atomically during connect, so there is no race condition. With the change above, the port is selected and opened, packets can arrive (from arbitrary addresses), and only then the socket is bound to the remote address. Thanks, Florian From daniel.fuchs at oracle.com Fri Oct 4 16:07:23 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 4 Oct 2019 17:07:23 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <87v9t4ednw.fsf@oldenburg2.str.redhat.com> References: <87v9t4ednw.fsf@oldenburg2.str.redhat.com> Message-ID: <8017e9ff-9d4f-a087-b04a-354c20bd5934@oracle.com> Hi Florian, On 04/10/2019 16:25, Florian Weimer wrote: > Doesn't it introduce a race condition where the socket has a port and > queue packets, but the kernel will not perform source address checking > because it is unconnected? The long standing behavior of DatagramSocket::connect is to throw out any packet in the queue after connecting, so this changes nothing. See at line 897: http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.00/src/java.base/share/classes/sun/nio/ch/DatagramChannelImpl.java.frames.html best regards, -- daniel From fweimer at redhat.com Fri Oct 4 16:08:22 2019 From: fweimer at redhat.com (Florian Weimer) Date: Fri, 04 Oct 2019 18:08:22 +0200 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <87eezsebw3.fsf@oldenburg2.str.redhat.com> (Florian Weimer's message of "Fri, 04 Oct 2019 18:03:24 +0200") References: <87v9t4ednw.fsf@oldenburg2.str.redhat.com> <7b26f142-47a6-d240-b623-2248d289af98@oracle.com> <87eezsebw3.fsf@oldenburg2.str.redhat.com> Message-ID: <877e5kebnt.fsf@oldenburg2.str.redhat.com> * Florian Weimer: > * Alan Bateman: > >> On 04/10/2019 16:25, Florian Weimer wrote: >>> : >>> @@ -873,10 +873,15 @@ >>> synchronized (stateLock) { >>> ensureOpen(); >>> if (state == ST_CONNECTED) >>> throw new AlreadyConnectedException(); >>> + // ensure that the socket is bound >>> + if (localAddress == null) { >>> + bindInternal(null); >>> + } >>> + >>> int n = Net.connect(family, >>> fd, >>> isa.getAddress(), >>> isa.getPort()); >>> if (n <= 0) >>> >>> >>> Doesn't it introduce a race condition where the socket has a port and >>> queue packets, but the kernel will not perform source address checking >>> because it is unconnected? >>> >> The socket is unbound, this change to the connect method in Daniel's >> patch just binds it before connecting. The kernel will do this anyway >> so it's not strictly needed here (at least for the case that the >> connect succeeds_. > > The kernel will do it atomically during connect, so there is no race > condition. With the change above, the port is selected and opened, > packets can arrive (from arbitrary addresses), and only then the socket > is bound to the remote address. Ah, sorry, there is this in connect(SocketAddress): // flush any packets already received. boolean blocking = isBlocking(); if (blocking) { IOUtil.configureBlocking(fd, false); } try { ByteBuffer buf = ByteBuffer.allocate(100); while (receive(fd, buf, false) > 0) { buf.clear(); } } finally { if (blocking) { IOUtil.configureBlocking(fd, true); } } So any pending datagrams are cleared in userspace, and the race is benign and not observable by applications. (I forgot that this was added in 2014, sorry.) Thanks, Florian From Alan.Bateman at oracle.com Fri Oct 4 16:15:06 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 4 Oct 2019 17:15:06 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <87eezsebw3.fsf@oldenburg2.str.redhat.com> References: <87v9t4ednw.fsf@oldenburg2.str.redhat.com> <7b26f142-47a6-d240-b623-2248d289af98@oracle.com> <87eezsebw3.fsf@oldenburg2.str.redhat.com> Message-ID: <07d32d2b-513d-8882-777d-eadeace79e1f@oracle.com> On 04/10/2019 17:03, Florian Weimer wrote: > : > The kernel will do it atomically during connect, so there is no race > condition. With the change above, the port is selected and opened, > packets can arrive (from arbitrary addresses), and only then the socket > is bound to the remote address. > I don't see an issue here as packets in the socket buffer are read and discarded after the address association is setup in the kernel. Also just to say that the spec has already set expectations that connect would do an explicit bind ("If this channel's socket is not bound ...") although that is only behavior observable when running with a custom security manager. -Alan. From fweimer at redhat.com Fri Oct 4 16:25:55 2019 From: fweimer at redhat.com (Florian Weimer) Date: Fri, 04 Oct 2019 18:25:55 +0200 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <07d32d2b-513d-8882-777d-eadeace79e1f@oracle.com> (Alan Bateman's message of "Fri, 4 Oct 2019 17:15:06 +0100") References: <87v9t4ednw.fsf@oldenburg2.str.redhat.com> <7b26f142-47a6-d240-b623-2248d289af98@oracle.com> <87eezsebw3.fsf@oldenburg2.str.redhat.com> <07d32d2b-513d-8882-777d-eadeace79e1f@oracle.com> Message-ID: <87muegcwa4.fsf@oldenburg2.str.redhat.com> * Alan Bateman: > On 04/10/2019 17:03, Florian Weimer wrote: >> : >> The kernel will do it atomically during connect, so there is no race >> condition. With the change above, the port is selected and opened, >> packets can arrive (from arbitrary addresses), and only then the socket >> is bound to the remote address. > I don't see an issue here as packets in the socket buffer are read and > discarded after the address association is setup in the kernel. On Solaris, yes, but historically not on Linux. I see a flush for the transmission queue, but not the reception queue. But given that the DatagramChannelImpl code flushes the queue manually after connect, that difference really should not matter. Thanks, Florian From fweimer at redhat.com Fri Oct 4 16:26:21 2019 From: fweimer at redhat.com (Florian Weimer) Date: Fri, 04 Oct 2019 18:26:21 +0200 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <8017e9ff-9d4f-a087-b04a-354c20bd5934@oracle.com> (Daniel Fuchs's message of "Fri, 4 Oct 2019 17:07:23 +0100") References: <87v9t4ednw.fsf@oldenburg2.str.redhat.com> <8017e9ff-9d4f-a087-b04a-354c20bd5934@oracle.com> Message-ID: <87lfu0cw9e.fsf@oldenburg2.str.redhat.com> * Daniel Fuchs: > Hi Florian, > > On 04/10/2019 16:25, Florian Weimer wrote: >> Doesn't it introduce a race condition where the socket has a port and >> queue packets, but the kernel will not perform source address checking >> because it is unconnected? > > The long standing behavior of DatagramSocket::connect is to throw out > any packet in the queue after connecting, so this changes nothing. > > See at line 897: > http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.00/src/java.base/share/classes/sun/nio/ch/DatagramChannelImpl.java.frames.html Yes, sorry, it's a bit non-obvious though. Maybe add a comment that arriving packets are either flushed by the kernel or by connect() further down? Thanks, Florian From daniel.fuchs at oracle.com Mon Oct 7 11:34:54 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 7 Oct 2019 12:34:54 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> References: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> Message-ID: <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> Hi Alan, Here is the new webrev - I believe I have addressed all your comments: http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.01 best regards, -- daniel On 04/10/2019 14:55, Alan Bateman wrote: > On 04/10/2019 14:34, Daniel Fuchs wrote: >> : >> >> webrev: >> http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.00/ >> > The apiNote looks okay. DatagramChannelImpl::disconnect looks okay but I > assume "might not preserve" should be "does not preserve" (if you make > that change then the "This is the expected ..." line isn't needed). > > One suggestion is to rename the test to AddressesAfterDisconnect to make > it a bit clearer that it tests the local/remote addresses after > disconnect. I also assume we can change this to a TestNG test to avoid > the need for the assertXXX methods. > > -Alan. From Alan.Bateman at oracle.com Mon Oct 7 11:44:10 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 7 Oct 2019 12:44:10 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> References: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> Message-ID: <656de46d-d2c8-91a8-b8b4-178c1e34aeb7@oracle.com> On 07/10/2019 12:34, Daniel Fuchs wrote: > Hi Alan, > > Here is the new webrev - I believe I have addressed all your comments: > > http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.01 Thanks for the update, this version looks good to me. -Alan. From chris.hegarty at oracle.com Mon Oct 7 12:33:36 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 7 Oct 2019 13:33:36 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> References: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> Message-ID: > On 7 Oct 2019, at 12:34, Daniel Fuchs wrote: > > Hi Alan, > > Here is the new webrev - I believe I have addressed all your comments: > > http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.01 Looks good Daniel. -Chris From fweimer at redhat.com Mon Oct 7 13:44:39 2019 From: fweimer at redhat.com (Florian Weimer) Date: Mon, 07 Oct 2019 15:44:39 +0200 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> (Daniel Fuchs's message of "Mon, 7 Oct 2019 12:34:54 +0100") References: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> Message-ID: <87wodgk6uw.fsf@oldenburg2.str.redhat.com> * Daniel Fuchs: > Here is the new webrev - I believe I have addressed all your comments: > > http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.01 Looks good to me too. Sorry for the distraction earlier. Thanks, Florian From Alan.Bateman at oracle.com Mon Oct 7 14:08:50 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 7 Oct 2019 15:08:50 +0100 Subject: 8231921: SelectorImpl.register does not need to set the attachment when it is null Message-ID: This is a small improvement to SelectableChannel.register to avoid needing to getAndSet the attachment when initially registered without an attachment.? While in the area, I've replaced the use of AtomicReferenceFieldUpdater with a VarHandle. The changes are simple: http://cr.openjdk.java.net/~alanb/8231921/webrev/index.html -Alan From macanaoire at hotmail.com Mon Oct 7 15:09:29 2019 From: macanaoire at hotmail.com (mark sheppard) Date: Mon, 7 Oct 2019 15:09:29 +0000 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> References: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com>, <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> Message-ID: Hi Daniel, wrt impl note ... would some explanation on the esoteric reason for a now possible BindException be useful, also? namely that on some platforms the socket is unbound during the disconnect and requires a re-bind, which for ephemeral ports allocated may result in a BindException. try { statusUpdateDatagramChannel.disconnect(); } catch (IOException ioEx) { clientLogger.debug("Unexpected exception " + ioEx.getClass().getName(); statusUpdateDatagramChannel.close(); statusUpdateDatagramChannel = createStatusServiceDatagramChannel(); } ... the output will show a java.net.BindException, which is sort of arcane for a disconnect ? If it is strongly recommended to close the DatagramChannel, why not actually close the DatagramChannel in the event that the Net.bind throws a BindException ? I presume that on linux, if the port is not an OS chosen ephemeral port, i.e. it is a port from the registered port range that the reset of the port to 0 doesn't occur in a disconnect ? on the ln 947 localAddress = isa; if this was localAddress = null; would that allow a connect to be subsequently called and the DatagramChannel usable again, even when an BindException on the disconnect has occurred, as the DatagramChannel would be rebound by the connect ? In any case, I think it would be informative to disclose some further details on the possibility of a (re-)bind failure in the disconnect ! best regards Mark ________________________________ From: net-dev on behalf of Daniel Fuchs Sent: Monday 7 October 2019 11:34 To: Alan Bateman ; nio-dev Cc: OpenJDK Network Dev list Subject: Re: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) Hi Alan, Here is the new webrev - I believe I have addressed all your comments: http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.01 best regards, -- daniel On 04/10/2019 14:55, Alan Bateman wrote: > On 04/10/2019 14:34, Daniel Fuchs wrote: >> : >> >> webrev: >> http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.00/ >> > The apiNote looks okay. DatagramChannelImpl::disconnect looks okay but I > assume "might not preserve" should be "does not preserve" (if you make > that change then the "This is the expected ..." line isn't needed). > > One suggestion is to rename the test to AddressesAfterDisconnect to make > it a bit clearer that it tests the local/remote addresses after > disconnect. I also assume we can change this to a TestNG test to avoid > the need for the assertXXX methods. > > -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Mon Oct 7 21:23:47 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 7 Oct 2019 14:23:47 -0700 Subject: 8231921: SelectorImpl.register does not need to set the attachment when it is null In-Reply-To: References: Message-ID: <1BD255A5-009A-420C-B8EA-B5C3E02268BB@oracle.com> > On Oct 7, 2019, at 7:08 AM, Alan Bateman wrote: > > This is a small improvement to SelectableChannel.register to avoid needing to getAndSet the attachment when initially registered without an attachment. While in the area, I've replaced the use of AtomicReferenceFieldUpdater with a VarHandle. The changes are simple: > > http://cr.openjdk.java.net/~alanb/8231921/webrev/index.html Looks all right. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Tue Oct 8 09:16:25 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 8 Oct 2019 10:16:25 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: References: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> Message-ID: Hi Mark, On 07/10/2019 16:09, mark sheppard wrote: > Hi Daniel, > ? ?wrt impl note ... would some explanation on the esoteric reason for > a now possible BindException? be > useful, also? [...] > the output will show a java.net.BindException, which is sort of arcane > for a disconnect ? There is such a thing as over specification for a dark system specific corner case of the API. The CSR [1] and release notes [2] should have enough documentation without engraving such details in stone in the Java SE specification. > If it is strongly recommended to close the DatagramChannel, why not > actually > close the DatagramChannel in the event that the Net.bind throws a > BindException ? That would be much more surprising and also have more backward compatibility risks than the proposed fix. > I presume that on linux, if the port is not an OS? chosen ephemeral > port,? i.e. it is a port from the registered port range > that the reset of the port to 0 doesn't occur in a disconnect ? Yes. Although AFAIK it's not the port range that matters, but whether the port was originally chosen by the system or not. > on the ln 947? ?localAddress = isa; > > if this was? localAddress = null;? would that allow a connect to be > subsequently called and the DatagramChannel usable again, > even when an BindException on the disconnect has occurred, as the > DatagramChannel would be rebound by the connect ? As the API note say: it is strongly recommended to close the channel if disconnect throws an IOException. Whether the channel is usable or not. If you don't then you'd be depending on unspecified behavior of the implementation. > In any case, I think it would be informative to disclose some further > details on the possibility of a (re-)bind failure? in the disconnect ! The release note is there for that - it can be referred to if the issue arises. best regards, -- daniel [1] https://bugs.openjdk.java.net/browse/JDK-8231880 [2] https://bugs.openjdk.java.net/browse/JDK-8231881 > > best regards > Mark > > > > ------------------------------------------------------------------------ > *From:* net-dev on behalf of Daniel > Fuchs > *Sent:* Monday 7 October 2019 11:34 > *To:* Alan Bateman ; nio-dev > > *Cc:* OpenJDK Network Dev list > *Subject:* Re: RFR: 8231260: (dc) DatagramChannel::disconnect changes > the port of the local address to 0 (lnx) > Hi Alan, > > Here is the new webrev - I believe I have addressed all your comments: > > http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.01 > > best regards, > > -- daniel > > On 04/10/2019 14:55, Alan Bateman wrote: >> On 04/10/2019 14:34, Daniel Fuchs wrote: >>> : >>> >>> webrev: >>> http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.00/ >>> >> The apiNote looks okay. DatagramChannelImpl::disconnect looks okay but I >> assume "might not preserve" should be "does not preserve" (if you make >> that change then the "This is the expected ..." line isn't needed). >> >> One suggestion is to rename the test to AddressesAfterDisconnect to make >> it a bit clearer that it tests the local/remote addresses after >> disconnect. I also assume we can change this to a TestNG test to avoid >> the need for the assertXXX methods. >> >> -Alan. > From akashche at redhat.com Tue Oct 8 11:36:34 2019 From: akashche at redhat.com (Alex Kashchenko) Date: Tue, 8 Oct 2019 12:36:34 +0100 Subject: RFC: 8216472: Stack overflow followed by crash Message-ID: <07b40ab0-8e40-280b-58e4-6f237e1e0f6f@redhat.com> Hi, At Red Hat, we've got a windows-only crash in nio.dll with a JBoss application, that appeared to be JDK-8216472 [1]. After some investigation it was found, that Java_sun_nio_ch_WindowsSelectorImpl_00024SubSelector_poll0 function [2] allocates more than 50Kb on stack - because FD_SETSIZE is set to 1024 [3], each fd_set structure takes 8200 bytes and 6 fd_set structures are placed on stack [4][5]. If this function is called from a deep recursive java call, it can cause stack overflow and JVM process is either aborted or killed by OS. One of the solutions to this may be to set hotspot StackShadowPages parameter to 14. Currently it is 7 on windows and 20 on linux. I'd like to solicit comments and suggestions about another solution, that changes this native call in NIO moving fd_sets to heap: http://cr.openjdk.java.net/~akasko/jdk/8216472/webrev.00/ PollOverflow reproducer there causes JVM abort (or silent kill) without the patch. We tested this patch internally and haven't found any regressions; tests included JCK and SPECjbb, though we haven't run any benchmarks specifically oriented on networking performance. [1] https://bugs.openjdk.java.net/browse/JDK-8216472 [2] https://hg.openjdk.java.net/jdk/jdk/file/7f34de3cdfe9/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c#l55 [3] https://hg.openjdk.java.net/jdk/jdk/file/7f34de3cdfe9/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c#l33 [4] https://hg.openjdk.java.net/jdk/jdk/file/7f34de3cdfe9/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c#l64 [5] https://hg.openjdk.java.net/jdk/jdk/file/7f34de3cdfe9/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c#l117 -- -Alex From Alan.Bateman at oracle.com Tue Oct 8 12:22:06 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 8 Oct 2019 13:22:06 +0100 Subject: RFC: 8216472: Stack overflow followed by crash In-Reply-To: <07b40ab0-8e40-280b-58e4-6f237e1e0f6f@redhat.com> References: <07b40ab0-8e40-280b-58e4-6f237e1e0f6f@redhat.com> Message-ID: <0c5ffe92-22d5-4ef4-6441-5a0c5dcb3332@oracle.com> On 08/10/2019 12:36, Alex Kashchenko wrote: > Hi, > > At Red Hat, we've got a windows-only crash in nio.dll with a JBoss > application,? that appeared to be JDK-8216472 [1]. Just a quick ack. This one will take a significant time to study and review. My initial reaction is that that this native method needs to be completely replaced so that all the work is done in the Java code, not in native code. While in the area, I should mention that we want to replace this Selector implementation (the Windows Selector implementation is only implementation that wasn't replaced in JDK 11). The new implementation is currently blocked due to a Windows bug. We have escalated the bug to Microsoft so we have to wait to see where that goes. -Alan From macanaoire at hotmail.com Tue Oct 8 14:20:08 2019 From: macanaoire at hotmail.com (mark sheppard) Date: Tue, 8 Oct 2019 14:20:08 +0000 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: References: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> , Message-ID: Hi Daniel, ? ? thanks for the reply ?? I also meant to add that as on the ln 947? ?localAddress = isa; so in the event of the Net.bind() throwing a BindException then the localAddress in DatgramSocket instance will have a port == 0, indicating that it is unbound ? if the localAddress.getPort() == 0? ?during the connect call,? would it be valid to also add a check for this in conjunction to the null check ? // ensure that the socket is bound if (localAddress == null) { bindInternal(null); } else if (localAddress.getPort() == 0) InetSocketAddress rebindAddress = localAddress; localAddress = null; bind(rebindAddress); } to generate a rebind to the original IP address but with a new ephemeral port ? Q: is localAddress.getPort() == 0 indicative that the DatagramChannel is unbound ? best regards Mark From: Daniel Fuchs Sent: Tuesday 8 October 2019 09:16 To: mark sheppard ; Alan Bateman ; nio-dev Cc: OpenJDK Network Dev list Subject: Re: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) ? Hi Mark, On 07/10/2019 16:09, mark sheppard wrote: > Hi Daniel, >? ? ?wrt impl note ... would some explanation on the esoteric reason for > a now possible BindException? be > useful, also? [...] > the output will show a java.net.BindException, which is sort of arcane > for a disconnect ? There is such a thing as over specification for a dark system specific corner case of the API. The CSR [1] and release notes [2] should have enough documentation without engraving such details in stone in the Java SE specification. > If it is strongly recommended to close the DatagramChannel, why not > actually > close the DatagramChannel in the event that the Net.bind throws a > BindException ? That would be much more surprising and also have more backward compatibility risks than the proposed fix. > I presume that on linux, if the port is not an OS? chosen ephemeral > port,? i.e. it is a port from the registered port range > that the reset of the port to 0 doesn't occur in a disconnect ? Yes. Although AFAIK it's not the port range that matters, but whether the port was originally chosen by the system or not. > on the ln 947? ?localAddress = isa; > > if this was? localAddress = null;? would that allow a connect to be > subsequently called and the DatagramChannel usable again, > even when an BindException on the disconnect has occurred, as the? > DatagramChannel would be rebound by the connect ? As the API note say: it is strongly recommended to close the channel if disconnect throws an IOException. Whether the channel is usable or not. If you don't then you'd be depending on unspecified behavior of the implementation. > In any case, I think it would be informative to disclose some further > details on the possibility of a (re-)bind failure? in the disconnect ! The release note is there for that - it can be referred to if the issue arises. best regards, -- daniel [1] https://bugs.openjdk.java.net/browse/JDK-8231880 [2] https://bugs.openjdk.java.net/browse/JDK-8231881 > > best regards > Mark > > > > ------------------------------------------------------------------------ > *From:* net-dev on behalf of Daniel > Fuchs > *Sent:* Monday 7 October 2019 11:34 > *To:* Alan Bateman ; nio-dev > > *Cc:* OpenJDK Network Dev list > *Subject:* Re: RFR: 8231260: (dc) DatagramChannel::disconnect changes > the port of the local address to 0 (lnx) > Hi Alan, > > Here is the new webrev - I believe I have addressed all your comments: > > http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.01 > > best regards, > > -- daniel > > On 04/10/2019 14:55, Alan Bateman wrote: >> On 04/10/2019 14:34, Daniel Fuchs wrote: >>> : >>> >>> webrev: >>> http://cr.openjdk.java.net/~dfuchs/webrev_8231260/webrev.00/ >>> >> The apiNote looks okay. DatagramChannelImpl::disconnect looks okay but I >> assume "might not preserve" should be "does not preserve" (if you make >> that change then the "This is the expected ..." line isn't needed). >> >> One suggestion is to rename the test to AddressesAfterDisconnect to make >> it a bit clearer that it tests the local/remote addresses after >> disconnect. I also assume we can change this to a TestNG test to avoid >> the need for the assertXXX methods. >> >> -Alan. > From Alan.Bateman at oracle.com Tue Oct 8 14:29:19 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 8 Oct 2019 15:29:19 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: References: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> Message-ID: <6778ed68-487c-8797-565d-53d04af87b81@oracle.com> On 08/10/2019 15:20, mark sheppard wrote: > : > > Q: is localAddress.getPort() == 0 indicative that the DatagramChannel is unbound ? > getLocalAddress() returns a SocketAddress when bound, it returns null when not bound. I don't think we should get too hung up on corner case that arises when the local port cannot be restored (by re-binding). The javadoc sets the expectations that the channel's socket is in an undefined state when disconnect fails. -Alan. From daniel.fuchs at oracle.com Tue Oct 8 14:40:38 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 8 Oct 2019 15:40:38 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <6778ed68-487c-8797-565d-53d04af87b81@oracle.com> References: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> <6778ed68-487c-8797-565d-53d04af87b81@oracle.com> Message-ID: <13df615a-b7b5-0985-0ebe-801004596edf@oracle.com> On 08/10/2019 15:29, Alan Bateman wrote: > On 08/10/2019 15:20, mark sheppard wrote: >> : >> >> Q: is? localAddress.getPort() == 0?? indicative that the >> DatagramChannel is unbound ? >> > getLocalAddress() returns a SocketAddress when bound, it returns null > when not bound. I don't think we should get too hung up on corner case > that arises when the local port cannot be restored (by re-binding). The > javadoc sets the expectations that the channel's socket is in an > undefined state when disconnect fails. +1 : you would be in uncharted territory and that's exactly why we want to throw an exception when rebind fails and why we recommend closing the channel when that happens. -- daniel > > -Alan. From sgehwolf at redhat.com Tue Oct 8 17:02:23 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 08 Oct 2019 19:02:23 +0200 Subject: RFR: 8232003: (fs) Files.write can leak file descriptor in the exception case Message-ID: <88a2da51c3d8aa55630a03e2e41e6a9e04b92c70.camel@redhat.com> Hi, Could I please get a review of this fix for a resource leak on the exception path in Files.write? The resource leak can surface, for example, when the underlying filesystem runs out of disk space, but some user code catches IOException and continues to try to write to some path on the full disk. The bug has some details. The proposed fix is to use try-with-resouces on the opened OutputStream in Files.write() in a similar way as is being done with the buffered writer. Bug: https://bugs.openjdk.java.net/browse/JDK-8232003 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8232003/01/webrev/ Testing: tier1 on linux x86_64 and jdk_nio tests. Manual testing of the fix by looking at the number of open files for the java process running the reproducer. Thoughts? Thanks, Severin From Alan.Bateman at oracle.com Tue Oct 8 17:42:39 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 8 Oct 2019 18:42:39 +0100 Subject: RFR: 8232003: (fs) Files.write can leak file descriptor in the exception case In-Reply-To: <88a2da51c3d8aa55630a03e2e41e6a9e04b92c70.camel@redhat.com> References: <88a2da51c3d8aa55630a03e2e41e6a9e04b92c70.camel@redhat.com> Message-ID: <57848468-d2da-70df-9e2e-c296266c6dc7@oracle.com> On 08/10/2019 18:02, Severin Gehwolf wrote: > Hi, > > Could I please get a review of this fix for a resource leak on the > exception path in Files.write? The resource leak can surface, for > example, when the underlying filesystem runs out of disk space, but > some user code catches IOException and continues to try to write to > some path on the full disk. The bug has some details. NFS is the other scenario where close fails as the buffered bytes cannot be flushed to the server. The patch looks good. -Alan From sgehwolf at redhat.com Tue Oct 8 17:55:19 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 08 Oct 2019 19:55:19 +0200 Subject: RFR: 8232003: (fs) Files.write can leak file descriptor in the exception case In-Reply-To: <57848468-d2da-70df-9e2e-c296266c6dc7@oracle.com> References: <88a2da51c3d8aa55630a03e2e41e6a9e04b92c70.camel@redhat.com> <57848468-d2da-70df-9e2e-c296266c6dc7@oracle.com> Message-ID: <281c6424ec57ee3fbeea018ca5f04235f40b3af9.camel@redhat.com> On Tue, 2019-10-08 at 18:42 +0100, Alan Bateman wrote: > > On 08/10/2019 18:02, Severin Gehwolf wrote: > > Hi, > > > > Could I please get a review of this fix for a resource leak on the > > exception path in Files.write? The resource leak can surface, for > > example, when the underlying filesystem runs out of disk space, but > > some user code catches IOException and continues to try to write to > > some path on the full disk. The bug has some details. > NFS is the other scenario where close fails as the buffered bytes cannot > be flushed to the server. The patch looks good. Thanks for the review, Alan! Cheers, Severin From brian.burkhalter at oracle.com Tue Oct 8 18:17:20 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 8 Oct 2019 11:17:20 -0700 Subject: RFR: 8232003: (fs) Files.write can leak file descriptor in the exception case In-Reply-To: <57848468-d2da-70df-9e2e-c296266c6dc7@oracle.com> References: <88a2da51c3d8aa55630a03e2e41e6a9e04b92c70.camel@redhat.com> <57848468-d2da-70df-9e2e-c296266c6dc7@oracle.com> Message-ID: +1 Brian > On Oct 8, 2019, at 10:42 AM, Alan Bateman wrote: > > The patch looks good. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Wed Oct 9 07:20:11 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 9 Oct 2019 07:20:11 +0000 Subject: RFR: 8232003: (fs) Files.write can leak file descriptor in the exception case In-Reply-To: <57848468-d2da-70df-9e2e-c296266c6dc7@oracle.com> References: <88a2da51c3d8aa55630a03e2e41e6a9e04b92c70.camel@redhat.com> <57848468-d2da-70df-9e2e-c296266c6dc7@oracle.com> Message-ID: Hi, the patch looks good to me, too. I'm wondering, though, whether StreamEncoder::implClose shouldn't be made more resilient? E.g. when writeBytes() fails at [0], shouldn't still a close on ch or out be attempted? Maybe under a separate bug? Best regards Christoph [0] http://hg.openjdk.java.net/jdk/jdk/file/eb68d459ba6a/src/java.base/share/classes/sun/nio/cs/StreamEncoder.java#l346 > -----Original Message----- > From: nio-dev On Behalf Of Alan > Bateman > Sent: Dienstag, 8. Oktober 2019 19:43 > To: Severin Gehwolf ; nio-dev at openjdk.java.net > Subject: Re: RFR: 8232003: (fs) Files.write can leak file descriptor in the > exception case > > > > On 08/10/2019 18:02, Severin Gehwolf wrote: > > Hi, > > > > Could I please get a review of this fix for a resource leak on the > > exception path in Files.write? The resource leak can surface, for > > example, when the underlying filesystem runs out of disk space, but > > some user code catches IOException and continues to try to write to > > some path on the full disk. The bug has some details. > NFS is the other scenario where close fails as the buffered bytes cannot > be flushed to the server. The patch looks good. > > -Alan From matthias.baesken at sap.com Wed Oct 9 08:21:47 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 9 Oct 2019 08:21:47 +0000 Subject: RFR [XS] : 8232045: cleanup AIX 5.3 workarounds from libnio/ch/Net.c Message-ID: Hello, please review this small change . It removes some definitions / declaration from Net.c that were included for AIX 5.3 . However from at least AIX 6.1 those definitions / declaration are coming from the system header ( netinet/in.h ) . So with the current jdk/jdk build platform AIX 7.X we do not need them any more. Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8232045 http://cr.openjdk.java.net/~mbaesken/webrevs/8232045.0/ Thanks, Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Oct 9 09:27:04 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 9 Oct 2019 10:27:04 +0100 Subject: RFR: 8232003: (fs) Files.write can leak file descriptor in the exception case In-Reply-To: References: <88a2da51c3d8aa55630a03e2e41e6a9e04b92c70.camel@redhat.com> <57848468-d2da-70df-9e2e-c296266c6dc7@oracle.com> Message-ID: On 09/10/2019 08:20, Langer, Christoph wrote: > Hi, > > the patch looks good to me, too. > > I'm wondering, though, whether StreamEncoder::implClose shouldn't be made more resilient? > > E.g. when writeBytes() fails at [0], shouldn't still a close on ch or out be attempted? Maybe under a separate bug? > Yes, there are several long standing issues here. Brian fixed some of the issues with JDK-8220477 but there are other wises when close fails that needs attention. My guess is that Severin is looking to backport the change for JDK-8232003 so the risk there is low. Changes to StreamEncoder will need a lot more bake time. -Alan. From macanaoire at hotmail.com Wed Oct 9 09:44:00 2019 From: macanaoire at hotmail.com (mark sheppard) Date: Wed, 9 Oct 2019 09:44:00 +0000 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: <13df615a-b7b5-0985-0ebe-801004596edf@oracle.com> References: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> <6778ed68-487c-8797-565d-53d04af87b81@oracle.com>, <13df615a-b7b5-0985-0ebe-801004596edf@oracle.com> Message-ID: OK Alan, Daniel thanks for the feedback and consideration ? but what you say does lend an argument that localAddress = null; rather than localAddress = isa; is a reasonable value in the event of Net.bind throwing a BindException ?? but, as you have said, that may have other implications leaving my dead donkey well and truly flogged!! an influencing (rhetorical) question is how soon is a port available for re-allocation/re-use after it has been released by the OS on Linux? with connected TCP sockets it would expected to be the TTL setting ? not sure about a connected UDP socket! Would it be instantaneously available after its release by the kernel, or subject to TTL lifetime? best regards Mark ________________________________ From: Daniel Fuchs Sent: Tuesday 8 October 2019 14:40 To: Alan Bateman ; mark sheppard ; nio-dev Cc: OpenJDK Network Dev list Subject: Re: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) On 08/10/2019 15:29, Alan Bateman wrote: > On 08/10/2019 15:20, mark sheppard wrote: >> : >> >> Q: is localAddress.getPort() == 0 indicative that the >> DatagramChannel is unbound ? >> > getLocalAddress() returns a SocketAddress when bound, it returns null > when not bound. I don't think we should get too hung up on corner case > that arises when the local port cannot be restored (by re-binding). The > javadoc sets the expectations that the channel's socket is in an > undefined state when disconnect fails. +1 : you would be in uncharted territory and that's exactly why we want to throw an exception when rebind fails and why we recommend closing the channel when that happens. -- daniel > > -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Wed Oct 9 09:47:51 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 9 Oct 2019 09:47:51 +0000 Subject: RFR [XS] : 8232045: cleanup AIX 5.3 workarounds from libnio/ch/Net.c In-Reply-To: References: Message-ID: Hi Matthias, this looks good to me. Thanks for the cleanup. Best regards Christoph From: nio-dev On Behalf Of Baesken, Matthias Sent: Mittwoch, 9. Oktober 2019 10:22 To: nio-dev at openjdk.java.net Subject: [CAUTION] RFR [XS] : 8232045: cleanup AIX 5.3 workarounds from libnio/ch/Net.c Hello, please review this small change . It removes some definitions / declaration from Net.c that were included for AIX 5.3 . However from at least AIX 6.1 those definitions / declaration are coming from the system header ( netinet/in.h ) . So with the current jdk/jdk build platform AIX 7.X we do not need them any more. Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8232045 http://cr.openjdk.java.net/~mbaesken/webrevs/8232045.0/ Thanks, Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Oct 9 10:35:15 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 9 Oct 2019 11:35:15 +0100 Subject: RFR [XS] : 8232045: cleanup AIX 5.3 workarounds from libnio/ch/Net.c In-Reply-To: References: Message-ID: On 09/10/2019 09:21, Baesken, Matthias wrote: > > http://cr.openjdk.java.net/~mbaesken/webrevs/8232045.0/ > > Looks good to me too. -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Wed Oct 9 11:18:40 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 9 Oct 2019 12:18:40 +0100 Subject: RFR [XS] : 8232045: cleanup AIX 5.3 workarounds from libnio/ch/Net.c In-Reply-To: References: Message-ID: <659f1233-6340-53d7-74c5-81342f36ad87@oracle.com> On 09/10/2019 09:21, Baesken, Matthias wrote: > Hello, please review this small change . > > It removes some definitions / declaration?? from Net.c?? ?that were > included for AIX 5.3 . > However from at least AIX 6.1 those definitions / declaration are coming > from the system header? ( netinet/in.h ) . > So with the current ?jdk/jdk??? build platform AIX 7.X we do not need > them any more. Excellent. > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8232045 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8232045.0/ Looks good. -Chris. From adinn at redhat.com Wed Oct 9 11:20:51 2019 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 9 Oct 2019 12:20:51 +0100 Subject: RFR: 8232003: (fs) Files.write can leak file descriptor in the exception case In-Reply-To: References: <88a2da51c3d8aa55630a03e2e41e6a9e04b92c70.camel@redhat.com> <57848468-d2da-70df-9e2e-c296266c6dc7@oracle.com> Message-ID: <8e2519ac-4842-ad1e-580b-c56a6407b99a@redhat.com> On 09/10/2019 10:27, Alan Bateman wrote: > My guess is that Severin is looking to backport > the change for JDK-8232003 so the risk there is low. Your guess is correct ;-) regards, Andrew Dinn ----------- From daniel.fuchs at oracle.com Wed Oct 9 11:23:13 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 9 Oct 2019 12:23:13 +0100 Subject: RFR: 8231260: (dc) DatagramChannel::disconnect changes the port of the local address to 0 (lnx) In-Reply-To: References: <060a0438-468d-7b83-d039-4f9d0b7ed14f@oracle.com> <858b0d5f-2bd6-ac06-6e96-cb7a9abebed3@oracle.com> <6778ed68-487c-8797-565d-53d04af87b81@oracle.com> <13df615a-b7b5-0985-0ebe-801004596edf@oracle.com> Message-ID: <4456b1e1-e78c-268d-acae-93f100ae63cc@oracle.com> Hi Mark, On 09/10/2019 10:44, mark sheppard wrote: > Would it be instantaneously available after its release by the kernel, > or subject to TTL lifetime? The fact that DatagramSocket has been doing that for years and that we haven't had any issue so far suggests that the case where rebind would fail are very rare. That - or nobody uses connect/disconnect/connect :-) best regards, -- daniel From brian.burkhalter at oracle.com Fri Oct 11 00:22:02 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 10 Oct 2019 17:22:02 -0700 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> Message-ID: <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> > On Oct 3, 2019, at 9:49 AM, Brian Burkhalter wrote: > >> On Oct 3, 2019, at 9:17 AM, Alan Bateman wrote: >> >> On 01/10/2019 17:07, Brian Burkhalter wrote: >>> : >>> Here is an updated version with default instead of abstract methods: >>> >>> http://cr.openjdk.java.net/~bpb/8162520/webrev.01/ >> The default methods look right. I'm not sure about the change to the existing methods to specify a negative value. That isn't going to work with code that calls say getUnallocatedSpace() to get an idea on whether there is enough space to create a large file. You'll probably end up picking the best of a bad bunch so finding at least some usages of the existing methods may help. > > So I think you are suggesting that returning Long.MAX_VALUE would signify that the result is actually >= Long.MAX_VALUE. That seems OK to me but I think would need to be documented. Also, I think some changes in the native layer would be needed to trap the cases where the unsigned values would overflow a signed long so that min(unsignedSize,9223372036854775807L) is returned instead of an overflowed long. An updated version is here http://cr.openjdk.java.net/~bpb/8162520/webrev.02/. This changes the various get*{Blocks,Space}() methods to return Long.MAX_VALUE if the respective size would overflow a long. The patch passes all the java.nio tests on all platforms. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon Oct 14 08:51:22 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 14 Oct 2019 09:51:22 +0100 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> Message-ID: On 11/10/2019 01:22, Brian Burkhalter wrote: > An updated version is here > http://cr.openjdk.java.net/~bpb/8162520/webrev.02/.?This changes the > various get*{Blocks,Space}() methods to return Long.MAX_VALUE if the > respective size would overflow a long. The patch passes all the > java.nio tests on all platforms. Thanks, I think this is probably the best of the options so far. The new methods need to link to getBlockSize to find out what a block is. In the javadoc, the "In this case" sentence seems to suggest that the new methods should be used as a fallback when MAX_VALUE is returned. I think we need to adjust this wording to make it clearer that the getXXXBlocks methods is the recommended way to get the space information. Minor nit in the @implSpec is "throws UOE" rather than "throws an UOE" might be better. The File.getXXXSpace methods will need attention too, maybe a separate issue to clarify their specs, fix the implementation, and link them to the methods in FileStore. -Alan From brian.burkhalter at oracle.com Tue Oct 15 17:46:53 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 15 Oct 2019 10:46:53 -0700 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> Message-ID: <0DAA8543-1EF8-4C3F-9F38-AF9340E853E1@oracle.com> > On Oct 14, 2019, at 1:51 AM, Alan Bateman wrote: > > On 11/10/2019 01:22, Brian Burkhalter wrote: >> An updated version is here http://cr.openjdk.java.net/~bpb/8162520/webrev.02/ . This changes the various get*{Blocks,Space}() methods to return Long.MAX_VALUE if the respective size would overflow a long. The patch passes all the java.nio tests on all platforms. > Thanks, I think this is probably the best of the options so far. > > The new methods need to link to getBlockSize to find out what a block is. In the javadoc, the "In this case" sentence seems to suggest that the new methods should be used as a fallback when MAX_VALUE is returned. I think we need to adjust this wording to make it clearer that the getXXXBlocks methods is the recommended way to get the space information. Minor nit in the @implSpec is "throws UOE" rather than "throws an UOE" might be better. An updated version which addresses the foregoing points is at http://cr.openjdk.java.net/~bpb/8162520/webrev.03/ . > The File.getXXXSpace methods will need attention too, maybe a separate issue to clarify their specs, fix the implementation, and link them to the methods in FileStore. I filed https://bugs.openjdk.java.net/browse/JDK-8232271 to address this. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From patrick.concannon at oracle.com Wed Oct 16 18:20:52 2019 From: patrick.concannon at oracle.com (Patrick Concannon) Date: Wed, 16 Oct 2019 19:20:52 +0100 Subject: RFR[8231570]: (dc) Clarify implicit bind behavior of DatagramChannel Message-ID: <0ac5b7ec-e36d-4e4a-8639-d53d2ed73711@oracle.com> [Resending with nio-dev in cc] Hi, Could someone please review my fix and CSR for issue JDK-8231570 '(dc) Clarify implicit bind behavior of DatagramChannel' ? The fix clarifies the security checks for DatagramChannel connect, send, and receive methods. webrev: http://cr.openjdk.java.net/~pconcannon/8231570/webrevs/webrev.02/ CSR: https://bugs.openjdk.java.net/browse/JDK-8232385 Kind regards, Patrick From Alan.Bateman at oracle.com Wed Oct 16 19:11:12 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 16 Oct 2019 20:11:12 +0100 Subject: RFR[8231570]: (dc) Clarify implicit bind behavior of DatagramChannel In-Reply-To: <0ac5b7ec-e36d-4e4a-8639-d53d2ed73711@oracle.com> References: <0ac5b7ec-e36d-4e4a-8639-d53d2ed73711@oracle.com> Message-ID: <02e90382-f5d7-ee60-a324-290de7421668@oracle.com> On 16/10/2019 19:20, Patrick Concannon wrote: > : > > > webrev: http://cr.openjdk.java.net/~pconcannon/8231570/webrevs/webrev.02/ Looks good. From Alan.Bateman at oracle.com Sat Oct 19 19:05:55 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 19 Oct 2019 20:05:55 +0100 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: <0DAA8543-1EF8-4C3F-9F38-AF9340E853E1@oracle.com> References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> <0DAA8543-1EF8-4C3F-9F38-AF9340E853E1@oracle.com> Message-ID: <54934789-31f6-20e4-203e-a6d55e7ceab5@oracle.com> On 15/10/2019 18:46, Brian Burkhalter wrote: > > An updated version which addresses the foregoing points is at > http://cr.openjdk.java.net/~bpb/8162520/webrev.03/. Thanks, that addresses all my issues (although I'm still a bit worried that this one will come back again in the future). -Alan From Alan.Bateman at oracle.com Sun Oct 20 08:12:45 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 20 Oct 2019 09:12:45 +0100 Subject: RFC: 8216472: Stack overflow followed by crash In-Reply-To: <07b40ab0-8e40-280b-58e4-6f237e1e0f6f@redhat.com> References: <07b40ab0-8e40-280b-58e4-6f237e1e0f6f@redhat.com> Message-ID: On 08/10/2019 12:36, Alex Kashchenko wrote: > Hi, > > At Red Hat, we've got a windows-only crash in nio.dll with a JBoss > application,? that appeared to be JDK-8216472 [1]. > > After some investigation it was found, that > Java_sun_nio_ch_WindowsSelectorImpl_00024SubSelector_poll0 function > [2] allocates more than 50Kb on stack - because FD_SETSIZE is set to > 1024 [3], each fd_set structure takes 8200 bytes and 6 fd_set > structures are placed on stack [4][5]. If this function is called from > a deep recursive java call, it can cause stack overflow and JVM > process is either aborted or killed by OS. > > One of the solutions to this may be to set hotspot StackShadowPages > parameter to 14. Currently it is 7 on windows and 20 on linux. > > I'd like to solicit comments and suggestions about another solution, > that changes this native call in NIO moving fd_sets to heap: > > http://cr.openjdk.java.net/~akasko/jdk/8216472/webrev.00/ > > PollOverflow reproducer there causes JVM abort (or silent kill) > without the patch. > > We tested this patch internally and haven't found any regressions; > tests included JCK and SPECjbb, though we haven't run any benchmarks > specifically oriented on networking performance. I finally got time to look at this a bit more. The Windows select based Selector does need some TLC. This issue with the FD_SET structures on the stack has been there since JDK 1.4 and I'm surprised it has been reported before this issue. The underlying issue is that this Selector wants poll and ends up translating a poll array to a set of FD_SET structures on each select. This shouldn't be needed, it should instead maintain the FD_SET in the java implementation so that the native method is a call to select, nothing more. Changing this would require a lot of work on the Selector (esp. the exceptfds) and is something we should do at some point. For the current issue I think we should do the following: 1. If select fails, throw the IOException and return IOS_THROWN. I don't think any of this code has been needed for many releases as the async close of a channel registered with a Selector will consistently delay closing the SOCKET until it is flushed from all registered Selectors. That should eliminate 3 of the 6 FD_SET structures. 2. Allocate the memory for the 3 FD_SET structures in the SubSelector. That is, manage the memory from the Java code, not the native method. That eliminates the 3 * malloc/free per select that is in the proposed patch. The memory can be freed when the sub selector thread terminates. My guess is that the changes will go through several iterations and happy to provide feedback on interim revisions as needed. -Alan From vkempik at azul.com Mon Oct 21 09:46:35 2019 From: vkempik at azul.com (Vladimir Kempik) Date: Mon, 21 Oct 2019 09:46:35 +0000 Subject: RFR(8u): 8229872: (fs) Increase buffer size used with getmntent Message-ID: <5CCC5140-4C75-45CD-A3A2-CC0237DD7AD9@azul.com> Hello Please review backport of https://bugs.openjdk.java.net/browse/JDK-8229872 to 8u: http://cr.openjdk.java.net/~vkempik/8229872/ojdk8/webrev.00/ Difference between 14/13/11 fix: getline is missing on solaris10, so getlinelen method was moved from UnixNativeDispatcher to LinuxNativeDispatcher. Original fix: https://hg.openjdk.java.net/jdk/jdk/rev/2d40e6a7ce8e Such fix was already included into azul?s october psu for openjdk8 and passed all release cycle tests. Thanks, Vladimir -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Mon Oct 21 16:23:42 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 21 Oct 2019 16:23:42 +0000 Subject: RFR(8u): 8229872: (fs) Increase buffer size used with getmntent In-Reply-To: <5CCC5140-4C75-45CD-A3A2-CC0237DD7AD9@azul.com> References: <5CCC5140-4C75-45CD-A3A2-CC0237DD7AD9@azul.com> Message-ID: <7B2B9DF5-F845-42FC-949C-102B72282975@amazon.com> +jdk8u-dev Paul From: nio-dev on behalf of Vladimir Kempik Date: Monday, October 21, 2019 at 2:47 AM To: "nio-dev at openjdk.java.net" Subject: RFR(8u): 8229872: (fs) Increase buffer size used with getmntent Hello Please review backport of https://bugs.openjdk.java.net/browse/JDK-8229872 to 8u: http://cr.openjdk.java.net/~vkempik/8229872/ojdk8/webrev.00/ Difference between 14/13/11 fix: getline is missing on solaris10, so getlinelen method was moved from UnixNativeDispatcher to LinuxNativeDispatcher. Original fix: https://hg.openjdk.java.net/jdk/jdk/rev/2d40e6a7ce8e Such fix was already included into azul?s october psu for openjdk8 and passed all release cycle tests. Thanks, Vladimir -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Oct 22 15:32:06 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 22 Oct 2019 16:32:06 +0100 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method Message-ID: Lots of good work in recent releases to replace the use of finalizers in the java.base module with the cleaner mechanism. The DatagramChannel implementation is still using a finalizer so it's time to replace that. While in the area I removed the wrapper around the cleaner used in the NIO SocketImpl as it is not needed. The proposed changes are here: ? http://cr.openjdk.java.net/~alanb/8212132/webrev/index.html -Alan From fweimer at redhat.com Tue Oct 22 15:38:09 2019 From: fweimer at redhat.com (Florian Weimer) Date: Tue, 22 Oct 2019 17:38:09 +0200 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: (Alan Bateman's message of "Tue, 22 Oct 2019 16:32:06 +0100") References: Message-ID: <8736fkkcym.fsf@oldenburg2.str.redhat.com> * Alan Bateman: > Lots of good work in recent releases to replace the use of finalizers > in the java.base module with the cleaner mechanism. The > DatagramChannel implementation is still using a finalizer so it's time > to replace that. While in the area I removed the wrapper around the > cleaner used in the NIO SocketImpl as it is not needed. > > The proposed changes are here: > ? http://cr.openjdk.java.net/~alanb/8212132/webrev/index.html What's the position on guarding against faillure in the cleaner registration itself, sinde registerCleaner()? Thanks, Florian From Alan.Bateman at oracle.com Tue Oct 22 15:45:02 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 22 Oct 2019 16:45:02 +0100 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: <8736fkkcym.fsf@oldenburg2.str.redhat.com> References: <8736fkkcym.fsf@oldenburg2.str.redhat.com> Message-ID: On 22/10/2019 16:38, Florian Weimer wrote: > : > What's the position on guarding against faillure in the cleaner > registration itself, sinde registerCleaner()? I assume you mean OOME. We don't handle that in the dozen or so other places that are using this mechanism so if it happens, and the application somehow recovers, then it would be a resource leak. -Alan From lance.andersen at oracle.com Tue Oct 22 17:09:27 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 22 Oct 2019 13:09:27 -0400 Subject: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file Message-ID: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> Hi all, This is a request to review the proposed fix for 8231766 which allows for Files.copy/move to correctly use the specified compression method when copying/moving a file within the same Zip file. The webrev can be found at: http://cr.openjdk.java.net/~lancea/8231766/webrev.00/ Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From brian.burkhalter at oracle.com Tue Oct 22 17:47:00 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 22 Oct 2019 10:47:00 -0700 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: Message-ID: > On Oct 22, 2019, at 8:32 AM, Alan Bateman wrote: > > Lots of good work in recent releases to replace the use of finalizers in the java.base module with the cleaner mechanism. The DatagramChannel implementation is still using a finalizer so it's time to replace that. While in the area I removed the wrapper around the cleaner used in the NIO SocketImpl as it is not needed. > > The proposed changes are here: > http://cr.openjdk.java.net/~alanb/8212132/webrev/index.html Looks all right. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Oct 22 18:31:45 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 22 Oct 2019 19:31:45 +0100 Subject: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file In-Reply-To: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> References: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> Message-ID: <09ad2667-9965-8277-6cf9-b76dd0560c75@oracle.com> On 22/10/2019 18:09, Lance Andersen wrote: > Hi all, > > This is a request to review the proposed fix for?8231766 which allows > for Files.copy/move to correctly use the specified compression method > when copying/moving a file within the same Zip file. > > The webrev can be found at: > http://cr.openjdk.java.net/~lancea/8231766/webrev.00/ > You should be able to avoid the copy loop by changing it to is.transfer(os). It might able be a bit more readable if you rename these two to "in" and "out" but that may be just me. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Tue Oct 22 18:33:54 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 22 Oct 2019 19:33:54 +0100 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: Message-ID: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> The source changes look good to me Alan! Regarding the test: I wonder though about using the UnixOperatingSystemMXBean: does that work on Windows? best regards, -- daniel On 22/10/2019 16:32, Alan Bateman wrote: > Lots of good work in recent releases to replace the use of finalizers in > the java.base module with the cleaner mechanism. The DatagramChannel > implementation is still using a finalizer so it's time to replace that. > While in the area I removed the wrapper around the cleaner used in the > NIO SocketImpl as it is not needed. > > The proposed changes are here: > ? http://cr.openjdk.java.net/~alanb/8212132/webrev/index.html > > -Alan From Alan.Bateman at oracle.com Tue Oct 22 18:45:02 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 22 Oct 2019 19:45:02 +0100 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> Message-ID: On 22/10/2019 19:33, Daniel Fuchs wrote: > The source changes look good to me Alan! > > Regarding the test: I wonder though about using the > UnixOperatingSystemMXBean: does that work on Windows? It skips on Windows because UnixOperatingSystemMXBean is not available. It would be useful if we had an equivalent MXBean on Windows to track handle usage but I don't think anyone has got around to that. An alternative would be an @requires and the os.family key but it won't make any difference. -Alan From lance.andersen at oracle.com Tue Oct 22 21:49:15 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 22 Oct 2019 17:49:15 -0400 Subject: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file In-Reply-To: <09ad2667-9965-8277-6cf9-b76dd0560c75@oracle.com> References: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> <09ad2667-9965-8277-6cf9-b76dd0560c75@oracle.com> Message-ID: <61FCC2BC-23F2-4006-8E8F-273F402068BF@oracle.com> > On Oct 22, 2019, at 2:31 PM, Alan Bateman wrote: > > On 22/10/2019 18:09, Lance Andersen wrote: >> Hi all, >> >> This is a request to review the proposed fix for 8231766 which allows for Files.copy/move to correctly use the specified compression method when copying/moving a file within the same Zip file. >> >> The webrev can be found at: http://cr.openjdk.java.net/~lancea/8231766/webrev.00/ > You should be able to avoid the copy loop by changing it to is.transfer(os). It might able be a bit more readable if you rename these two to "in" and "out" but that may be just me. > Thank you Alan. I will try the Inputstream.transferTo(OutputStream) once I am able to build again on my mac :-( > -Alan Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From chris.hegarty at oracle.com Wed Oct 23 09:52:14 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 23 Oct 2019 10:52:14 +0100 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> Message-ID: > On 22 Oct 2019, at 19:45, Alan Bateman wrote: > > On 22/10/2019 19:33, Daniel Fuchs wrote: >> The source changes look good to me Alan! + 1. I?ve taken a careful pass over the source changes, since they follow a pattern of leaking `this` in the constructor and implementing a cleaning action with be a lambda, both of which require careful consideration, but the changes LGTM. >> Regarding the test: I wonder though about using the UnixOperatingSystemMXBean: does that work on Windows? > It skips on Windows because UnixOperatingSystemMXBean is not available. It would be useful if we had an equivalent MXBean on Windows to track handle usage but I don't think anyone has got around to that. An alternative would be an @requires and the os.family key but it won't make any difference. The difference is whether the test will be selected to-be-run by jtreg or not. SkippedException is fine, but it seems highly unlikely that UnixOperatingSystemMXBean will every be implemented on Windows, so the test in its current state just doesn?t make sense to be run on non-Unix platforms. The test is cute, it is a pity that it uses a mechanism that is known to not be supported on Windows. -Chris. From fweimer at redhat.com Wed Oct 23 10:16:52 2019 From: fweimer at redhat.com (Florian Weimer) Date: Wed, 23 Oct 2019 12:16:52 +0200 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: (Alan Bateman's message of "Tue, 22 Oct 2019 16:45:02 +0100") References: <8736fkkcym.fsf@oldenburg2.str.redhat.com> Message-ID: <87h83zhiln.fsf@oldenburg2.str.redhat.com> * Alan Bateman: > On 22/10/2019 16:38, Florian Weimer wrote: >> : >> What's the position on guarding against faillure in the cleaner >> registration itself, sinde registerCleaner()? > I assume you mean OOME. Yes, OOME or maybe StackOverflow Error (although that is much more tenuous). > We don't handle that in the dozen or so other places that are using > this mechanism so if it happens, and the application somehow recovers, > then it would be a resource leak. I see. I wonder if these additional resource leaks have caused problems in practice, since it is so common to catch OOME and keep running. In any case, this change matches the other migration to Cleaner, so it shouldn't make matters much worse. Thanks, Florian From daniel.fuchs at oracle.com Wed Oct 23 17:07:07 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 23 Oct 2019 18:07:07 +0100 Subject: RFR[8231570]: (dc) Clarify implicit bind behavior of DatagramChannel In-Reply-To: <0ac5b7ec-e36d-4e4a-8639-d53d2ed73711@oracle.com> References: <0ac5b7ec-e36d-4e4a-8639-d53d2ed73711@oracle.com> Message-ID: Hi Patrick, That looks good to me too. I can sponsor it for you after you generate a proper hg-importable changeset. best regards, -- daniel On 16/10/2019 19:20, Patrick Concannon wrote: > [Resending with nio-dev in cc] > > Hi, > > > Could someone please review my fix and CSR for issue JDK-8231570 '(dc) > Clarify implicit bind behavior of DatagramChannel' ? > > The fix clarifies the security checks for DatagramChannel connect, send, > and receive methods. > > > webrev: http://cr.openjdk.java.net/~pconcannon/8231570/webrevs/webrev.02/ > > CSR: https://bugs.openjdk.java.net/browse/JDK-8232385 > > > > Kind regards, > > Patrick > From lance.andersen at oracle.com Wed Oct 23 17:48:19 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 23 Oct 2019 13:48:19 -0400 Subject: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file In-Reply-To: <61FCC2BC-23F2-4006-8E8F-273F402068BF@oracle.com> References: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> <09ad2667-9965-8277-6cf9-b76dd0560c75@oracle.com> <61FCC2BC-23F2-4006-8E8F-273F402068BF@oracle.com> Message-ID: <43188D78-3D96-4426-9D4A-41DBC0C04A25@oracle.com> > On Oct 22, 2019, at 5:49 PM, Lance Andersen wrote: > >> >> On Oct 22, 2019, at 2:31 PM, Alan Bateman > wrote: >> >> On 22/10/2019 18:09, Lance Andersen wrote: >>> Hi all, >>> >>> This is a request to review the proposed fix for 8231766 which allows for Files.copy/move to correctly use the specified compression method when copying/moving a file within the same Zip file. >>> >>> The webrev can be found at: http://cr.openjdk.java.net/~lancea/8231766/webrev.00/ >> You should be able to avoid the copy loop by changing it to is.transfer(os). It might able be a bit more readable if you rename these two to "in" and "out" but that may be just me. >> > > Thank you Alan. I will try the Inputstream.transferTo(OutputStream) once I am able to build again on my mac :-( Thanks for the suggestion Alan. Webrev updated: http://cr.openjdk.java.net/~lancea/8231766/webrev.01/index.html While I have typically used 'is' for InputStream and ?os' for OutputStream myself, I renamed them to ?in?, ?out? per your suggestion. Mach5 jdk-tier1 through idk-tier3 still continue to pass Best Lance >> -Alan > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From martinrb at google.com Wed Oct 23 18:21:40 2019 From: martinrb at google.com (Martin Buchholz) Date: Wed, 23 Oct 2019 11:21:40 -0700 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> Message-ID: On Wed, Oct 23, 2019 at 2:52 AM Chris Hegarty wrote: > > + 1. I?ve taken a careful pass over the source changes, since they follow > a pattern of leaking `this` in the constructor and implementing a cleaning > action with be a lambda, both of which require careful consideration, but > the changes LGTM. > One possible safety idiom is to consistently have a static method returning a lambda (and don't pass in this!) , e.g. private static Runnable cleanerFor(FileDescriptor fd) { return () -> ...; } -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Oct 23 19:59:17 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 23 Oct 2019 20:59:17 +0100 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> Message-ID: <5b2b0f2f-94f8-67d3-b39d-9a049d5794cd@oracle.com> On 23/10/2019 19:21, Martin Buchholz wrote: > > : > > One possible safety idiom is to consistently have a static method > returning a lambda (and don't pass in this!) , e.g. > > private static Runnable cleanerFor(FileDescriptor fd) { return () -> > ...; } The cleaner is on the channel, the cleaning action is to close its file descriptor, so we need the reference to the channel. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Oct 23 20:30:54 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 23 Oct 2019 21:30:54 +0100 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> Message-ID: <05f1ca87-b884-2596-b8e6-96053a404fa9@oracle.com> On 23/10/2019 10:52, Chris Hegarty wrote: > : > The difference is whether the test will be selected to-be-run by jtreg or not. SkippedException is fine, but it seems highly unlikely that UnixOperatingSystemMXBean will every be implemented on Windows, so the test in its current state just doesn?t make sense to be run on non-Unix platforms. The test is cute, it is a pity that it uses a mechanism that is known to not be supported on Windows. We can add an @requires on the os.family so that it skips on Windows. Maybe someday, they will be an equally useful MXBean to get the handle as it is very useful for tests like this. -Alan From martinrb at google.com Wed Oct 23 20:38:46 2019 From: martinrb at google.com (Martin Buchholz) Date: Wed, 23 Oct 2019 13:38:46 -0700 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: <5b2b0f2f-94f8-67d3-b39d-9a049d5794cd@oracle.com> References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> <5b2b0f2f-94f8-67d3-b39d-9a049d5794cd@oracle.com> Message-ID: On Wed, Oct 23, 2019 at 12:59 PM Alan Bateman wrote: > On 23/10/2019 19:21, Martin Buchholz wrote: > > One possible safety idiom is to consistently have a static method > returning a lambda (and don't pass in this!) , e.g. > > private static Runnable cleanerFor(FileDescriptor fd) { return () -> ...; } > > The cleaner is on the channel, the cleaning action is to close its file > descriptor, so we need the reference to the channel. > You would call it like this: this.cleanable = CleanerFactory.cleaner().register(this, cleanerFor(fd)); It's just moving code around, and only slightly safer/more disciplined. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Oct 23 20:46:02 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 23 Oct 2019 21:46:02 +0100 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> <5b2b0f2f-94f8-67d3-b39d-9a049d5794cd@oracle.com> Message-ID: On 23/10/2019 21:38, Martin Buchholz wrote: > > : > > You would call it like this: > this.cleanable = CleanerFactory.cleaner().register(this, cleanerFor(fd)); > It's just moving code around, and only slightly safer/more disciplined. Okay, that might be a bit cleaner although it would be two static factories as the cleaning action is different for TCP vs. UDP. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Oct 23 20:49:31 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 23 Oct 2019 21:49:31 +0100 Subject: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file In-Reply-To: <43188D78-3D96-4426-9D4A-41DBC0C04A25@oracle.com> References: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> <09ad2667-9965-8277-6cf9-b76dd0560c75@oracle.com> <61FCC2BC-23F2-4006-8E8F-273F402068BF@oracle.com> <43188D78-3D96-4426-9D4A-41DBC0C04A25@oracle.com> Message-ID: <60320ab4-c9ac-dd2c-4626-2cdfa12a4068@oracle.com> On 23/10/2019 18:48, Lance Andersen wrote: > Thanks for the suggestion Alan. > > Webrev updated: > http://cr.openjdk.java.net/~lancea/8231766/webrev.01/index.html > > Thanks, looks much simpler. The test is big and I don't have time to go through that in detail right now. Maybe Christoph or someone else might have cycles to help. -Alan. From brian.burkhalter at oracle.com Thu Oct 24 00:03:24 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 23 Oct 2019 17:03:24 -0700 Subject: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file In-Reply-To: <60320ab4-c9ac-dd2c-4626-2cdfa12a4068@oracle.com> References: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> <09ad2667-9965-8277-6cf9-b76dd0560c75@oracle.com> <61FCC2BC-23F2-4006-8E8F-273F402068BF@oracle.com> <43188D78-3D96-4426-9D4A-41DBC0C04A25@oracle.com> <60320ab4-c9ac-dd2c-4626-2cdfa12a4068@oracle.com> Message-ID: <2260FF4B-7A1D-4551-B9BA-520FB3CBBE92@oracle.com> Hi Lance, Please see comments on the test below listed by line numbers. Brian > On Oct 23, 2019, at 1:49 PM, Alan Bateman wrote: > > On 23/10/2019 18:48, Lance Andersen wrote: >> Thanks for the suggestion Alan. >> >> Webrev updated: http://cr.openjdk.java.net/~lancea/8231766/webrev.01/index.html >> >> > Thanks, looks much simpler. The test is big and I don't have time to go through that in detail right now. Maybe Christoph or someone else might have cycles to help. 114 How can verify() return true if compression != expectedCompression? It looks like at 111 the contents of e0 are copied with e00?s name but retain e0?s compression method. Probably I am missing something. 148 Similar question as for line 114. 251 Similar question as for line 114 286 Similar question as for line 114 336 Could TestNG ?expectedExceptions? parameter of the @Test annotation be used instead of catching the FAE (and put line 352 in a finally block)? 363-371 Could Files.createTempFile() be used instead of having this method? 447 Line commented out 110, 144, 181, 247, 280, 347, 399 Indentation -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Thu Oct 24 10:26:33 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 24 Oct 2019 11:26:33 +0100 Subject: RFR[8231570]: (dc) Clarify implicit bind behavior of DatagramChannel In-Reply-To: <0ac5b7ec-e36d-4e4a-8639-d53d2ed73711@oracle.com> References: <0ac5b7ec-e36d-4e4a-8639-d53d2ed73711@oracle.com> Message-ID: > On 16 Oct 2019, at 19:20, Patrick Concannon wrote: > > [Resending with nio-dev in cc] > > Hi, > > > Could someone please review my fix and CSR for issue JDK-8231570 '(dc) Clarify implicit bind behavior of DatagramChannel' ? > > The fix clarifies the security checks for DatagramChannel connect, send, and receive methods. > > > webrev: http://cr.openjdk.java.net/~pconcannon/8231570/webrevs/webrev.02/ LGTM. -Chris. From lance.andersen at oracle.com Thu Oct 24 12:21:52 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 24 Oct 2019 08:21:52 -0400 Subject: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file In-Reply-To: <2260FF4B-7A1D-4551-B9BA-520FB3CBBE92@oracle.com> References: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> <09ad2667-9965-8277-6cf9-b76dd0560c75@oracle.com> <61FCC2BC-23F2-4006-8E8F-273F402068BF@oracle.com> <43188D78-3D96-4426-9D4A-41DBC0C04A25@oracle.com> <60320ab4-c9ac-dd2c-4626-2cdfa12a4068@oracle.com> <2260FF4B-7A1D-4551-B9BA-520FB3CBBE92@oracle.com> Message-ID: <984EA15C-4D6E-4346-B7F5-CC0ED62B764E@oracle.com> Hi Brian, Thank you for the review. > On Oct 23, 2019, at 8:03 PM, Brian Burkhalter wrote: > > Hi Lance, > > Please see comments on the test below listed by line numbers. > > Brian > >> On Oct 23, 2019, at 1:49 PM, Alan Bateman > wrote: >> >> On 23/10/2019 18:48, Lance Andersen wrote: >>> Thanks for the suggestion Alan. >>> >>> Webrev updated: http://cr.openjdk.java.net/~lancea/8231766/webrev.01/index.html >>> >>> >> Thanks, looks much simpler. The test is big and I don't have time to go through that in detail right now. Maybe Christoph or someone else might have cycles to help. > > 114 > How can verify() return true if compression != expectedCompression? It looks like at 111 the contents of e0 are copied with e00?s name but retain e0?s compression method. Probably I am missing something. The initial zip is created just adding e0 and e1 and the compression used for adding these files is determined via the setting of the no compression property (or default value) when the Zip FS is created/accessed. The expected compression for e00 will match what the targetCompression is set to when the ZipFS is being created to update the file. > > 148 > Similar question as for line 114. > > 251 > Similar question as for line 114 > > 286 > Similar question as for line 114 > > 336 > Could TestNG ?expectedExceptions? parameter of the @Test annotation be used instead of catching the FAE (and put line 352 in a finally block)? > Past reviews have suggested to move away from expectedException annotation element and use assertThrows() which the reason I went this way. > 363-371 > Could Files.createTempFile() be used instead of having this method? In this case, I am not creating the file, just generating a path name > > 447 > Line commented out > Thank you. I will remove(thought I had) > 110, 144, 181, 247, 280, 347, 399 > Indentation I ran the intellij reformat code command, but can do it again. Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From Alan.Bateman at oracle.com Thu Oct 24 14:20:57 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 24 Oct 2019 15:20:57 +0100 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> <5b2b0f2f-94f8-67d3-b39d-9a049d5794cd@oracle.com> Message-ID: On 23/10/2019 21:46, Alan Bateman wrote: > On 23/10/2019 21:38, Martin Buchholz wrote: >> >> : >> >> You would call it like this: >> this.cleanable = CleanerFactory.cleaner().register(this, cleanerFor(fd)); >> It's just moving code around, and only slightly safer/more disciplined. > Okay, that might be a bit cleaner although it would be two static > factories as the cleaning action is different for TCP vs. UDP. I've updated the proposed change to invoke the register method in the constructor. Also added the @requires to the test so that it is clearer that it doesn't run on Windows. I've left the check for UnixOperatingSystemMXBean in the test as I can't be sure that it can be obtained on some of exotic ports. The new webrev is here: http://cr.openjdk.java.net/~alanb/8212132/webrev.02/index.html -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Thu Oct 24 15:26:54 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 24 Oct 2019 16:26:54 +0100 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> <5b2b0f2f-94f8-67d3-b39d-9a049d5794cd@oracle.com> Message-ID: Hi Alan, On 24/10/2019 15:20, Alan Bateman wrote: > I've updated the proposed change to invoke the register method in the > constructor. Also added the @requires to the test so that it is clearer > that it doesn't run on Windows. I've left the check for > UnixOperatingSystemMXBean in the test as I can't be sure that it can be > obtained on some of exotic ports. The new webrev is here: > > http://cr.openjdk.java.net/~alanb/8212132/webrev.02/index.html > > -Alan. 73 // Used to make native read and write calls 74 private static NativeDispatcher nd = new DatagramDispatcher(); Is there any reason why `nd` is not final? nit: in NioSocketImpl:closerFor I'd have a slight preference for capturing both `stream` and `fd` in the lambda rather than having two lambdas. Otherwise looks good to me. best regards, -- daniel From Alan.Bateman at oracle.com Thu Oct 24 15:32:12 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 24 Oct 2019 16:32:12 +0100 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> <5b2b0f2f-94f8-67d3-b39d-9a049d5794cd@oracle.com> Message-ID: <17cacace-5ba9-9e47-34a9-7cf8c4b880ac@oracle.com> On 24/10/2019 16:26, Daniel Fuchs wrote: > > ? 73???? // Used to make native read and write calls > ? 74???? private static NativeDispatcher nd = new DatagramDispatcher(); > > Is there any reason why `nd` is not final? Look like a long standing oversight, it should be final. I can fix that as part of this change. -Alan. From brian.burkhalter at oracle.com Thu Oct 24 17:49:35 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 24 Oct 2019 10:49:35 -0700 Subject: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file In-Reply-To: <984EA15C-4D6E-4346-B7F5-CC0ED62B764E@oracle.com> References: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> <09ad2667-9965-8277-6cf9-b76dd0560c75@oracle.com> <61FCC2BC-23F2-4006-8E8F-273F402068BF@oracle.com> <43188D78-3D96-4426-9D4A-41DBC0C04A25@oracle.com> <60320ab4-c9ac-dd2c-4626-2cdfa12a4068@oracle.com> <2260FF4B-7A1D-4551-B9BA-520FB3CBBE92@oracle.com> <984EA15C-4D6E-4346-B7F5-CC0ED62B764E@oracle.com> Message-ID: Hi Lance, OK that all sounds good. No need to update the webrev. Thanks, Brian > On Oct 24, 2019, at 5:21 AM, Lance Andersen wrote: > > Thank you for the review. >> On Oct 23, 2019, at 8:03 PM, Brian Burkhalter > wrote: >> >> Hi Lance, >> >> Please see comments on the test below listed by line numbers. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Thu Oct 24 18:05:26 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 24 Oct 2019 14:05:26 -0400 Subject: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file In-Reply-To: References: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> <09ad2667-9965-8277-6cf9-b76dd0560c75@oracle.com> <61FCC2BC-23F2-4006-8E8F-273F402068BF@oracle.com> <43188D78-3D96-4426-9D4A-41DBC0C04A25@oracle.com> <60320ab4-c9ac-dd2c-4626-2cdfa12a4068@oracle.com> <2260FF4B-7A1D-4551-B9BA-520FB3CBBE92@oracle.com> <984EA15C-4D6E-4346-B7F5-CC0ED62B764E@oracle.com> Message-ID: <51C4C7BD-E42A-4DD2-B59A-86EDD0613445@oracle.com> Hi Brian, Thank you again for your time and review. Best Lance > On Oct 24, 2019, at 1:49 PM, Brian Burkhalter wrote: > > Hi Lance, > > OK that all sounds good. No need to update the webrev. > > Thanks, > > Brian > >> On Oct 24, 2019, at 5:21 AM, Lance Andersen > wrote: >> >> Thank you for the review. >>> On Oct 23, 2019, at 8:03 PM, Brian Burkhalter > wrote: >>> >>> Hi Lance, >>> >>> Please see comments on the test below listed by line numbers. >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From christoph.langer at sap.com Fri Oct 25 08:00:23 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 25 Oct 2019 08:00:23 +0000 Subject: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file In-Reply-To: References: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> <09ad2667-9965-8277-6cf9-b76dd0560c75@oracle.com> <61FCC2BC-23F2-4006-8E8F-273F402068BF@oracle.com> <43188D78-3D96-4426-9D4A-41DBC0C04A25@oracle.com> <60320ab4-c9ac-dd2c-4626-2cdfa12a4068@oracle.com> <2260FF4B-7A1D-4551-B9BA-520FB3CBBE92@oracle.com> <984EA15C-4D6E-4346-B7F5-CC0ED62B764E@oracle.com> Message-ID: Hi Lance, I?ve had a look as well now. Thanks for doing this fix and providing extensive testing ?? The change in ZipFileSystem looks correct. I have a few remarks for the test(s): UpdateEntryTest: testReplaceStoredEntry maybe add some comment that we expect the updated entry to have the default compression method of the zip file system (which is DEFLATED for this test case) Maybe you can fix some minor thing that was already in the test, while touching it: line 121: We don't use rc, so either remove it or add @SuppressWarnings("unused") if you think it's important to have for debugging purposes. I'd rather vote for removing :) lines 163ff: Maybe you can initialize the properties map in one line: Map.of("noCompression", e1.method != ZipEntry.DEFLATED) and use this in place in line 177 The only drawback is that you wouldn't have a check whether e1.method is in (ZipEntry.DEFLATED, ZipEntry.STORED) but is that needed? 10 fewer lines of code to read/understand ;-) CopyMoveTests line 110: use "expectedCompression == ZipEntry.STORED" instead of targetCompression (remove line 102). I think that makes the code more readable This also applies to line 144, 181, 247 and 280/282 line 329: grammar: Validate that a FileAlreadyExistsException is thrown line 338: formatMap printout: I guess this is for debugging and it does only exist in this one place although something like that could be as useful everywhere. I suggest to either remove it here or insert it in each test method, maybe hidden behind some debug switch line 387: comment should be: Create a Zip file containing the given entries, using Zip File System and the specified properties map Thanks Christoph From: nio-dev On Behalf Of Brian Burkhalter Sent: Donnerstag, 24. Oktober 2019 19:50 To: Lance Andersen Cc: nio-dev Subject: Re: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file Hi Lance, OK that all sounds good. No need to update the webrev. Thanks, Brian On Oct 24, 2019, at 5:21 AM, Lance Andersen > wrote: Thank you for the review. On Oct 23, 2019, at 8:03 PM, Brian Burkhalter > wrote: Hi Lance, Please see comments on the test below listed by line numbers. -------------- next part -------------- An HTML attachment was scrubbed... URL: From akashche at redhat.com Fri Oct 25 11:31:37 2019 From: akashche at redhat.com (Alex Kashchenko) Date: Fri, 25 Oct 2019 12:31:37 +0100 Subject: RFC: 8216472: Stack overflow followed by crash In-Reply-To: References: <07b40ab0-8e40-280b-58e4-6f237e1e0f6f@redhat.com> Message-ID: Hi Alan, On 10/20/2019 09:12 AM, Alan Bateman wrote: > On 08/10/2019 12:36, Alex Kashchenko wrote: >> Hi, >> >> At Red Hat, we've got a windows-only crash in nio.dll with a JBoss >> application,? that appeared to be JDK-8216472 [1]. >> >> After some investigation it was found, that >> Java_sun_nio_ch_WindowsSelectorImpl_00024SubSelector_poll0 function >> [2] allocates more than 50Kb on stack - because FD_SETSIZE is set to >> 1024 [3], each fd_set structure takes 8200 bytes and 6 fd_set >> structures are placed on stack [4][5]. If this function is called from >> a deep recursive java call, it can cause stack overflow and JVM >> process is either aborted or killed by OS. >> >> One of the solutions to this may be to set hotspot StackShadowPages >> parameter to 14. Currently it is 7 on windows and 20 on linux. >> >> I'd like to solicit comments and suggestions about another solution, >> that changes this native call in NIO moving fd_sets to heap: >> >> http://cr.openjdk.java.net/~akasko/jdk/8216472/webrev.00/ >> >> PollOverflow reproducer there causes JVM abort (or silent kill) >> without the patch. >> >> We tested this patch internally and haven't found any regressions; >> tests included JCK and SPECjbb, though we haven't run any benchmarks >> specifically oriented on networking performance. > I finally got time to look at this a bit more. > > The Windows select based Selector does need some TLC. This issue with > the FD_SET structures on the stack has been there since JDK 1.4 and I'm > surprised it has been reported before this issue. > > The underlying issue is that this Selector wants poll and ends up > translating a poll array to a set of FD_SET structures on each select. > This shouldn't be needed, it should instead maintain the FD_SET in the > java implementation so that the native method is a call to select, > nothing more. Changing this would require a lot of work on the Selector > (esp. the exceptfds) and is something we should do at some point. > > For the current issue I think we should do the following: > > 1. If select fails, throw the IOException and return IOS_THROWN. I don't > think any of this code has been needed for many releases as the async > close of a channel registered with a Selector will consistently delay > closing the SOCKET until it is flushed from all registered Selectors. > That should eliminate 3 of the 6 FD_SET structures. > > 2. Allocate the memory for the 3 FD_SET structures in the SubSelector. > That is, manage the memory from the Java code, not the native method. > That eliminates the 3 * malloc/free per select that is in the proposed > patch. The memory can be freed when the sub selector thread terminates. > > My guess is that the changes will go through several iterations and > happy to provide feedback on interim revisions as needed. Thanks for your comments! Updated the patch including them: http://cr.openjdk.java.net/~akasko/jdk/8216472/webrev.01/ -- -Alex From martinrb at google.com Fri Oct 25 16:46:16 2019 From: martinrb at google.com (Martin Buchholz) Date: Fri, 25 Oct 2019 09:46:16 -0700 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> <5b2b0f2f-94f8-67d3-b39d-9a049d5794cd@oracle.com> Message-ID: The Cleaner code looks good to me. Of course, ensuring the concurrency correctness of this code is a much harder problem. I was especially frightened when I saw 1259 // wait for any read/write operations to complete before trying to close 1260 readLock.lock(); 1261 readLock.unlock(); 1262 writeLock.lock(); 1263 writeLock.unlock(); 1264 synchronized (stateLock) { I have come to think of GC finalization as yet another mutual exclusion problem - between the object and the patient GC vulture waiting to collect it and call its cleaner. On Thu, Oct 24, 2019 at 7:23 AM Alan Bateman wrote: > > > On 23/10/2019 21:46, Alan Bateman wrote: > > On 23/10/2019 21:38, Martin Buchholz wrote: > > > : > > You would call it like this: > this.cleanable = CleanerFactory.cleaner().register(this, cleanerFor(fd)); > It's just moving code around, and only slightly safer/more disciplined. > > Okay, that might be a bit cleaner although it would be two static > factories as the cleaning action is different for TCP vs. UDP. > > I've updated the proposed change to invoke the register method in the > constructor. Also added the @requires to the test so that it is clearer > that it doesn't run on Windows. I've left the check for > UnixOperatingSystemMXBean in the test as I can't be sure that it can be > obtained on some of exotic ports. The new webrev is here: > > http://cr.openjdk.java.net/~alanb/8212132/webrev.02/index.html > > -Alan. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Fri Oct 25 17:48:40 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 25 Oct 2019 13:48:40 -0400 Subject: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file In-Reply-To: References: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> <09ad2667-9965-8277-6cf9-b76dd0560c75@oracle.com> <61FCC2BC-23F2-4006-8E8F-273F402068BF@oracle.com> <43188D78-3D96-4426-9D4A-41DBC0C04A25@oracle.com> <60320ab4-c9ac-dd2c-4626-2cdfa12a4068@oracle.com> <2260FF4B-7A1D-4551-B9BA-520FB3CBBE92@oracle.com> <984EA15C-4D6E-4346-B7F5-CC0ED62B764E@oracle.com> Message-ID: <4F819975-F191-4419-9A53-48F30CA0C3F9@oracle.com> Hi Christoph, Thank you for the review, please see below. > On Oct 25, 2019, at 4:00 AM, Langer, Christoph wrote: > > Hi Lance, > > I?ve had a look as well now. Thanks for doing this fix and providing extensive testing ?? :-) > > The change in ZipFileSystem looks correct. thank you > I have a few remarks for the test(s): > > UpdateEntryTest: > > testReplaceStoredEntry > > maybe add some comment that we expect the updated entry to have the default compression method of the zip file system (which is DEFLATED for this test case) done > > Maybe you can fix some minor thing that was already in the test, while touching it: > line 121: We don't use rc, so either remove it or add @SuppressWarnings("unused") if you think it's important to have for debugging purposes. I'd rather vote for removing :) OK, done. I added this originally because Intellij complained via the inspect option and then complained after the change that it was an unused variable. so I just removed it per your request. > > lines 163ff: Maybe you can initialize the properties map in one line: Map.of("noCompression", e1.method != ZipEntry.DEFLATED) and use this in place in line 177 > The only drawback is that you wouldn't have a check whether e1.method is in (ZipEntry.DEFLATED, ZipEntry.STORED) but is that needed? 10 fewer lines of code to read/understand ;-) OK done in a similar way to your suggestion. > > > CopyMoveTests > > line 110: use "expectedCompression == ZipEntry.STORED" instead of targetCompression (remove line 102). I think that makes the code more readable > This also applies to line 144, 181, 247 and 280/282 OK, while I think this is a personal style choice, I made the suggested change. > > line 329: grammar: Validate that a FileAlreadyExistsException is thrown done > > line 338: formatMap printout: I guess this is for debugging and it does only exist in this one place although something like that could be as useful everywhere. I suggest to either remove it here or insert it in each test method, maybe hidden behind some debug switch Added a DEBUG flag and also updated the System.out.printf in the createZip method > > line 387: comment should be: > Create a Zip file containing the given entries, using Zip File System and the specified properties map Updated the comment The updated webrev can be found at: http://cr.openjdk.java.net/~lancea/8231766/webrev.02/index.html Currently re-running mach5 jdk-tier1 through jdk-tier3 Best Lance > > > Thanks > Christoph > > From: nio-dev On Behalf Of Brian Burkhalter > Sent: Donnerstag, 24. Oktober 2019 19:50 > To: Lance Andersen > Cc: nio-dev > Subject: Re: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file > > Hi Lance, > > OK that all sounds good. No need to update the webrev. > > Thanks, > > Brian > > > On Oct 24, 2019, at 5:21 AM, Lance Andersen > wrote: > > Thank you for the review. > On Oct 23, 2019, at 8:03 PM, Brian Burkhalter > wrote: > > Hi Lance, > > Please see comments on the test below listed by line numbers. Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From Alan.Bateman at oracle.com Sat Oct 26 16:40:56 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 26 Oct 2019 17:40:56 +0100 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> <5b2b0f2f-94f8-67d3-b39d-9a049d5794cd@oracle.com> Message-ID: <1a851e42-1579-bd88-dfc7-6f362a3964f5@oracle.com> On 25/10/2019 17:46, Martin Buchholz wrote: > The Cleaner code looks good to me. Thanks. > > Of course, ensuring the concurrency correctness of this code is a much > harder problem.? I was especially frightened when I saw > > 1259 ? ? ? ? // wait for any read/write operations to complete before > trying to close > 1260 ? ? ? ? readLock.lock(); > 1261 ? ? ? ? readLock.unlock(); > 1262 ? ? ? ? writeLock.lock(); > 1263 ? ? ? ? writeLock.unlock(); > 1264 ? ? ? ? synchronized (stateLock) { > > I have come to think of GC finalization as yet another mutual > exclusion problem - between the object and the patient GC vulture > waiting to collect it and call its cleaner. We haven't changed anything here. If someone is calling close then it has to coordinate to any read/write or channel channel operations that are in flight at the time. The above is the case where the channel is configured non-blocking or any threads doing read/write won't block in syscalls. It might look unusual on first sight but allows read/write on channels configured non-blocking to use a single lock. Anyway, if the explicit close invokes the cleaner when we are guaranteed (by the cleaner API) that the cleaning action will never run a second time. If we didn't have that guaranteed then we'll need something like the FileDescriptorCloser. -Alan From Alan.Bateman at oracle.com Sat Oct 26 17:13:37 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 26 Oct 2019 18:13:37 +0100 Subject: RFC: 8216472: Stack overflow followed by crash In-Reply-To: References: <07b40ab0-8e40-280b-58e4-6f237e1e0f6f@redhat.com> Message-ID: <988b9d89-1561-1ebf-4472-d208359da465@oracle.com> On 25/10/2019 12:31, Alex Kashchenko wrote: > > Thanks for your comments! Updated the patch including them: > > http://cr.openjdk.java.net/~akasko/jdk/8216472/webrev.01/ > Thanks for the update, I just quickly looked at it and this is much better. I suspect the native method will need to ignore WSANOTSOCK and return 0. I've really busy just now and I will get back to you in the next few days with more on this. -Alan From Alan.Bateman at oracle.com Sun Oct 27 14:16:51 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 27 Oct 2019 14:16:51 +0000 Subject: 8232673: (dc) DatagramChannel socket adaptor issues Message-ID: DatagramChannel::socket returns a socket adaptor, essentially a DatagramSocket view on the channel. The original motivation for the socket method was binding and socket options but it also created the potential for migration or interop with code using the older DatagramSocket API. The adaptor has been on my radar for sometime as it implements a blocking receive method that relies on the channel's blockingLock. This locking is problematic for several reasons, including preventing a send to proceed when a thread is blocked in receive. The main change in the patch proposed here is to re-implememt the adaptor's send and receive methods so they can execute concurrently and don't block while holding a built-in monitor. I've also changed these methods to use the buffer cache mechanism and avoid creating a ByteBuffer per I/O operation. The new receive implementation also works around two flaws in the legacy DatagramPacket API/implementation (discussed in JDK-8232817). While in the area, a few other smaller issues are fixed too: 1. A connected DatagramChannel needs to disconnect before connecting it to another address. DatagramSocket doesn't require this. The adaptor is changed to allow it connect without throwing an exception. The connect methods are also fixed to throw IllegalArgumentException rather than NullPointerException when invoked with a null address. 2. The adaptor's getLocalSocketAddress method returns the local address when channel was closed. This is an inconsistency in DatagramSocket's API that was missed in the adaptor. I've expanded the test coverage. The new AdaptorGetters test attempts to exercise all the getter methods when the channel is in different states so gives us confidence that there aren't further issues with these methods. The webrev with the proposed changes is here: ??? http://cr.openjdk.java.net/~alanb/8232673/webrev/index.html -Alan From martinrb at google.com Sun Oct 27 17:49:10 2019 From: martinrb at google.com (Martin Buchholz) Date: Sun, 27 Oct 2019 10:49:10 -0700 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: <1a851e42-1579-bd88-dfc7-6f362a3964f5@oracle.com> References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> <5b2b0f2f-94f8-67d3-b39d-9a049d5794cd@oracle.com> <1a851e42-1579-bd88-dfc7-6f362a3964f5@oracle.com> Message-ID: On Sat, Oct 26, 2019 at 9:41 AM Alan Bateman wrote: > > > On 25/10/2019 17:46, Martin Buchholz wrote: > > The Cleaner code looks good to me. > Thanks. > > > > > Of course, ensuring the concurrency correctness of this code is a much > > harder problem. I was especially frightened when I saw > > > > 1259 // wait for any read/write operations to complete before > > trying to close > > 1260 readLock.lock(); > > 1261 readLock.unlock(); > > 1262 writeLock.lock(); > > 1263 writeLock.unlock(); > > 1264 synchronized (stateLock) { > > > > I have come to think of GC finalization as yet another mutual > > exclusion problem - between the object and the patient GC vulture > > waiting to collect it and call its cleaner. > We haven't changed anything here. Right. I'm just an interested bystander musing about the complexity of concurrency and finalization. (No need to respond.) Because I think of finalization as a mutual exclusion problem with the GC, I am thinking "this class is dealing with 4 different locks" (also, there's no read-write-lock as one would expect from seeing a readLock and a writeLock. But I have no better name (hmmm maybe readerLock and writerLock?)) Wearing my finalization/reachability hat, it looks to me like technically a method like public int write(ByteBuffer buf) needs a reachabilityFence to prevent fd from being closed while in use, but in practice no misbehavior will ever be provoked, in part because hotspot refuses to hoist final fields into locals. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon Oct 28 06:47:25 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 28 Oct 2019 06:47:25 +0000 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> <5b2b0f2f-94f8-67d3-b39d-9a049d5794cd@oracle.com> <1a851e42-1579-bd88-dfc7-6f362a3964f5@oracle.com> Message-ID: <4dbe7de8-3cf0-891f-998d-f2e33597f532@oracle.com> On 27/10/2019 17:49, Martin Buchholz wrote: > : > > (also, there's no read-write-lock as one would expect from seeing a > readLock and a writeLock.? But I have no better name (hmmm? ?maybe > readerLock and writerLock?)) They have been named "readLock" and "writeLock" since 1.4 but you may have a point. > > Wearing my finalization/reachability hat, it looks to me like > technically a method like > public int write(ByteBuffer buf) > needs a reachabilityFence to prevent fd from being closed while in > use, but in practice no misbehavior will ever be provoked, in part > because hotspot refuses to hoist final fields into locals. The channel implementation has a strong reference to the FileDescriptor so I think you are suggesting that, at least in theory, the channel (fd, locks, ...) could be GC'ed while a thread is a read/write/send/receive method. I don't see how the compiler could look through the endRead/endWrite methods but I'll study it further. Also if we had issues here then I would expect the IBM folks to speak up as they tend to find them with their J9 VM. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From vkempik at azul.com Mon Oct 28 09:20:22 2019 From: vkempik at azul.com (Vladimir Kempik) Date: Mon, 28 Oct 2019 09:20:22 +0000 Subject: RFR(8u): 8229872: (fs) Increase buffer size used with getmntent In-Reply-To: <7B2B9DF5-F845-42FC-949C-102B72282975@amazon.com> References: <5CCC5140-4C75-45CD-A3A2-CC0237DD7AD9@azul.com> <7B2B9DF5-F845-42FC-949C-102B72282975@amazon.com> Message-ID: Hello Can anyone please take a look at this patch? The difference between 11 and 8 is pretty simple: move one method from superclass to subclass to prevent it being compiled on every Unix system(it can?t be compiled on solaris). This code is only used on Linux so far. Thanks, Vladimir 21 ???. 2019 ?., ? 19:23, Hohensee, Paul > ???????(?): +jdk8u-dev Paul From: nio-dev > on behalf of Vladimir Kempik > Date: Monday, October 21, 2019 at 2:47 AM To: "nio-dev at openjdk.java.net" > Subject: RFR(8u): 8229872: (fs) Increase buffer size used with getmntent Hello Please review backport of https://bugs.openjdk.java.net/browse/JDK-8229872 to 8u: http://cr.openjdk.java.net/~vkempik/8229872/ojdk8/webrev.00/ Difference between 14/13/11 fix: getline is missing on solaris10, so getlinelen method was moved from UnixNativeDispatcher to LinuxNativeDispatcher. Original fix: https://hg.openjdk.java.net/jdk/jdk/rev/2d40e6a7ce8e Such fix was already included into azul?s october psu for openjdk8 and passed all release cycle tests. Thanks, Vladimir -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Mon Oct 28 09:21:21 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 28 Oct 2019 09:21:21 +0000 Subject: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file In-Reply-To: <4F819975-F191-4419-9A53-48F30CA0C3F9@oracle.com> References: <02D81AE6-50DF-4876-A35E-7E27EA0A6106@oracle.com> <09ad2667-9965-8277-6cf9-b76dd0560c75@oracle.com> <61FCC2BC-23F2-4006-8E8F-273F402068BF@oracle.com> <43188D78-3D96-4426-9D4A-41DBC0C04A25@oracle.com> <60320ab4-c9ac-dd2c-4626-2cdfa12a4068@oracle.com> <2260FF4B-7A1D-4551-B9BA-520FB3CBBE92@oracle.com> <984EA15C-4D6E-4346-B7F5-CC0ED62B764E@oracle.com> <4F819975-F191-4419-9A53-48F30CA0C3F9@oracle.com> Message-ID: Hi Lance, thanks for doing the updates. Looks fine, except that you should have a look at the comment in line 389/390 of CopyMoveTests.java again ? this seems to be a little bit edgy still ?? But no need for updated webrev? Best regards Christoph From: Lance Andersen Sent: Freitag, 25. Oktober 2019 19:49 To: Langer, Christoph Cc: Brian Burkhalter ; nio-dev Subject: Re: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file Hi Christoph, Thank you for the review, please see below. On Oct 25, 2019, at 4:00 AM, Langer, Christoph > wrote: Hi Lance, I?ve had a look as well now. Thanks for doing this fix and providing extensive testing ?? :-) The change in ZipFileSystem looks correct. thank you I have a few remarks for the test(s): UpdateEntryTest: testReplaceStoredEntry maybe add some comment that we expect the updated entry to have the default compression method of the zip file system (which is DEFLATED for this test case) done Maybe you can fix some minor thing that was already in the test, while touching it: line 121: We don't use rc, so either remove it or add @SuppressWarnings("unused") if you think it's important to have for debugging purposes. I'd rather vote for removing :) OK, done. I added this originally because Intellij complained via the inspect option and then complained after the change that it was an unused variable. so I just removed it per your request. lines 163ff: Maybe you can initialize the properties map in one line: Map.of("noCompression", e1.method != ZipEntry.DEFLATED) and use this in place in line 177 The only drawback is that you wouldn't have a check whether e1.method is in (ZipEntry.DEFLATED, ZipEntry.STORED) but is that needed? 10 fewer lines of code to read/understand ;-) OK done in a similar way to your suggestion. CopyMoveTests line 110: use "expectedCompression == ZipEntry.STORED" instead of targetCompression (remove line 102). I think that makes the code more readable This also applies to line 144, 181, 247 and 280/282 OK, while I think this is a personal style choice, I made the suggested change. line 329: grammar: Validate that a FileAlreadyExistsException is thrown done line 338: formatMap printout: I guess this is for debugging and it does only exist in this one place although something like that could be as useful everywhere. I suggest to either remove it here or insert it in each test method, maybe hidden behind some debug switch Added a DEBUG flag and also updated the System.out.printf in the createZip method line 387: comment should be: Create a Zip file containing the given entries, using Zip File System and the specified properties map Updated the comment The updated webrev can be found at: http://cr.openjdk.java.net/~lancea/8231766/webrev.02/index.html Currently re-running mach5 jdk-tier1 through jdk-tier3 Best Lance Thanks Christoph From: nio-dev > On Behalf Of Brian Burkhalter Sent: Donnerstag, 24. Oktober 2019 19:50 To: Lance Andersen > Cc: nio-dev > Subject: Re: Resend RFR 8231766: Files.copy/move do not honor requested compression method when copying/moving within the same zip file Hi Lance, OK that all sounds good. No need to update the webrev. Thanks, Brian On Oct 24, 2019, at 5:21 AM, Lance Andersen > wrote: Thank you for the review. On Oct 23, 2019, at 8:03 PM, Brian Burkhalter > wrote: Hi Lance, Please see comments on the test below listed by line numbers. [cid:image001.gif at 01D58D79.70CB2B60] Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 658 bytes Desc: image001.gif URL: From martinrb at google.com Mon Oct 28 14:14:46 2019 From: martinrb at google.com (Martin Buchholz) Date: Mon, 28 Oct 2019 07:14:46 -0700 Subject: 8212132: (dc) Remove DatagramChannelImpl finalize method In-Reply-To: <4dbe7de8-3cf0-891f-998d-f2e33597f532@oracle.com> References: <3304ea70-0c19-7a55-7fe4-c9d6944346de@oracle.com> <5b2b0f2f-94f8-67d3-b39d-9a049d5794cd@oracle.com> <1a851e42-1579-bd88-dfc7-6f362a3964f5@oracle.com> <4dbe7de8-3cf0-891f-998d-f2e33597f532@oracle.com> Message-ID: On Sun, Oct 27, 2019 at 11:47 PM Alan Bateman wrote: > On 27/10/2019 17:49, Martin Buchholz wrote: > > > Wearing my finalization/reachability hat, it looks to me like technically > a method like > public int write(ByteBuffer buf) > needs a reachabilityFence to prevent fd from being closed while in use, > but in practice no misbehavior will ever be provoked, in part because > hotspot refuses to hoist final fields into locals. > > The channel implementation has a strong reference to the FileDescriptor so > I think you are suggesting that, at least in theory, the channel (fd, > locks, ...) could be GC'ed while a thread is a read/write/send/receive > method. I don't see how the compiler could look through the > endRead/endWrite methods but I'll study it further. Also if we had issues > here then I would expect the IBM folks to speak up as they tend to find > them with their J9 VM. > I would be surprised if you could ever repro a failure due to finalization here. OTOH it's reasonable to try to bullet-proof methods by wrapping all public methods that use fd in a try { ... } finally { reachabilityFence(this); } In j.u.concurrent, we copy final lock fields to locals because hotspot won't final ReentrantLock stateLock = this.stateLock; synchronized (stateLock) ... but that refactoring will make it more likely to provoke early finalization -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Tue Oct 29 15:18:35 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 29 Oct 2019 15:18:35 +0000 Subject: 8232673: (dc) DatagramChannel socket adaptor issues In-Reply-To: References: Message-ID: Hi Alan, Thanks for this patch that fixes and brings into the mainline many of the small issues we noticed while working on JDK-8230211. I looked at the patch and I think it is good. A couple of comments still: DatagramChannelImpl: 452 assert sender != null; pedantically that should be: assert sender != null || !isOpen(); Indeed if the channel is closed at this point the finally clause at lines: 456 endRead(blocking, n > 0); 457 assert IOStatus.check(n); will either erase the assertion error by replacing it with an IOException thrown by endRead() at line 456 or will replace it by another assertion error at line 457. In any case it should be legal for `sender` to be null at line 452 if the channel has been asynchronously closed or closed by interrupt. The same would apply for asserts at lines: 566 assert sender != null; 601 assert sender != null; I also believe a similar-ish (but maybe more relevant) kind of issue would arise for: 491 assert n >= 0; at this point - and as per line 484 while (IOStatus.okayToRetry(n) && isOpen()) { then n could well be < 0 if the channel is closed. I suspect the assertion at line 491 should therefore be preceded with: if (n < 0) return n; either after line 487 or after line 490. best regards, -- daniel On 27/10/2019 14:16, Alan Bateman wrote: > DatagramChannel::socket returns a socket adaptor, essentially a > DatagramSocket view on the channel. The original motivation for the > socket method was binding and socket options but it also created the > potential for migration or interop with code using the older > DatagramSocket API. The adaptor has been on my radar for sometime as it > implements a blocking receive method that relies on the channel's > blockingLock. This locking is problematic for several reasons, including > preventing a send to proceed when a thread is blocked in receive. The > main change in the patch proposed here is to re-implememt the adaptor's > send and receive methods so they can execute concurrently and don't > block while holding a built-in monitor. I've also changed these methods > to use the buffer cache mechanism and avoid creating a ByteBuffer per > I/O operation. The new receive implementation also works around two > flaws in the legacy DatagramPacket API/implementation (discussed in > JDK-8232817). > > While in the area, a few other smaller issues are fixed too: > > 1. A connected DatagramChannel needs to disconnect before connecting it > to another address. DatagramSocket doesn't require this. The adaptor is > changed to allow it connect without throwing an exception. The connect > methods are also fixed to throw IllegalArgumentException rather than > NullPointerException when invoked with a null address. > > 2. The adaptor's getLocalSocketAddress method returns the local address > when channel was closed. This is an inconsistency in DatagramSocket's > API that was missed in the adaptor. > > I've expanded the test coverage. The new AdaptorGetters test attempts to > exercise all the getter methods when the channel is in different states > so gives us confidence that there aren't further issues with these methods. > > The webrev with the proposed changes is here: > ??? http://cr.openjdk.java.net/~alanb/8232673/webrev/index.html > > -Alan From Alan.Bateman at oracle.com Tue Oct 29 15:38:30 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 29 Oct 2019 15:38:30 +0000 Subject: 8232673: (dc) DatagramChannel socket adaptor issues In-Reply-To: References: Message-ID: <7597094f-e024-7d41-89a8-45defb20e6ed@oracle.com> On 29/10/2019 15:18, Daniel Fuchs wrote: > Hi Alan, > > Thanks for this patch that fixes and brings into the mainline > many of the small issues we noticed while working on JDK-8230211. > > I looked at the patch and I think it is good. > > A couple of comments still: > > DatagramChannelImpl: > > ?452???????????????????? assert sender != null; > > pedantically that should be: > > ???????????????????????? assert sender != null || !isOpen(); The only place where it is safe to assert on the channel open/close state is implCloseSelectableChannel. The reason is that the channel can be (asynchronously) closed at any time and this sets the closed state before it coordinates the close with any threads that are doing I/O. > : > > I also believe a similar-ish (but maybe more relevant) kind of > issue would arise for: > > ?491???????????????? assert n >= 0; > > at this point - and as per line > > ?484???????????????????? while (IOStatus.okayToRetry(n) && isOpen()) { > > then n could well be < 0 if the channel is closed. > I suspect the assertion at line 491 should therefore be preceded > with: > > ?? if (n < 0) return n; > > either after line 487 or after line 490. I will re-examine the asserts (I agree they are hard to be confident that they are valid). -Alan From chris.hegarty at oracle.com Tue Oct 29 19:11:55 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 29 Oct 2019 19:11:55 +0000 Subject: 8232673: (dc) DatagramChannel socket adaptor issues In-Reply-To: References: Message-ID: <71B49D00-C1A4-41CE-8CA8-65C15218355A@oracle.com> > On 29 Oct 2019, at 15:18, Daniel Fuchs wrote: > > Hi Alan, > > Thanks for this patch that fixes and brings into the mainline > many of the small issues we noticed while working on JDK-8230211. > > I looked at the patch and I think it is good. +1 Mainly looks very good. A few comments: The window is small but, lockedConfigureBlocking should not call ensureOpen ( that may throw ), when invoked from the finally block, since there could be received data ( that might otherwise be thrown away ). The call to ensureOpen should be conditional. Any reason why auxiliary classes are used, rather than static nested classes? I share similar concerns about the use of assertions in combination with finally blocks, that Daniel has. -Chris. From brian.burkhalter at oracle.com Tue Oct 29 19:16:39 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 29 Oct 2019 12:16:39 -0700 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: <54934789-31f6-20e4-203e-a6d55e7ceab5@oracle.com> References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> <0DAA8543-1EF8-4C3F-9F38-AF9340E853E1@oracle.com> <54934789-31f6-20e4-203e-a6d55e7ceab5@oracle.com> Message-ID: > On Oct 19, 2019, at 12:05 PM, Alan Bateman wrote: > > On 15/10/2019 18:46, Brian Burkhalter wrote: >> >> An updated version which addresses the foregoing points is at http://cr.openjdk.java.net/~bpb/8162520/webrev.03/. > Thanks, that addresses all my issues (although I'm still a bit worried that this one will come back again in the future). Here?s an updated version [1] that does not add the get*Blocks() methods and merely changes the get*Space() methods to return Long.MAX_VALUE instead of a negative value. Adding the get*Blocks() methods may be considered later when a carrier type with greater bit depth becomes available. The test is no longer included as the get*Blocks() methods don?t need to be tested, and it is too difficult to test the negative return case for get*Space(). Thanks, Brian [1] http://cr.openjdk.java.net/~bpb/8162520/webrev.04/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.levart at gmail.com Tue Oct 29 22:30:20 2019 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 29 Oct 2019 23:30:20 +0100 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> <0DAA8543-1EF8-4C3F-9F38-AF9340E853E1@oracle.com> <54934789-31f6-20e4-203e-a6d55e7ceab5@oracle.com> Message-ID: Hi Brian, On 10/29/19 8:16 PM, Brian Burkhalter wrote: > >> On Oct 19, 2019, at 12:05 PM, Alan Bateman > > wrote: >> >> On 15/10/2019 18:46, Brian Burkhalter wrote: >>> >>> An updated version which addresses the foregoing points is at >>> http://cr.openjdk.java.net/~bpb/8162520/webrev.03/. >> Thanks, that addresses all my issues (although I'm still a bit >> worried that this one will come back again in the future). > > Here?s an updated version [1] that does not add the get*Blocks() > methods and merely changes the get*Space() methods to?return > Long.MAX_VALUE instead of a negative value. Adding the get*Blocks() > methods may be considered later when a carrier type with greater > bit?depth becomes available. The test is no longer included as the > get*Blocks() methods don?t need to be tested, and it is too?difficult > to test the negative return case for get*Space(). > There is a carrier type with greater (i.e. unlimited) bit depth available - BigInteger. Why not add 3 methods (getBigXxx...) to return sizes in BigInteger bytes? These are not performance critical methods and hence waiting for value types is not absolutely necessary. What do you think? Regards, Peter > Thanks, > > Brian > > [1] http://cr.openjdk.java.net/~bpb/8162520/webrev.04/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Oct 29 23:09:28 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 29 Oct 2019 16:09:28 -0700 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> <0DAA8543-1EF8-4C3F-9F38-AF9340E853E1@oracle.com> <54934789-31f6-20e4-203e-a6d55e7ceab5@oracle.com> Message-ID: <9C0C24A9-AD52-474E-9CA3-F65DDC21A14F@oracle.com> Hi Peter, > On Oct 29, 2019, at 3:30 PM, Peter Levart wrote: > > There is a carrier type with greater (i.e. unlimited) bit depth available - BigInteger. Why not add 3 methods (getBigXxx...) to return sizes in BigInteger bytes? These are not performance critical methods and hence waiting for value types is not absolutely necessary. > > What do you think? Yes, that was considered. I do not have anything against it myself. The idea of adding the get*Blocks() methods was however partly to avoid selecting a new carrier type for the size-in-bytes values by returning the respective block counts and letting the caller make the decision of where to store the product numBlocks * blockSize. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Oct 30 09:06:16 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 30 Oct 2019 09:06:16 +0000 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> <0DAA8543-1EF8-4C3F-9F38-AF9340E853E1@oracle.com> <54934789-31f6-20e4-203e-a6d55e7ceab5@oracle.com> Message-ID: <219aeec2-2f4b-7965-111d-4a4aded71dbd@oracle.com> On 29/10/2019 22:30, Peter Levart wrote: > > There is a carrier type with greater (i.e. unlimited) bit depth > available - BigInteger. Why not add 3 methods (getBigXxx...) to return > sizes in BigInteger bytes? These are not performance critical methods > and hence waiting for value types is not absolutely necessary. > > What do you think? BigInteger was on Brian's initial set of options but would look unusual here. I think we'd like to defer extending this API for as long as possible so that we have the option to return a 128 or 256-bit result. -Alan From Alan.Bateman at oracle.com Wed Oct 30 13:27:08 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 30 Oct 2019 13:27:08 +0000 Subject: 8232673: (dc) DatagramChannel socket adaptor issues In-Reply-To: <71B49D00-C1A4-41CE-8CA8-65C15218355A@oracle.com> References: <71B49D00-C1A4-41CE-8CA8-65C15218355A@oracle.com> Message-ID: <7a081af2-d4cc-c662-e9d2-2fa9e9ec69d8@oracle.com> On 29/10/2019 19:11, Chris Hegarty wrote: > : > > The window is small but, lockedConfigureBlocking should not call ensureOpen ( that may throw ), when invoked from the finally block, since there could be received data ( that might otherwise be thrown away ). The call to ensureOpen should be conditional. Okay, make sense to just have the I/O operation complete successfully if the datagram has been received. The same thing applies to the other adaptor support in the SC/SSC implementations so I've changed those too so that the restoring of the blocking mode does nothing rather than throw an exception when the channel has been closed. > > Any reason why auxiliary classes are used, rather than static nested classes? Thanks, that was not intentional. > > I share similar concerns about the use of assertions in combination with finally blocks, that Daniel has. > I removed the potentially troublesome asserts as they are a distraction here. The updated webrev is here: ? http://cr.openjdk.java.net/~alanb/8232673/webrev.02/index.html -Alan From brian.burkhalter at oracle.com Wed Oct 30 15:03:22 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 30 Oct 2019 08:03:22 -0700 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: <219aeec2-2f4b-7965-111d-4a4aded71dbd@oracle.com> References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> <0DAA8543-1EF8-4C3F-9F38-AF9340E853E1@oracle.com> <54934789-31f6-20e4-203e-a6d55e7ceab5@oracle.com> <219aeec2-2f4b-7965-111d-4a4aded71dbd@oracle.com> Message-ID: <678000A4-3D81-4425-9BA6-FA903C9A5632@oracle.com> > On Oct 30, 2019, at 2:06 AM, Alan Bateman wrote: > > On 29/10/2019 22:30, Peter Levart wrote: >> >> There is a carrier type with greater (i.e. unlimited) bit depth available - BigInteger. Why not add 3 methods (getBigXxx...) to return sizes in BigInteger bytes? These are not performance critical methods and hence waiting for value types is not absolutely necessary. >> >> What do you think? > BigInteger was on Brian's initial set of options but would look unusual here. I think we'd like to defer extending this API for as long as possible so that we have the option to return a 128 or 256-bit result. Does this constitute approval for the current patch? Perhaps another issue should be filed for the new return type yet to be determined? Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Oct 30 15:07:41 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 30 Oct 2019 15:07:41 +0000 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: <678000A4-3D81-4425-9BA6-FA903C9A5632@oracle.com> References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> <0DAA8543-1EF8-4C3F-9F38-AF9340E853E1@oracle.com> <54934789-31f6-20e4-203e-a6d55e7ceab5@oracle.com> <219aeec2-2f4b-7965-111d-4a4aded71dbd@oracle.com> <678000A4-3D81-4425-9BA6-FA903C9A5632@oracle.com> Message-ID: <512e7f44-7403-3bdc-f75f-8f032f3080ac@oracle.com> On 30/10/2019 15:03, Brian Burkhalter wrote: > : > > Does this constitute approval for the current patch? Perhaps another > issue should be filed for the new return type yet to be determined? > webrev.04 looks okay. I assume you'll create a CSR for the javadoc change. -Alan. From brian.burkhalter at oracle.com Wed Oct 30 15:14:33 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 30 Oct 2019 08:14:33 -0700 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: <512e7f44-7403-3bdc-f75f-8f032f3080ac@oracle.com> References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> <0DAA8543-1EF8-4C3F-9F38-AF9340E853E1@oracle.com> <54934789-31f6-20e4-203e-a6d55e7ceab5@oracle.com> <219aeec2-2f4b-7965-111d-4a4aded71dbd@oracle.com> <678000A4-3D81-4425-9BA6-FA903C9A5632@oracle.com> <512e7f44-7403-3bdc-f75f-8f032f3080ac@oracle.com> Message-ID: <45A0FCDD-F55F-4952-A1BD-5F0957263AD1@oracle.com> > On Oct 30, 2019, at 8:07 AM, Alan Bateman wrote: > > On 30/10/2019 15:03, Brian Burkhalter wrote: >> : >> >> Does this constitute approval for the current patch? Perhaps another issue should be filed for the new return type yet to be determined? >> > webrev.04 looks okay. I assume you'll create a CSR for the javadoc change. Yes, thanks, I was waiting for the RFR to complete in this case. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Oct 30 21:10:17 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 30 Oct 2019 14:10:17 -0700 Subject: 8162520: (fs) FileStore should support file stores with > Long.MAX_VALUE capacity In-Reply-To: <45A0FCDD-F55F-4952-A1BD-5F0957263AD1@oracle.com> References: <49606851-5b61-6913-0e22-654e4368ad93@oracle.com> <6CE812A3-01A6-41A2-B1C3-BE82CDA2F985@oracle.com> <3555D132-99CC-4215-A8B8-3C6783F8A557@oracle.com> <91BEE173-1E29-481D-A28B-25C00801A7C9@oracle.com> <016D4C1D-98C8-429C-8250-4420CE7461CD@oracle.com> <19A929D3-7AB6-41A2-9A97-B5E0349E12D8@oracle.com> <0DAA8543-1EF8-4C3F-9F38-AF9340E853E1@oracle.com> <54934789-31f6-20e4-203e-a6d55e7ceab5@oracle.com> <219aeec2-2f4b-7965-111d-4a4aded71dbd@oracle.com> <678000A4-3D81-4425-9BA6-FA903C9A5632@oracle.com> <512e7f44-7403-3bdc-f75f-8f032f3080ac@oracle.com> <45A0FCDD-F55F-4952-A1BD-5F0957263AD1@oracle.com> Message-ID: <4720480B-0051-402B-BB2D-18E04A15D1F8@oracle.com> > On Oct 30, 2019, at 8:14 AM, Brian Burkhalter wrote: > >> On Oct 30, 2019, at 8:07 AM, Alan Bateman > wrote: >> >> On 30/10/2019 15:03, Brian Burkhalter wrote: >>> : >>> >>> Does this constitute approval for the current patch? Perhaps another issue should be filed for the new return type yet to be determined? >>> >> webrev.04 looks okay. I assume you'll create a CSR for the javadoc change. > > Yes, thanks, I was waiting for the RFR to complete in this case. CSR filed: https://bugs.openjdk.java.net/browse/JDK-8233264. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Oct 31 00:42:18 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 30 Oct 2019 17:42:18 -0700 Subject: 8146298: (dc spec) connect and disconnect methods should specify that they may block Message-ID: <079224C1-8434-47F4-8013-EB7CE8E4B940@oracle.com> For the issue [1] to improve the specification of DatagramChannel please review this doc change [2]. Hopefully I?ve captured the behavior accurately. Note the class-level doc change to match SocketChannel. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8146298 [2] http://cr.openjdk.java.net/~bpb/8146298/webrev.00/ From Alan.Bateman at oracle.com Thu Oct 31 13:49:57 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 31 Oct 2019 13:49:57 +0000 Subject: 8146298: (dc spec) connect and disconnect methods should specify that they may block In-Reply-To: <079224C1-8434-47F4-8013-EB7CE8E4B940@oracle.com> References: <079224C1-8434-47F4-8013-EB7CE8E4B940@oracle.com> Message-ID: <9f1c2029-cc8c-2c36-1278-8c7835390247@oracle.com> On 31/10/2019 00:42, Brian Burkhalter wrote: > For the issue [1] to improve the specification of DatagramChannel please review this doc change [2]. Hopefully I?ve captured the behavior accurately. Note the class-level doc change to match SocketChannel. > I don't think the change to the class description is needed because connect/disconnect aren't doing I/O, they are just setting up or dissolving an address association. The sentence in connect/disconnect to describe that it will blocking until outstanding I/O operations complete is okay but the version in connect has "however" in the middle of the sentence, I think you can drop that so that it is consistent with disconnect. I assume we should? move the statement about discarding datagrams to an implNote. -Alan From brian.burkhalter at oracle.com Thu Oct 31 18:26:34 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 31 Oct 2019 11:26:34 -0700 Subject: 8146298: (dc spec) connect and disconnect methods should specify that they may block In-Reply-To: <9f1c2029-cc8c-2c36-1278-8c7835390247@oracle.com> References: <079224C1-8434-47F4-8013-EB7CE8E4B940@oracle.com> <9f1c2029-cc8c-2c36-1278-8c7835390247@oracle.com> Message-ID: <13E91021-35D4-4964-81C2-D2838C654035@oracle.com> > On Oct 31, 2019, at 6:49 AM, Alan Bateman wrote: > > On 31/10/2019 00:42, Brian Burkhalter wrote: >> For the issue [1] to improve the specification of DatagramChannel please review this doc change [2]. Hopefully I?ve captured the behavior accurately. Note the class-level doc change to match SocketChannel. >> > I don't think the change to the class description is needed because connect/disconnect aren't doing I/O, they are just setting up or dissolving an address association. Specifically it was intended to match this sentence from the SocketChannel class description: "The connect and finishConnect methods are mutually synchronized against each other, and an attempt to initiate a read or write operation while an invocation of one of these methods is in progress will block until that invocation is complete." > The sentence in connect/disconnect to describe that it will blocking until outstanding I/O operations complete is okay but the version in connect has "however" in the middle of the sentence, I think you can drop that so that it is consistent with disconnect. Unless I am mistaken both had ?however? which also matches the specifications of receive() and send(), but I agree it?s better to remove it. > I assume we should move the statement about discarding datagrams to an implNote. Done [*]. Thanks, Brian [*] http://cr.openjdk.java.net/~bpb/8146298/webrev.01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Oct 31 18:34:46 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 31 Oct 2019 18:34:46 +0000 Subject: 8146298: (dc spec) connect and disconnect methods should specify that they may block In-Reply-To: <13E91021-35D4-4964-81C2-D2838C654035@oracle.com> References: <079224C1-8434-47F4-8013-EB7CE8E4B940@oracle.com> <9f1c2029-cc8c-2c36-1278-8c7835390247@oracle.com> <13E91021-35D4-4964-81C2-D2838C654035@oracle.com> Message-ID: <7d558d8d-dc7b-ead9-9c4b-809ac9cc9790@oracle.com> On 31/10/2019 18:26, Brian Burkhalter wrote: > : > > Specifically it was intended to match this sentence from the > SocketChannel class description: > > "The connect?and finishConnect?methods are?mutually synchronized > against each other, and an attempt to initiate a read?or?write > operation while an invocation of one of these methods is in?progress > will block until that invocation is complete." > "in progress" makes sense here because it can take significant time to establish connection or wait for a connection to be established. With a DatagramChannel, the connect/disconnect is just setting or dissolving an address association so I'm concerned the proposed addition to the class description is giving the wrong impression. So my view is that we don't need this part of the change. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Oct 31 18:35:39 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 31 Oct 2019 11:35:39 -0700 Subject: 8146298: (dc spec) connect and disconnect methods should specify that they may block In-Reply-To: <7d558d8d-dc7b-ead9-9c4b-809ac9cc9790@oracle.com> References: <079224C1-8434-47F4-8013-EB7CE8E4B940@oracle.com> <9f1c2029-cc8c-2c36-1278-8c7835390247@oracle.com> <13E91021-35D4-4964-81C2-D2838C654035@oracle.com> <7d558d8d-dc7b-ead9-9c4b-809ac9cc9790@oracle.com> Message-ID: <02163C85-2CC8-476C-9A37-7A7CC55739FE@oracle.com> > On Oct 31, 2019, at 11:34 AM, Alan Bateman wrote: > > On 31/10/2019 18:26, Brian Burkhalter wrote: >> : >> >> Specifically it was intended to match this sentence from the SocketChannel class description: >> >> "The connect and finishConnect methods are mutually synchronized against each other, and an attempt to initiate a read or write operation while an invocation of one of these methods is in progress will block until that invocation is complete." >> > "in progress" makes sense here because it can take significant time to establish connection or wait for a connection to be established. With a DatagramChannel, the connect/disconnect is just setting or dissolving an address association so I'm concerned the proposed addition to the class description is giving the wrong impression. So my view is that we don't need this part of the change. I am fine with leaving it out. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Oct 31 20:07:16 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 31 Oct 2019 13:07:16 -0700 Subject: 8146298: (dc spec) connect and disconnect methods should specify that they may block In-Reply-To: <02163C85-2CC8-476C-9A37-7A7CC55739FE@oracle.com> References: <079224C1-8434-47F4-8013-EB7CE8E4B940@oracle.com> <9f1c2029-cc8c-2c36-1278-8c7835390247@oracle.com> <13E91021-35D4-4964-81C2-D2838C654035@oracle.com> <7d558d8d-dc7b-ead9-9c4b-809ac9cc9790@oracle.com> <02163C85-2CC8-476C-9A37-7A7CC55739FE@oracle.com> Message-ID: <2C272D07-EC10-4E6E-A11A-D6D2A687A4B8@oracle.com> > On Oct 31, 2019, at 11:35 AM, Brian Burkhalter wrote: > >> On Oct 31, 2019, at 11:34 AM, Alan Bateman > wrote: >> >> On 31/10/2019 18:26, Brian Burkhalter wrote: >>> : >>> >>> Specifically it was intended to match this sentence from the SocketChannel class description: >>> >>> "The connect and finishConnect methods are mutually synchronized against each other, and an attempt to initiate a read or write operation while an invocation of one of these methods is in progress will block until that invocation is complete." >>> >> "in progress" makes sense here because it can take significant time to establish connection or wait for a connection to be established. With a DatagramChannel, the connect/disconnect is just setting or dissolving an address association so I'm concerned the proposed addition to the class description is giving the wrong impression. So my view is that we don't need this part of the change. > > I am fine with leaving it out. CSR filed: https://bugs.openjdk.java.net/browse/JDK-8233342. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: