From fweimer at redhat.com Wed Apr 1 14:26:23 2020 From: fweimer at redhat.com (Florian Weimer) Date: Wed, 01 Apr 2020 16:26:23 +0200 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <65CD68C6-07A8-4811-B4FD-0F1277DA8697@oracle.com> (Brian Burkhalter's message of "Thu, 26 Mar 2020 08:49:27 -0700") References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <87wo8ti01g.fsf@oldenburg2.str.redhat.com> <824238F5-D399-4088-A1AA-A9AC8321577F@oracle.com> <87eev1hwxa.fsf@oldenburg2.str.redhat.com> <8736bhhvst.fsf@oldenburg2.str.redhat.com> <6B7BB465-EA1F-4763-8E2D-927B71EADFC6@oracle.com> <87imjsxk4j.fsf@oldenburg2.str.redhat.com> <65CD68C6-07A8-4811-B4FD-0F1277DA8697@oracle.com> Message-ID: <87y2rfi8og.fsf@oldenburg2.str.redhat.com> * Brian Burkhalter: > Resuscitating the thread [1] concerning [2]. An updated version of the > proposed patch is [3]. This differs from the previous version in that > it permits the source and destination buffers to share the same > backing storage. Hopefully I have implemented this correctly (it is > assumed that System.arraycopy() [4] and Unsafe copying handle > overlapping copies as documented for the new methods $Type$Buffer > put?(int, $Type$Buffer, int, int) {}). > [3] http://cr.openjdk.java.net/~bpb/8219014/webrev.01/ Hmm. This part + * of both buffers are unchanged. If the source buffer and this buffer share + * the same backing storage and the source and destination ranges overlap, + * then the transfer is performed as if the elements at positions still references ?backing storage?? Should it say ?backing array? instead? I don't think you can make such a promise for direct buffers. Thanks, Florian From brian.burkhalter at oracle.com Wed Apr 1 15:57:24 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 1 Apr 2020 08:57:24 -0700 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <87y2rfi8og.fsf@oldenburg2.str.redhat.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <87wo8ti01g.fsf@oldenburg2.str.redhat.com> <824238F5-D399-4088-A1AA-A9AC8321577F@oracle.com> <87eev1hwxa.fsf@oldenburg2.str.redhat.com> <8736bhhvst.fsf@oldenburg2.str.redhat.com> <6B7BB465-EA1F-4763-8E2D-927B71EADFC6@oracle.com> <87imjsxk4j.fsf@oldenburg2.str.redhat.com> <65CD68C6-07A8-4811-B4FD-0F1277DA8697@oracle.com> <87y2rfi8og.fsf@oldenburg2.str.redhat.com> Message-ID: <805C5BDD-C826-4C61-A9A8-11656FD71267@oracle.com> Hi Florian, > On Apr 1, 2020, at 7:26 AM, Florian Weimer wrote: > >> Resuscitating the thread [1] concerning [2]. An updated version of the >> proposed patch is [3]. This differs from the previous version in that >> it permits the source and destination buffers to share the same >> backing storage. Hopefully I have implemented this correctly (it is >> assumed that System.arraycopy() [4] and Unsafe copying handle >> overlapping copies as documented for the new methods $Type$Buffer >> put?(int, $Type$Buffer, int, int) {}). > >> [3] http://cr.openjdk.java.net/~bpb/8219014/webrev.01/ > > Hmm. This part > > + * of both buffers are unchanged. If the source buffer and this buffer share > + * the same backing storage and the source and destination ranges overlap, > + * then the transfer is performed as if the elements at positions > > still references ?backing storage?? Should it say ?backing array? > instead? > > I don't think you can make such a promise for direct buffers. I think the fallback code that does put(j, get(i)) instead of using arraycopy() or Unsafe needs to be checked for this, but the latter two approaches insofar as we know will handle overlaps. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Wed Apr 1 17:42:20 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 1 Apr 2020 18:42:20 +0100 Subject: 8241786: 3 DatagramChannel tests and 2 MulticastSocket tests fail due to "SocketException: No buffer space available" Message-ID: Hi, Please find below a fix for: 8241786: 3 DatagramChannel tests and 2 MulticastSocket tests fail due to "SocketException: No buffer space available" https://bugs.openjdk.java.net/browse/JDK-8241786 webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8241786/webrev.01/ The real fix is in these three files: DefaultInterface.java (macOS): is modified to prefer interfaces that have non link local addresses NetworkConfiguration.java (test lib): is modified to skip interfaces that have only link-local addresses (whether IPv4 or IPv6) for multicasting. MulticastSocket/SetOutgoingIf.java: this test does its own thing, and is modified similarly to NetworkConfiguration Additional cleanup: The other tests are just modified to add diagnostic and use try-with-resources whenever possible, and also use /othervm to avoid some undesirable side effects (like not seeing any trace): MulticastSocket/SetLoopbackMode.java MulticastSocket/SetLoopbackModeIPv4.java DatagramChannel/AdaptorMulticasting.java DatagramChannel/MulticastSendReceiveTests.java DatagramChannel/Promiscuous.java best regards, -- daniel From Alan.Bateman at oracle.com Wed Apr 1 19:50:31 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 1 Apr 2020 20:50:31 +0100 Subject: 8241786: 3 DatagramChannel tests and 2 MulticastSocket tests fail due to "SocketException: No buffer space available" In-Reply-To: References: Message-ID: On 01/04/2020 18:42, Daniel Fuchs wrote: > Hi, > > Please find below a fix for: > > 8241786: 3 DatagramChannel tests and 2 MulticastSocket tests > ???????? fail due to "SocketException: No buffer space available" > https://bugs.openjdk.java.net/browse/JDK-8241786 > > webrev: > http://cr.openjdk.java.net/~dfuchs/webrev_8241786/webrev.01/ > > The real fix is in these three files: > > DefaultInterface.java (macOS): is modified to prefer interfaces > ??? that have non link local addresses > NetworkConfiguration.java (test lib): is modified to skip > ??? interfaces that have only link-local addresses (whether IPv4 > ??? or IPv6) for multicasting. > MulticastSocket/SetOutgoingIf.java: this test does its own thing, > ??? and is modified similarly to NetworkConfiguration > > Additional cleanup: > > The other tests are just modified to add diagnostic and use > try-with-resources whenever possible, and also use /othervm to avoid > some undesirable side effects (like not seeing any trace): > > MulticastSocket/SetLoopbackMode.java > MulticastSocket/SetLoopbackModeIPv4.java > DatagramChannel/AdaptorMulticasting.java > DatagramChannel/MulticastSendReceiveTests.java > DatagramChannel/Promiscuous.java The change to java.net.DefaultInterface to workaround bad configurations on macOS looks good. Hopefully existing usages of the 1-arg joinGroup will migrate to the 2-arg variant and avoid needing the JDK to choose on multi-homed systems. DefaultInterface might be better in a test infra class (in java/net) to avoid duplicate it in several tests. The updates to the DatagramChannel tests look okay but I think we should change the new trace messages added to AdaptorMulticasting to be consistent with the existing trace, e.g. the test cycles through the families and interfaces and prints output like this following before each run: Test family=INET, multicast group=/225.4.5.6, interface=en0 The changes in the patch duplicate a subset of this in a different format. It might be simpler to just not duplicate it but print just print the name of the sub-tests, whatever is easiest. NetworkConfiguration looks okay although I could imagine the comment added at L101-103 not being clearer for future maintainers. I think we'll need to do a cleanup of this test class at some point. -Alan From gnu.andrew at redhat.com Thu Apr 2 03:46:54 2020 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Thu, 2 Apr 2020 04:46:54 +0100 Subject: [8u] RFR: 8229888: (zipfs) Updating an existing zip file does not preserve original permissions In-Reply-To: <09d61305-07c1-d493-60af-7e96e0bfc7b3@redhat.com> References: <09d61305-07c1-d493-60af-7e96e0bfc7b3@redhat.com> Message-ID: <7366bd30-b282-656a-1c2e-0cc336f815ef@redhat.com> On 26/03/2020 23:15, Alex Kashchenko wrote: > Hi, > > Please review the backport of JDK-8229888 to 8u: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8229888 > > Original review thread: > http://mail.openjdk.java.net/pipermail/nio-dev/2019-December/006894.html > > 11u changeset: > https://hg.openjdk.java.net/jdk-updates/jdk11u/rev/57fd597352b8 > > 8u webrev: http://cr.openjdk.java.net/~akasko/jdk8u/8229888/webrev.00/ > > 8u patch is very close to 11u one: ZipFileSystem part is the same, test > is changed to replace API not available in 8u, changes to policy file > are not included. > It would be helpful if you could explain why such changes had to be made, rather than leaving the reviewer to guess. The test changes are fairly clear, though the replacement for Set.of, createSet, returns a modifiable set, whereas of returns an unmodifiable set. This doesn't really matter for a test case, but may in JDK code. It could also potentially be simplified to just: private static Set createSet(Object... entries) { Set set = new LinkedHashSet<>(Arrays.asList(entries)); return Collections.unmodifiableSet(set); } I'm not sure why the policy file change is unnecessary. I would guess it's because the zipfs code is only demo code in 8u, but it would be good to have more reassurance. Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: OpenPGP digital signature URL: From chris.hegarty at oracle.com Thu Apr 2 08:40:53 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 2 Apr 2020 09:40:53 +0100 Subject: 8241786: 3 DatagramChannel tests and 2 MulticastSocket tests fail due to "SocketException: No buffer space available" In-Reply-To: References: Message-ID: <63D83EB8-7CA6-407B-97DC-AF4133417FD5@oracle.com> Daniel, > On 1 Apr 2020, at 18:42, Daniel Fuchs wrote: > > Hi, > > Please find below a fix for: > > 8241786: 3 DatagramChannel tests and 2 MulticastSocket tests > fail due to "SocketException: No buffer space available" > https://bugs.openjdk.java.net/browse/JDK-8241786 Good work. A really tough issue to diagnose. This is a functional change that affects the product build. Maybe that would be a little clearer from a different bug synopsis. Suggest something like: Improve heuristic to determine default network interface on macOS > webrev: > http://cr.openjdk.java.net/~dfuchs/webrev_8241786/webrev.01/ DefaultInterface.java Unless I'm mistaken, the first IPv4+IPv6+nonLinkLocal interface is not necessarily returned. The second will be returned if there are two such interfaces. I'm not saying that this is necessarily a problem, just trying to reconsile the code with the comment. Maybe the `preferred` can be skipped, just returning such an IPv4+IPv6+nonLinkLocal if found. -Chris. From daniel.fuchs at oracle.com Thu Apr 2 08:48:57 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 2 Apr 2020 09:48:57 +0100 Subject: 8241786: 3 DatagramChannel tests and 2 MulticastSocket tests fail due to "SocketException: No buffer space available" In-Reply-To: References: Message-ID: Hi Alan, On 01/04/2020 20:50, Alan Bateman wrote: > The change to java.net.DefaultInterface to workaround bad configurations > on macOS looks good. Hopefully existing usages of the 1-arg joinGroup > will migrate to the 2-arg variant and avoid needing the JDK to choose on > multi-homed systems. Thanks. > DefaultInterface might be better in a test infra class (in java/net) to > avoid duplicate it in several tests. I have considered that. The problem is that it requires the test that uses it to add @modules java.base/java.net:+open directive in the test itself. At the moment only one test uses it - so there is no duplication (yet). Maybe we should wait until we need it elsewhere before thinking of ways to move it to the test library? > The updates to the DatagramChannel tests look okay but I think we should > change the new trace messages added to AdaptorMulticasting to be > consistent with the existing trace, e.g. the test cycles through the > families and interfaces and prints output like this following before > each run: > > Test family=INET, multicast group=/225.4.5.6, interface=en0 > > The changes in the patch duplicate a subset of this in a different > format. It might be simpler to just not duplicate it but print just > print the name of the sub-tests, whatever is easiest. When I ran this test it was failing in timeout with *no* traces. I hate that. I added traces to figure out at which point it was failing. There might be a bit of duplication in the information, but I'd rather have that than wonder if the test is actually doing what I think it does. When unexplainable things happen, I start doubting everything ;-) If you insist I'll reluctantly remove the redundant information, but I do think it helps. > NetworkConfiguration looks okay although I could imagine the comment > added at L101-103 not being clearer for future maintainers. I think > we'll need to do a cleanup of this test class at some point. I'll see if I can clarify this - or maybe I'll just remove it. I agree it could be confusing. best regards, -- daniel > > -Alan > From Alan.Bateman at oracle.com Thu Apr 2 08:56:05 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 2 Apr 2020 09:56:05 +0100 Subject: 8241786: 3 DatagramChannel tests and 2 MulticastSocket tests fail due to "SocketException: No buffer space available" In-Reply-To: References: Message-ID: <26688831-e801-3d17-54f3-22ec3db585c4@oracle.com> On 02/04/2020 09:48, Daniel Fuchs wrote: > : > >> DefaultInterface might be better in a test infra class (in java/net) >> to avoid duplicate it in several tests. > > I have considered that. The problem is that it requires the test that > uses it to add @modules java.base/java.net:+open directive in the test > itself. At the moment only one test uses it - so there is no > duplication (yet). Maybe we should wait until we need it elsewhere > before thinking of ways to move it to the test library? Okay, I was assuming it would end up being used by several MulticastSocket tests. > : > > When I ran this test it was failing in timeout with *no* traces. > I hate that. I added traces to figure out at which point it was > failing. There might be a bit of duplication in the information, > but I'd rather have that than wonder if the test is actually > doing what I think it does. When unexplainable things happen, > I start doubting everything ;-) > > If you insist I'll reluctantly remove the redundant information, > but I do think it helps. All I'm asking is that additional tracing being consistent with the existing traces. Stack traces and thread dumps the most useful when there is a timeout or test timeout. I wonder if there is an? issue with the timeout/failure handler right now so you should have got one here in the event that the receive blocks indefinitely. -Alan From daniel.fuchs at oracle.com Thu Apr 2 09:33:41 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 2 Apr 2020 10:33:41 +0100 Subject: 8241786: 3 DatagramChannel tests and 2 MulticastSocket tests fail due to "SocketException: No buffer space available" In-Reply-To: <63D83EB8-7CA6-407B-97DC-AF4133417FD5@oracle.com> References: <63D83EB8-7CA6-407B-97DC-AF4133417FD5@oracle.com> Message-ID: Hi Chris, On 02/04/2020 09:40, Chris Hegarty wrote: > This is a functional change that affects the product build. Maybe that > would be a little clearer from a different bug synopsis. Suggest > something like: > > Improve heuristic to determine default network interface on macOS I can do that - or I can split the change if needed: that is push the changes to DefaultInterface on its on? Maybe I should add an @bug 8241786 to some of these tests too. >> webrev: >> http://cr.openjdk.java.net/~dfuchs/webrev_8241786/webrev.01/ > > DefaultInterface.java > > Unless I'm mistaken, the first IPv4+IPv6+nonLinkLocal interface is not > necessarily returned. Damn. Good catch! But that's your bug Chris: it was already there :-) http://cr.openjdk.java.net/~dfuchs/webrev_8241786/webrev.02 includes yours and Alan's feedback. -- daniel > The second will be returned if there are two such > interfaces. I'm not saying that this is necessarily a problem, just > trying to reconsile the code with the comment. Maybe the `preferred` can > be skipped, just returning such an IPv4+IPv6+nonLinkLocal if found. > > -Chris. > From chris.hegarty at oracle.com Fri Apr 3 08:59:10 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 3 Apr 2020 09:59:10 +0100 Subject: 8241786: 3 DatagramChannel tests and 2 MulticastSocket tests fail due to "SocketException: No buffer space available" In-Reply-To: References: <63D83EB8-7CA6-407B-97DC-AF4133417FD5@oracle.com> Message-ID: <113A2261-001E-4418-8D07-49C51897AEAB@oracle.com> Daniel, > On 2 Apr 2020, at 10:33, Daniel Fuchs wrote: > > Hi Chris, > > On 02/04/2020 09:40, Chris Hegarty wrote: >> This is a functional change that affects the product build. Maybe that >> would be a little clearer from a different bug synopsis. Suggest >> something like: >> Improve heuristic to determine default network interface on macOS > > I can do that - An updated synopsis should be fine. > or I can split the change if needed: that is push > the changes to DefaultInterface on its on? > Maybe I should add an @bug 8241786 to some of these tests too. > >>> webrev: >>> http://cr.openjdk.java.net/~dfuchs/webrev_8241786/webrev.01/ >> DefaultInterface.java >> Unless I'm mistaken, the first IPv4+IPv6+nonLinkLocal interface is not >> necessarily returned. > > Damn. Good catch! But that's your bug Chris: it was already there :-) > > http://cr.openjdk.java.net/~dfuchs/webrev_8241786/webrev.02 LGTM. -Chris. From daniel.fuchs at oracle.com Fri Apr 3 11:04:32 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 3 Apr 2020 12:04:32 +0100 Subject: 8241786: 3 DatagramChannel tests and 2 MulticastSocket tests fail due to "SocketException: No buffer space available" In-Reply-To: <26688831-e801-3d17-54f3-22ec3db585c4@oracle.com> References: <26688831-e801-3d17-54f3-22ec3db585c4@oracle.com> Message-ID: <67864fc4-804a-28fd-5345-749dc190a345@oracle.com> Hi Alan, Hopefully that's fixed now. Only change is in AdaptorMulticasting.java http://cr.openjdk.java.net/~dfuchs/webrev_8241786/webrev.03 best regards, -- daniel On 02/04/2020 09:56, Alan Bateman wrote: > All I'm asking is that additional tracing being consistent with the > existing traces. Stack traces and thread dumps the most useful when > there is a timeout or test timeout. I wonder if there is an? issue with > the timeout/failure handler right now so you should have got one here in > the event that the receive blocks indefinitely. From Alan.Bateman at oracle.com Fri Apr 3 11:31:47 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 3 Apr 2020 12:31:47 +0100 Subject: 8241786: 3 DatagramChannel tests and 2 MulticastSocket tests fail due to "SocketException: No buffer space available" In-Reply-To: <67864fc4-804a-28fd-5345-749dc190a345@oracle.com> References: <26688831-e801-3d17-54f3-22ec3db585c4@oracle.com> <67864fc4-804a-28fd-5345-749dc190a345@oracle.com> Message-ID: On 03/04/2020 12:04, Daniel Fuchs wrote: > Hi Alan, > > Hopefully that's fixed now. Only change is in AdaptorMulticasting.java > > http://cr.openjdk.java.net/~dfuchs/webrev_8241786/webrev.03 Thanks, looks good. -Alan From lance.andersen at oracle.com Fri Apr 3 13:23:32 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 3 Apr 2020 09:23:32 -0400 Subject: RFR 8242006: (zipfs) Improve Zip FS FileChannel and SeekableByteChannel test coverage Message-ID: Hi all, Please review the patch for 8242006, which improves the Zip FS test coverage for FileChannel and SeekableByteChannel. The webrev can be found at: http://cr.openjdk.java.net/~lancea/8242006/webrev.00/index.html Best, Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From patrick.concannon at oracle.com Fri Apr 3 13:47:15 2020 From: patrick.concannon at oracle.com (Patrick Concannon) Date: Fri, 3 Apr 2020 06:47:15 -0700 (PDT) Subject: RFR[8240533]: 'Inconsistent Exceptions are thrown by DatagramSocket and DatagramChannel when sending a DatagramPacket to port 0.' In-Reply-To: <6BC55414-E001-482C-A5B2-F9ACC984A5D0@oracle.com> References: <6ee50304-1122-64f7-91fe-d3ca6ca33237@oracle.com> <6a60e4a1-f9a5-c902-c2be-013d2ed41b40@oracle.com> <6BC55414-E001-482C-A5B2-F9ACC984A5D0@oracle.com> Message-ID: <0911616f-d0dd-877c-de99-1c2cd85f7b1a@oracle.com> Hi, Thanks for the feedback. Lance - I swapped out expectThrows for assertThrows, as requested. Chris - I put in an extra check in the tests to ensure that the new code doesn?t interfere with the Security Manager checks already present in the source. The new webrev can be found here: http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.01/ Kind regards, Patrick On 31/03/2020 15:33, Chris Hegarty wrote: > Patrick, > >> On 31 Mar 2020, at 15:08, Daniel Fuchs wrote: >> >>> .. >>> bug: https://bugs.openjdk.java.net/browse/JDK-8240533 >>> webrev: http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.00/ > Look good Patrick. > > The check is deliberately performed after the security manager checks, right? If so, it is worth asserting this in a test. > > -Chris. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Fri Apr 3 16:08:32 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 3 Apr 2020 09:08:32 -0700 (PDT) Subject: RFR[8240533]: 'Inconsistent Exceptions are thrown by DatagramSocket and DatagramChannel when sending a DatagramPacket to port 0.' In-Reply-To: <0911616f-d0dd-877c-de99-1c2cd85f7b1a@oracle.com> References: <6ee50304-1122-64f7-91fe-d3ca6ca33237@oracle.com> <6a60e4a1-f9a5-c902-c2be-013d2ed41b40@oracle.com> <6BC55414-E001-482C-A5B2-F9ACC984A5D0@oracle.com> <0911616f-d0dd-877c-de99-1c2cd85f7b1a@oracle.com> Message-ID: <57F70695-2696-423F-8286-354723E778E9@oracle.com> Hi Patrick, > On 3 Apr 2020, at 14:47, Patrick Concannon wrote: > > ... > > Chris - I put in an extra check in the tests to ensure that the new code doesn?t interfere with the Security Manager checks already present in the source. Thank you Patrick. > The new webrev can be found here: http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.01/ > > I have just a few more minor comments: I think there is a small cut?n?paste issue with test/jdk/java/nio/channels/DatagramChannel/SendPortZero.java 59 * @run testng/othervm -Djava.net.preferIPv4Stack=true ConnectPortZero <<< should be SendPortZero typo: Execption in two test summary tags There is no need to use two instance initializers, when one will suffice ;-) : 123 { perms.add( 124 new SocketPermission("127.0.0.1:0", "connect")); } 125 { perms.add( 126 new SocketPermission("0.0.0.0:0", "connect")); } Just: { perms.add(new SocketPermission("127.0.0.1:0", "connect")); perms.add(new SocketPermission("0.0.0.0:0", "connect")); } Thanks, -Chris. From daniel.fuchs at oracle.com Fri Apr 3 16:16:22 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 3 Apr 2020 17:16:22 +0100 Subject: RFR[8240533]: 'Inconsistent Exceptions are thrown by DatagramSocket and DatagramChannel when sending a DatagramPacket to port 0.' In-Reply-To: <0911616f-d0dd-877c-de99-1c2cd85f7b1a@oracle.com> References: <6ee50304-1122-64f7-91fe-d3ca6ca33237@oracle.com> <6a60e4a1-f9a5-c902-c2be-013d2ed41b40@oracle.com> <6BC55414-E001-482C-A5B2-F9ACC984A5D0@oracle.com> <0911616f-d0dd-877c-de99-1c2cd85f7b1a@oracle.com> Message-ID: Hi Patrick, 120 { perms.add(new SocketPermission("127.0.0.1:0", 121 "connect,accept")); } 122 { perms.add(new SocketPermission("0.0.0.0:0", 123 "connect,accept")); } there in other tests - I think a single permission: { perms.add(new SocketPermission("*:0")); } would be more robust as it would take care of both IPv6 and IPv4 in one go. We should strive to avoid to hard-code 127.0.0.1 and 0.0.0.0 in tests. best regards, -- daniel On 03/04/2020 14:47, Patrick Concannon wrote: > Hi, > > Thanks for the feedback. > > Lance - I swapped out expectThrows for assertThrows, as requested. > > Chris - I put in an extra check in the tests to ensure that the new code > doesn?t interfere with the Security Manager checks already present in > the source. > > The new webrev can be found here: > http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.01/ > > > > Kind regards, > > Patrick > > On 31/03/2020 15:33, Chris Hegarty wrote: >> Patrick, >> >>> On 31 Mar 2020, at 15:08, Daniel Fuchs wrote: >>> >>>> .. >>>> bug:https://bugs.openjdk.java.net/browse/JDK-8240533 >>>> webrev:http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.00/ >> Look good Patrick. >> >> The check is deliberately performed after the security manager checks, right? If so, it is worth asserting this in a test. >> >> -Chris. >> From akashche at redhat.com Sat Apr 4 11:28:42 2020 From: akashche at redhat.com (Alex Kashchenko) Date: Sat, 4 Apr 2020 12:28:42 +0100 Subject: [8u] RFR: 8229888: (zipfs) Updating an existing zip file does not preserve original permissions In-Reply-To: <7366bd30-b282-656a-1c2e-0cc336f815ef@redhat.com> References: <09d61305-07c1-d493-60af-7e96e0bfc7b3@redhat.com> <7366bd30-b282-656a-1c2e-0cc336f815ef@redhat.com> Message-ID: On 04/02/2020 04:46 AM, Andrew Hughes wrote: > On 26/03/2020 23:15, Alex Kashchenko wrote: >> Hi, >> >> Please review the backport of JDK-8229888 to 8u: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8229888 >> >> Original review thread: >> http://mail.openjdk.java.net/pipermail/nio-dev/2019-December/006894.html >> >> 11u changeset: >> https://hg.openjdk.java.net/jdk-updates/jdk11u/rev/57fd597352b8 >> >> 8u webrev: http://cr.openjdk.java.net/~akasko/jdk8u/8229888/webrev.00/ >> >> 8u patch is very close to 11u one: ZipFileSystem part is the same, test >> is changed to replace API not available in 8u, changes to policy file >> are not included. >> > > It would be helpful if you could explain why such changes had to be > made, rather than leaving the reviewer to guess. > > The test changes are fairly clear, though the replacement for Set.of, > createSet, returns a modifiable set, whereas of returns an unmodifiable > set. This doesn't really matter for a test case, but may in JDK code. > > It could also potentially be simplified to just: > > private static Set createSet(Object... entries) { > Set set = new LinkedHashSet<>(Arrays.asList(entries)); > return Collections.unmodifiableSet(set); > } Thanks for the review! I simplified this part of test removing createSet() helper: https://cr.openjdk.java.net/~akasko/jdk8u/8229888/webrev.01/ > > I'm not sure why the policy file change is unnecessary. I would guess > it's because the zipfs code is only demo code in 8u, but it would be > good to have more reassurance. AFAIU these changes to policy file are only necessary for jdk9+ with JPMS, some details on this policy change in jdk11: https://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020-March/002644.html https://mail.openjdk.java.net/pipermail/nio-dev/2019-March/005978.html -- -Alex From patrick.concannon at oracle.com Sat Apr 4 15:26:48 2020 From: patrick.concannon at oracle.com (Patrick Concannon) Date: Sat, 4 Apr 2020 16:26:48 +0100 Subject: RFR[8240533]: 'Inconsistent Exceptions are thrown by DatagramSocket and DatagramChannel when sending a DatagramPacket to port 0.' In-Reply-To: References: <6ee50304-1122-64f7-91fe-d3ca6ca33237@oracle.com> <6a60e4a1-f9a5-c902-c2be-013d2ed41b40@oracle.com> <6BC55414-E001-482C-A5B2-F9ACC984A5D0@oracle.com> <0911616f-d0dd-877c-de99-1c2cd85f7b1a@oracle.com> Message-ID: Hi Chris and Daniel, Well spotted, Chris. Thanks for that! Thanks too Daniel, that's a good idea. I've made those changes and included them in a new webrev, which you can find below. http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.02/ Kind regards, Patrick On 03/04/2020 17:16, Daniel Fuchs wrote: > Hi Patrick, > > ?120???????? { perms.add(new SocketPermission("127.0.0.1:0", > ?121???????????????? "connect,accept")); } > ?122???????? { perms.add(new SocketPermission("0.0.0.0:0", > ?123???????????????? "connect,accept")); } > > there in other tests - I think a single permission: > > ??? { perms.add(new SocketPermission("*:0")); } > > would be more robust as it would take care of both IPv6 and IPv4 in one > go. We should strive to avoid to hard-code 127.0.0.1 and 0.0.0.0 > in tests. > > best regards, > > -- daniel > > On 03/04/2020 14:47, Patrick Concannon wrote: >> Hi, >> >> Thanks for the feedback. >> >> Lance - I swapped out expectThrows for assertThrows, as requested. >> >> Chris - I put in an extra check in the tests to ensure that the new >> code doesn?t interfere with the Security Manager checks already >> present in the source. >> >> The new webrev can be found here: >> http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.01/ >> >> >> >> Kind regards, >> >> Patrick >> >> On 31/03/2020 15:33, Chris Hegarty wrote: >>> Patrick, >>> >>>> On 31 Mar 2020, at 15:08, Daniel Fuchs? >>>> wrote: >>>> >>>>> .. >>>>> bug:https://bugs.openjdk.java.net/browse/JDK-8240533 >>>>> webrev:http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.00/ >>>>> >>> Look good Patrick. >>> >>> The check is deliberately performed after the security manager >>> checks, right? If so, it is worth asserting this in a test. >>> >>> -Chris. >>> > From lance.andersen at oracle.com Sat Apr 4 17:00:10 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Sat, 4 Apr 2020 13:00:10 -0400 Subject: RFR[8240533]: 'Inconsistent Exceptions are thrown by DatagramSocket and DatagramChannel when sending a DatagramPacket to port 0.' In-Reply-To: References: <6ee50304-1122-64f7-91fe-d3ca6ca33237@oracle.com> <6a60e4a1-f9a5-c902-c2be-013d2ed41b40@oracle.com> <6BC55414-E001-482C-A5B2-F9ACC984A5D0@oracle.com> <0911616f-d0dd-877c-de99-1c2cd85f7b1a@oracle.com> Message-ID: <8CFED7A4-CDBA-4969-A9C4-77B574297F33@oracle.com> Hi Patrick, This looks good to me.. Best Lance > On Apr 4, 2020, at 11:26 AM, Patrick Concannon wrote: > > Hi Chris and Daniel, > > > Well spotted, Chris. Thanks for that! > > Thanks too Daniel, that's a good idea. I've made those changes and included them in a new webrev, which you can find below. > > http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.02/ > > > Kind regards, > > Patrick > > On 03/04/2020 17:16, Daniel Fuchs wrote: >> Hi Patrick, >> >> 120 { perms.add(new SocketPermission("127.0.0.1:0", >> 121 "connect,accept")); } >> 122 { perms.add(new SocketPermission("0.0.0.0:0", >> 123 "connect,accept")); } >> >> there in other tests - I think a single permission: >> >> { perms.add(new SocketPermission("*:0")); } >> >> would be more robust as it would take care of both IPv6 and IPv4 in one >> go. We should strive to avoid to hard-code 127.0.0.1 and 0.0.0.0 >> in tests. >> >> best regards, >> >> -- daniel >> >> On 03/04/2020 14:47, Patrick Concannon wrote: >>> Hi, >>> >>> Thanks for the feedback. > > >>> >>> Lance - I swapped out expectThrows for assertThrows, as requested. >>> >>> Chris - I put in an extra check in the tests to ensure that the new code doesn?t interfere with the Security Manager checks already present in the source. >>> >>> The new webrev can be found here: http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.01/ > >>> >>> >>> >>> Kind regards, >>> >>> Patrick >>> >>> On 31/03/2020 15:33, Chris Hegarty wrote: >>>> Patrick, >>>> >>>>> On 31 Mar 2020, at 15:08, Daniel Fuchs wrote: >>>>> >>>>>> .. >>>>>> bug:https://bugs.openjdk.java.net/browse/JDK-8240533 >>>>>> webrev:http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.00/ >>>> Look good Patrick. >>>> >>>> The check is deliberately performed after the security manager checks, right? If so, it is worth asserting this in a test. >>>> >>>> -Chris. >>>> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From daniel.fuchs at oracle.com Mon Apr 6 08:17:15 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 6 Apr 2020 09:17:15 +0100 Subject: RFR[8240533]: 'Inconsistent Exceptions are thrown by DatagramSocket and DatagramChannel when sending a DatagramPacket to port 0.' In-Reply-To: References: <6ee50304-1122-64f7-91fe-d3ca6ca33237@oracle.com> <6a60e4a1-f9a5-c902-c2be-013d2ed41b40@oracle.com> <6BC55414-E001-482C-A5B2-F9ACC984A5D0@oracle.com> <0911616f-d0dd-877c-de99-1c2cd85f7b1a@oracle.com> Message-ID: <5012e625-155d-9b6c-3008-d59699dbe11a@oracle.com> Looks good to me Patrick! best regards, -- daniel On 04/04/2020 16:26, Patrick Concannon wrote: > Hi Chris and Daniel, > > > Well spotted, Chris. Thanks for that! > > Thanks too Daniel, that's a good idea. I've made those changes and > included them in a new webrev, which you can find below. > > http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.02/ > > > Kind regards, > > Patrick From christoph.langer at sap.com Mon Apr 6 08:48:18 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 6 Apr 2020 08:48:18 +0000 Subject: RFR 8242006: (zipfs) Improve Zip FS FileChannel and SeekableByteChannel test coverage In-Reply-To: References: Message-ID: Hi Lance, this looks like a comprising set of tests for FileChannels and ByteChannels. Thanks for adding them. ?? I didn?t find anything significant besides some formatting nits ? no new webrev required of course: line 211, 239, 377, 1326: indentation looks not quite correct yet line 357, 385: strange linebreak in try statement line 1387: the comma should be in the line before to match the style of the rest of the file. I ran a test cycle on Linux and the added tests seem to pass. So, fine for me. Best regards Christoph From: nio-dev On Behalf Of Lance Andersen Sent: Freitag, 3. April 2020 15:24 To: nio-dev Subject: RFR 8242006: (zipfs) Improve Zip FS FileChannel and SeekableByteChannel test coverage Hi all, Please review the patch for 8242006, which improves the Zip FS test coverage for FileChannel and SeekableByteChannel. The webrev can be found at: http://cr.openjdk.java.net/~lancea/8242006/webrev.00/index.html Best, Lance [cid:image001.gif at 01D60BFF.4EB96FF0] Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 658 bytes Desc: image001.gif URL: From chris.hegarty at oracle.com Mon Apr 6 09:49:08 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 6 Apr 2020 10:49:08 +0100 Subject: RFR[8240533]: 'Inconsistent Exceptions are thrown by DatagramSocket and DatagramChannel when sending a DatagramPacket to port 0.' In-Reply-To: <5012e625-155d-9b6c-3008-d59699dbe11a@oracle.com> References: <6ee50304-1122-64f7-91fe-d3ca6ca33237@oracle.com> <6a60e4a1-f9a5-c902-c2be-013d2ed41b40@oracle.com> <6BC55414-E001-482C-A5B2-F9ACC984A5D0@oracle.com> <0911616f-d0dd-877c-de99-1c2cd85f7b1a@oracle.com> <5012e625-155d-9b6c-3008-d59699dbe11a@oracle.com> Message-ID: <5676BA5A-2D67-4DC3-B618-6583B91B44B0@oracle.com> > On 04/04/2020 16:26, Patrick Concannon wrote: >> Hi Chris and Daniel, >> Well spotted, Chris. Thanks for that! >> Thanks too Daniel, that's a good idea. I've made those changes and included them in a new webrev, which you can find below. >> http://cr.openjdk.java.net/~pconcannon/8240533/webrevs/webrev.02/ LGTM. -Chris. From david.lloyd at redhat.com Mon Apr 6 16:50:48 2020 From: david.lloyd at redhat.com (David Lloyd) Date: Mon, 6 Apr 2020 11:50:48 -0500 Subject: Small cleanup/improvement to epoll creation Message-ID: Since glibc 2.9 (released at the end of 2008), there has been an improved function for EPoll creation. I would suggest the following cleanup, which uses the "epoll_create1" function to set the FD to close-on-exec (which is generally just good practice AFAICT, even if it's not exactly one that has been universally adopted by OpenJDK thus far). It's more useful than passing in an ignored integer at any rate. Here's the suggested change: diff --git a/src/java.base/linux/native/libnio/ch/EPoll.c b/src/java.base/linux/native/libnio/ch/EPoll.c index 2100de5f215..4959a573e96 100644 --- a/src/java.base/linux/native/libnio/ch/EPoll.c +++ b/src/java.base/linux/native/libnio/ch/EPoll.c @@ -57,8 +57,7 @@ Java_sun_nio_ch_EPoll_dataOffset(JNIEnv* env, jclass clazz) JNIEXPORT jint JNICALL Java_sun_nio_ch_EPoll_create(JNIEnv *env, jclass clazz) { - /* size hint not used in modern kernels */ - int epfd = epoll_create(256); + int epfd = epoll_create1(EPOLL_CLOEXEC); if (epfd < 0) { JNU_ThrowIOExceptionWithLastError(env, "epoll_create failed"); } -- - DML From lance.andersen at oracle.com Mon Apr 6 23:41:39 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 6 Apr 2020 19:41:39 -0400 Subject: RFR 8242006: (zipfs) Improve Zip FS FileChannel and SeekableByteChannel test coverage In-Reply-To: References: Message-ID: <1DC82472-EC7A-4A41-96A8-ABEF04546EA7@oracle.com> HI Christoph Thank you for the review. I corrected the formatting issues below. I think I looked at it so many times, I just saw what I wanted to. Doing one last Mach 5 run and will push tomorrow Best Lance > On Apr 6, 2020, at 4:48 AM, Langer, Christoph wrote: > > Hi Lance, > > this looks like a comprising set of tests for FileChannels and ByteChannels. Thanks for adding them. ?? > > I didn?t find anything significant besides some formatting nits ? no new webrev required of course: > > line 211, 239, 377, 1326: indentation looks not quite correct yet > line 357, 385: strange linebreak in try statement > line 1387: the comma should be in the line before to match the style of the rest of the file. > > I ran a test cycle on Linux and the added tests seem to pass. > > So, fine for me. > > Best regards > Christoph > > From: nio-dev > On Behalf Of Lance Andersen > Sent: Freitag, 3. April 2020 15:24 > To: nio-dev > > Subject: RFR 8242006: (zipfs) Improve Zip FS FileChannel and SeekableByteChannel test coverage > > Hi all, > > Please review the patch for 8242006, which improves the Zip FS test coverage for FileChannel and SeekableByteChannel. > > The webrev can be found at: http://cr.openjdk.java.net/~lancea/8242006/webrev.00/index.html > > Best, > Lance > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From lance.andersen at oracle.com Tue Apr 7 16:18:29 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 7 Apr 2020 12:18:29 -0400 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE Message-ID: Hi all, Please review the fix for JDK-8241883, which addresses an issue with Zip FS where multiple calls to SeekableByteChannel::close results in an NPE. This issue was discovered while adding the additional Zip FS test coverage for FileChannel and SeekableByteChannel Here is the diff for the patch: ?????????? $ hg diff diff -r f4c174bf0276 src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java Tue Apr 07 09:03:05 2020 -0400 +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java Tue Apr 07 11:38:29 2020 -0400 @@ -892,6 +892,10 @@ @Override public void close() throws IOException { + // Check to see if close() has been previously called + if(!isOpen()) { + return; + } // will update the entry try (OutputStream os = getOutputStream(e)) { os.write(toByteArray()); diff -r f4c174bf0276 test/jdk/jdk/nio/zipfs/testng/test/ChannelTests.java --- a/test/jdk/jdk/nio/zipfs/testng/test/ChannelTests.java Tue Apr 07 09:03:05 2020 -0400 +++ b/test/jdk/jdk/nio/zipfs/testng/test/ChannelTests.java Tue Apr 07 11:38:29 2020 -0400 @@ -339,14 +339,12 @@ /** * Validate when SeekableByteChannel::close is called more than once, that * no error occurs - *

- * Note: this is currently disabled due to bug JDK-8241883 * * @param env Zip FS properties to use when creating the Zip file * @param compression The compression used when writing the entries * @throws Exception If an error occurs */ - @Test(dataProvider = "zipfsMap", enabled = false) + @Test(dataProvider = "zipfsMap") public void sbcCloseTest(final Map env, final int compression) throws Exception { Path zipFile = generatePath(HERE, "test", ".zip"); $ ?????????????? Mach 5 tier1 - tier3 is runs without failure Best, Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From brian.burkhalter at oracle.com Tue Apr 7 18:33:55 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 7 Apr 2020 11:33:55 -0700 Subject: 8242292: java.nio.file.FileSystems.getFileSystem(URI) should throw IAE if the URI's scheme is null Message-ID: <886D115E-C637-41E6-9E69-FC73F24D799B@oracle.com> Please review the change included below to address [1] in hope of shedding some light on [2]. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8242292 [2] https://bugs.openjdk.java.net/browse/JDK-8239569 @@ -226,6 +226,9 @@ */ public static FileSystem getFileSystem(URI uri) { String scheme = uri.getScheme(); + if (scheme == null) { + throw new IllegalArgumentException(uri.toString()); + } for (FileSystemProvider provider: FileSystemProvider.installedProviders()) { -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Tue Apr 7 18:40:55 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 7 Apr 2020 14:40:55 -0400 Subject: 8242292: java.nio.file.FileSystems.getFileSystem(URI) should throw IAE if the URI's scheme is null In-Reply-To: <886D115E-C637-41E6-9E69-FC73F24D799B@oracle.com> References: <886D115E-C637-41E6-9E69-FC73F24D799B@oracle.com> Message-ID: <1F212108-8002-4C53-83B1-34CF1E858CE5@oracle.com> +1 > On Apr 7, 2020, at 2:33 PM, Brian Burkhalter wrote: > > Please review the change included below to address [1] in hope of shedding some light on [2]. > > Thanks, > > Brian > > [1] https://bugs.openjdk.java.net/browse/JDK-8242292 > [2] https://bugs.openjdk.java.net/browse/JDK-8239569 > > > @@ -226,6 +226,9 @@ > */ > public static FileSystem getFileSystem(URI uri) { > String scheme = uri.getScheme(); > + if (scheme == null) { > + throw new IllegalArgumentException(uri.toString()); > + } > for (FileSystemProvider provider: FileSystemProvider.installedProviders()) { > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From Alan.Bateman at oracle.com Tue Apr 7 19:24:33 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 7 Apr 2020 20:24:33 +0100 Subject: 8242292: java.nio.file.FileSystems.getFileSystem(URI) should throw IAE if the URI's scheme is null In-Reply-To: <886D115E-C637-41E6-9E69-FC73F24D799B@oracle.com> References: <886D115E-C637-41E6-9E69-FC73F24D799B@oracle.com> Message-ID: <6f24739c-2e37-c8ff-e209-95bc99b14b21@oracle.com> On 07/04/2020 19:33, Brian Burkhalter wrote: > Please review the change included below to address [1] in hope of > shedding some light on [2]. The change looks okay as it's an oversight that it throws NPE rather than IAE then the URI doesn't have a scheme. We should probably update one of the existing tests for this case. -Alan From Alan.Bateman at oracle.com Tue Apr 7 19:27:23 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 7 Apr 2020 20:27:23 +0100 Subject: Small cleanup/improvement to epoll creation In-Reply-To: References: Message-ID: On 06/04/2020 17:50, David Lloyd wrote: > Since glibc 2.9 (released at the end of 2008), there has been an > improved function for EPoll creation. I would suggest the following > cleanup, which uses the "epoll_create1" function to set the FD to > close-on-exec (which is generally just good practice AFAICT, even if > it's not exactly one that has been universally adopted by OpenJDK thus > far). This looks okay, just need to update the exception message too. You are right that we don't use CLOEXEC consistently. The JDK gets away due to heroics in Process code to ensure that file descriptors are closed. -Alan From Alan.Bateman at oracle.com Tue Apr 7 19:30:09 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 7 Apr 2020 20:30:09 +0100 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: References: Message-ID: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> On 07/04/2020 17:18, Lance Andersen wrote: > Hi all, > > Please review the fix for?JDK-8241883, which addresses an issue with > Zip FS where multiple calls to SeekableByteChannel::close results in > an NPE. > > This issue was discovered while adding the additional Zip FS test > coverage for FileChannel and SeekableByteChannel > > Here is the diff for the patch: > ?????????? > $ hg diff > *diff -r f4c174bf0276 > src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java* > *--- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java ? > ? Tue Apr 07 09:03:05 2020 -0400* > *+++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java ? > ? Tue Apr 07 11:38:29 2020 -0400* > @@ -892,6 +892,10 @@ > > > ? ? ? @Override > ? ? ? public void close() throws IOException { > +? ? ? ? ? ? // Check to see if close() has been previously called > +? ? ? ? ? ? if(!isOpen()) { > +? ? ? ? ? ? ? ? return; > +? ? ? ? ? ? } > ? ? ? ? ? // will update the entry > ? ? ? ? ? try (OutputStream os = getOutputStream(e)) { > ? ? ? ? ? ? ? os.write(toByteArray()); > How robust is async close in the zip file system? Just wondering how it works when a SeekableByteChannel is being accessed in one thread when the zip file system is closed by another. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Tue Apr 7 20:11:08 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 7 Apr 2020 16:11:08 -0400 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> Message-ID: > On Apr 7, 2020, at 3:30 PM, Alan Bateman wrote: > > On 07/04/2020 17:18, Lance Andersen wrote: >> Hi all, >> >> Please review the fix for JDK-8241883, which addresses an issue with Zip FS where multiple calls to SeekableByteChannel::close results in an NPE. >> >> This issue was discovered while adding the additional Zip FS test coverage for FileChannel and SeekableByteChannel >> >> Here is the diff for the patch: >> ?????????? >> $ hg diff >> diff -r f4c174bf0276 src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java >> --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java Tue Apr 07 09:03:05 2020 -0400 >> +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java Tue Apr 07 11:38:29 2020 -0400 >> @@ -892,6 +892,10 @@ >> >> @Override >> public void close() throws IOException { >> + // Check to see if close() has been previously called >> + if(!isOpen()) { >> + return; >> + } >> // will update the entry >> try (OutputStream os = getOutputStream(e)) { >> os.write(toByteArray()); >> > How robust is async close in the zip file system? Just wondering how it works when a SeekableByteChannel is being accessed in one thread when the zip file system is closed by another. I can look to add some tests going forward as I am not sure we have any existing tests Best Lance > > -Alan. Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From brian.burkhalter at oracle.com Tue Apr 7 20:22:56 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 7 Apr 2020 13:22:56 -0700 Subject: 8242292: java.nio.file.FileSystems.getFileSystem(URI) should throw IAE if the URI's scheme is null In-Reply-To: <6f24739c-2e37-c8ff-e209-95bc99b14b21@oracle.com> References: <886D115E-C637-41E6-9E69-FC73F24D799B@oracle.com> <6f24739c-2e37-c8ff-e209-95bc99b14b21@oracle.com> Message-ID: <2633E3AD-C10A-46B9-B964-9D614379C0C0@oracle.com> > On Apr 7, 2020, at 12:24 PM, Alan Bateman wrote: > > On 07/04/2020 19:33, Brian Burkhalter wrote: >> Please review the change included below to address [1] in hope of shedding some light on [2]. > The change looks okay as it's an oversight that it throws NPE rather than IAE then the URI doesn't have a scheme. We should probably update one of the existing tests for this case. Here?s a version of the patch including a modified, existing test: http://cr.openjdk.java.net/~bpb/8242292/webrev.00/ The impl change itself is the same. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.lloyd at redhat.com Tue Apr 7 20:40:05 2020 From: david.lloyd at redhat.com (David Lloyd) Date: Tue, 7 Apr 2020 15:40:05 -0500 Subject: Small cleanup/improvement to epoll creation In-Reply-To: References: Message-ID: On Tue, Apr 7, 2020 at 2:27 PM Alan Bateman wrote: > > On 06/04/2020 17:50, David Lloyd wrote: > > Since glibc 2.9 (released at the end of 2008), there has been an > > improved function for EPoll creation. I would suggest the following > > cleanup, which uses the "epoll_create1" function to set the FD to > > close-on-exec (which is generally just good practice AFAICT, even if > > it's not exactly one that has been universally adopted by OpenJDK thus > > far). > This looks okay, just need to update the exception message too. Ah yes, I missed that. > You are right that we don't use CLOEXEC consistently. The JDK gets away due to > heroics in Process code to ensure that file descriptors are closed. I seem to recall quite a lot of discussion on this... heroics is exactly the right word :) diff --git a/src/java.base/linux/native/libnio/ch/EPoll.c b/src/java.base/linux/native/libnio/ch/EPoll.c index 2100de5f215..f41fb49c405 100644 --- a/src/java.base/linux/native/libnio/ch/EPoll.c +++ b/src/java.base/linux/native/libnio/ch/EPoll.c @@ -57,10 +57,9 @@ Java_sun_nio_ch_EPoll_dataOffset(JNIEnv* env, jclass clazz) JNIEXPORT jint JNICALL Java_sun_nio_ch_EPoll_create(JNIEnv *env, jclass clazz) { - /* size hint not used in modern kernels */ - int epfd = epoll_create(256); + int epfd = epoll_create1(EPOLL_CLOEXEC); if (epfd < 0) { - JNU_ThrowIOExceptionWithLastError(env, "epoll_create failed"); + JNU_ThrowIOExceptionWithLastError(env, "epoll_create1 failed"); } return epfd; } -- - DML From thomas.stuefe at gmail.com Wed Apr 8 04:46:17 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 8 Apr 2020 06:46:17 +0200 Subject: Small cleanup/improvement to epoll creation In-Reply-To: References: Message-ID: Looks good to me too. Thanks, Thomas On Tue, Apr 7, 2020 at 10:41 PM David Lloyd wrote: > On Tue, Apr 7, 2020 at 2:27 PM Alan Bateman > wrote: > > > > On 06/04/2020 17:50, David Lloyd wrote: > > > Since glibc 2.9 (released at the end of 2008), there has been an > > > improved function for EPoll creation. I would suggest the following > > > cleanup, which uses the "epoll_create1" function to set the FD to > > > close-on-exec (which is generally just good practice AFAICT, even if > > > it's not exactly one that has been universally adopted by OpenJDK thus > > > far). > > This looks okay, just need to update the exception message too. > > Ah yes, I missed that. > > > You are right that we don't use CLOEXEC consistently. The JDK gets away > due to > > heroics in Process code to ensure that file descriptors are closed. > > I seem to recall quite a lot of discussion on this... heroics is > exactly the right word :) > > diff --git a/src/java.base/linux/native/libnio/ch/EPoll.c > b/src/java.base/linux/native/libnio/ch/EPoll.c > index 2100de5f215..f41fb49c405 100644 > --- a/src/java.base/linux/native/libnio/ch/EPoll.c > +++ b/src/java.base/linux/native/libnio/ch/EPoll.c > @@ -57,10 +57,9 @@ Java_sun_nio_ch_EPoll_dataOffset(JNIEnv* env, jclass > clazz) > > JNIEXPORT jint JNICALL > Java_sun_nio_ch_EPoll_create(JNIEnv *env, jclass clazz) { > - /* size hint not used in modern kernels */ > - int epfd = epoll_create(256); > + int epfd = epoll_create1(EPOLL_CLOEXEC); > if (epfd < 0) { > - JNU_ThrowIOExceptionWithLastError(env, "epoll_create failed"); > + JNU_ThrowIOExceptionWithLastError(env, "epoll_create1 failed"); > } > return epfd; > } > > > -- > - DML > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Apr 8 07:11:50 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 8 Apr 2020 08:11:50 +0100 Subject: 8242292: java.nio.file.FileSystems.getFileSystem(URI) should throw IAE if the URI's scheme is null In-Reply-To: <2633E3AD-C10A-46B9-B964-9D614379C0C0@oracle.com> References: <886D115E-C637-41E6-9E69-FC73F24D799B@oracle.com> <6f24739c-2e37-c8ff-e209-95bc99b14b21@oracle.com> <2633E3AD-C10A-46B9-B964-9D614379C0C0@oracle.com> Message-ID: On 07/04/2020 21:22, Brian Burkhalter wrote: > > Here?s a version of the patch including a modified, existing test: > > http://cr.openjdk.java.net/~bpb/8242292/webrev.00/ > > The impl change itself is the same. > The test doesn't look right quite because it creates a URI from a file path, I think this is closer to what you need here: URI absoluteUri = Path.of("foo.bar").toUri(); URI relativeUri = URI.create(absoluteUri.getSchemeSpecificPart()); FileSystem fs = FileSystems.getFileSystem(relativeUri); -Alan. From Alan.Bateman at oracle.com Wed Apr 8 10:07:13 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 8 Apr 2020 11:07:13 +0100 Subject: Small cleanup/improvement to epoll creation In-Reply-To: References: Message-ID: On 07/04/2020 21:40, David Lloyd wrote: > On Tue, Apr 7, 2020 at 2:27 PM Alan Bateman wrote: >> On 06/04/2020 17:50, David Lloyd wrote: >>> Since glibc 2.9 (released at the end of 2008), there has been an >>> improved function for EPoll creation. I would suggest the following >>> cleanup, which uses the "epoll_create1" function to set the FD to >>> close-on-exec (which is generally just good practice AFAICT, even if >>> it's not exactly one that has been universally adopted by OpenJDK thus >>> far). >> This looks okay, just need to update the exception message too. > Ah yes, I missed that. > > I've pushed the change to jdk/jdk: ? http://hg.openjdk.java.net/jdk/jdk/rev/fcbb67d1ebeb From Alan.Bateman at oracle.com Wed Apr 8 10:12:34 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 8 Apr 2020 11:12:34 +0100 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> Message-ID: <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> On 07/04/2020 21:11, Lance Andersen wrote: > : > > I can look to add some tests going forward as I am not sure we have > any existing tests > The close method should be idempotent, also should be thread safe in this zipfs implementation to avoid random exceptions. If I read the proposed fix correctly then it still have timing issues when there are several thread accessing the zip file system, is that correct? -Alan From brian.burkhalter at oracle.com Wed Apr 8 15:45:35 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 8 Apr 2020 08:45:35 -0700 Subject: 8242292: java.nio.file.FileSystems.getFileSystem(URI) should throw IAE if the URI's scheme is null In-Reply-To: <2633E3AD-C10A-46B9-B964-9D614379C0C0@oracle.com> References: <886D115E-C637-41E6-9E69-FC73F24D799B@oracle.com> <6f24739c-2e37-c8ff-e209-95bc99b14b21@oracle.com> <2633E3AD-C10A-46B9-B964-9D614379C0C0@oracle.com> Message-ID: <64AEBE21-BBA7-4316-918D-CE1532E38016@oracle.com> > On Apr 7, 2020, at 1:22 PM, Brian Burkhalter wrote: > > >> On Apr 7, 2020, at 12:24 PM, Alan Bateman > wrote: >> >> On 07/04/2020 19:33, Brian Burkhalter wrote: >>> Please review the change included below to address [1] in hope of shedding some light on [2]. >> The change looks okay as it's an oversight that it throws NPE rather than IAE then the URI doesn't have a scheme. We should probably update one of the existing tests for this case. > > Here?s a version of the patch including a modified, existing test: > > http://cr.openjdk.java.net/~bpb/8242292/webrev.00/ > > The impl change itself is the same. Please see http://cr.openjdk.java.net/~bpb/8242292/webrev.01/. Test job in progress. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Wed Apr 8 19:36:24 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 8 Apr 2020 15:36:24 -0400 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> Message-ID: <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> > On Apr 8, 2020, at 6:12 AM, Alan Bateman wrote: > > On 07/04/2020 21:11, Lance Andersen wrote: >> : >> >> I can look to add some tests going forward as I am not sure we have any existing tests >> > The close method should be idempotent, also should be thread safe in this zipfs implementation to avoid random exceptions. If I read the proposed fix correctly then it still have timing issues when there are several thread accessing the zip file system, is that correct? Ok, I believe the revised patch below is in line with what you are suggesting: ?????????? $ hg diff diff -r f4c174bf0276 src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java Tue Apr 07 09:03:05 2020 -0400 +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java Wed Apr 08 11:07:30 2020 -0400 @@ -876,6 +876,7 @@ // channel is closed private class EntryOutputChannel extends ByteArrayChannel { final Entry e; + final ReadWriteLock rwlock = new ReentrantReadWriteLock(); EntryOutputChannel(Entry e) { super(e.size > 0? (int)e.size : 8192, false); @@ -892,11 +893,24 @@ @Override public void close() throws IOException { - // will update the entry - try (OutputStream os = getOutputStream(e)) { - os.write(toByteArray()); + beginWrite(); + try { + if(!isOpen()) + return; + // will update the entry + try (OutputStream os = getOutputStream(e)) { + os.write(toByteArray()); + } + super.close(); + } finally { + endWrite(); } - super.close(); + } + private final void beginWrite() { + rwlock.writeLock().lock(); + } + private final void endWrite() { + rwlock.writeLock().unlock(); } } ??????????????? Best Lance > > -Alan Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From brian.burkhalter at oracle.com Wed Apr 8 20:20:27 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 8 Apr 2020 13:20:27 -0700 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> Message-ID: <77309FA5-A40B-4E0B-B0A0-FEAE45945F45@oracle.com> > On Apr 8, 2020, at 12:36 PM, Lance Andersen wrote: > >> On Apr 8, 2020, at 6:12 AM, Alan Bateman > wrote: >> >> On 07/04/2020 21:11, Lance Andersen wrote: >>> : >>> >>> I can look to add some tests going forward as I am not sure we have any existing tests >>> >> The close method should be idempotent, also should be thread safe in this zipfs implementation to avoid random exceptions. If I read the proposed fix correctly then it still have timing issues when there are several thread accessing the zip file system, is that correct? > > > Ok, I believe the revised patch below is in line with what you are suggesting: It looks all right to me. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Apr 8 20:23:48 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 8 Apr 2020 21:23:48 +0100 Subject: 8242292: java.nio.file.FileSystems.getFileSystem(URI) should throw IAE if the URI's scheme is null In-Reply-To: <64AEBE21-BBA7-4316-918D-CE1532E38016@oracle.com> References: <886D115E-C637-41E6-9E69-FC73F24D799B@oracle.com> <6f24739c-2e37-c8ff-e209-95bc99b14b21@oracle.com> <2633E3AD-C10A-46B9-B964-9D614379C0C0@oracle.com> <64AEBE21-BBA7-4316-918D-CE1532E38016@oracle.com> Message-ID: On 08/04/2020 16:45, Brian Burkhalter wrote: > > Please see http://cr.openjdk.java.net/~bpb/8242292/webrev.01/. Test > job in progress. > I assume the "throws URISyntaxException" isn't needed, otherwise looks okay to me. -Alan From Alan.Bateman at oracle.com Wed Apr 8 20:26:12 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 8 Apr 2020 21:26:12 +0100 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> Message-ID: <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> On 08/04/2020 20:36, Lance Andersen wrote: > : > > ?????????? > $ hg diff > *diff -r f4c174bf0276 > src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java* > *--- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java ? > ? Tue Apr 07 09:03:05 2020 -0400* > *+++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java ? > ? Wed Apr 08 11:07:30 2020 -0400* > @@ -876,6 +876,7 @@ > ?? ? // channel is closed > ?? ? private class EntryOutputChannel extends ByteArrayChannel { > ?? ? ? ? final Entry e; > + ? ? final ReadWriteLock rwlock = new ReentrantReadWriteLock(); > > ?? ? ? ? EntryOutputChannel(Entry e) { > ?? ? ? ? ? ? super(e.size > 0? (int)e.size : 8192, false); > @@ -892,11 +893,24 @@ > > ?? ? ? ? @Override > ?? ? ? ? public void close() throws IOException { > - ? ? ? ? // will update the entry > - ? ? ? ? try (OutputStream os = getOutputStream(e)) { > - ? ? ? ? ? ? os.write(toByteArray()); > + ? ? ? ? beginWrite(); > + ? ? ? ? try { > + ? ? ? ? ? ? if(!isOpen()) > + ? ? ? ? ? ? ? ? return; > + ? ? ? ? ? ? // will update the entry > + ? ? ? ? ? ? try (OutputStream os = getOutputStream(e)) { > + ? ? ? ? ? ? ? ? os.write(toByteArray()); > + ? ? ? ? ? ? } > + ? ? ? ? ? ? super.close(); > + ? ? ? ? } finally { > + ? ? ? ? ? ? endWrite(); > ?? ? ? ? ? ? } > - ? ? ? ? super.close(); > + ? ? } > + ? ? private final void beginWrite() { > + ? ? ? ? rwlock.writeLock().lock(); > + ? ? } > + ? ? private final void endWrite() { > + ? ? ? ? rwlock.writeLock().unlock(); > ?? ? ? ? } > That is the equivalent of close synchronizing on any object. Are there any code paths use the readLock? -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Apr 8 20:28:41 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 8 Apr 2020 13:28:41 -0700 Subject: =?utf-8?Q?8241952=3A_=28fs=29_FileChannel=2Ewrite=E2=80=8B=28Byte?= =?utf-8?Q?Buffer_src=2C_long_position=29_does_not_check_for_the_FileChann?= =?utf-8?Q?el_being_closed_first?= Message-ID: To address [1] please see [2]. The same thing applies to FileChannel.read(ByteBuffer,long). It is not clear whether the exceptions should be expected in the order documented. If so, an alternative would be to change the documentation. I noticed that the documentation of the FileChannel methods implementing {Scattering,Gathering}ByteChannel does not inherit all the throws cases documented in these latter interfaces. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8241952 [2] http://cr.openjdk.java.net/~bpb/8241952/webrev.00/ From brian.burkhalter at oracle.com Wed Apr 8 20:34:32 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 8 Apr 2020 13:34:32 -0700 Subject: 8242292: java.nio.file.FileSystems.getFileSystem(URI) should throw IAE if the URI's scheme is null In-Reply-To: References: <886D115E-C637-41E6-9E69-FC73F24D799B@oracle.com> <6f24739c-2e37-c8ff-e209-95bc99b14b21@oracle.com> <2633E3AD-C10A-46B9-B964-9D614379C0C0@oracle.com> <64AEBE21-BBA7-4316-918D-CE1532E38016@oracle.com> Message-ID: > On Apr 8, 2020, at 1:23 PM, Alan Bateman wrote: > > On 08/04/2020 16:45, Brian Burkhalter wrote: >> >> Please see http://cr.openjdk.java.net/~bpb/8242292/webrev.01/ . Test job in progress. >> > I assume the "throws URISyntaxException" isn't needed, otherwise looks okay to me. Apparently not; webrev.01 update in place. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Wed Apr 8 20:40:30 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 8 Apr 2020 16:40:30 -0400 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> Message-ID: <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> > On Apr 8, 2020, at 4:26 PM, Alan Bateman wrote: > > > > On 08/04/2020 20:36, Lance Andersen wrote: >> : >> >> ?????????? >> $ hg diff >> diff -r f4c174bf0276 src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java >> --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java Tue Apr 07 09:03:05 2020 -0400 >> +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java Wed Apr 08 11:07:30 2020 -0400 >> @@ -876,6 +876,7 @@ >> // channel is closed >> private class EntryOutputChannel extends ByteArrayChannel { >> final Entry e; >> + final ReadWriteLock rwlock = new ReentrantReadWriteLock(); >> >> EntryOutputChannel(Entry e) { >> super(e.size > 0? (int)e.size : 8192, false); >> @@ -892,11 +893,24 @@ >> >> @Override >> public void close() throws IOException { >> - // will update the entry >> - try (OutputStream os = getOutputStream(e)) { >> - os.write(toByteArray()); >> + beginWrite(); >> + try { >> + if(!isOpen()) >> + return; >> + // will update the entry >> + try (OutputStream os = getOutputStream(e)) { >> + os.write(toByteArray()); >> + } >> + super.close(); >> + } finally { >> + endWrite(); >> } >> - super.close(); >> + } >> + private final void beginWrite() { >> + rwlock.writeLock().lock(); >> + } >> + private final void endWrite() { >> + rwlock.writeLock().unlock(); >> } >> > That is the equivalent of close synchronizing on any object. Are there any code paths use the readLock? In the case of the close method above, ByteArrayChannel::toByteArray uses a readLock ByteArrayChannel::close uses a writeLock Best Lance > > -Alan Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From lance.andersen at oracle.com Wed Apr 8 21:02:49 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 8 Apr 2020 17:02:49 -0400 Subject: =?utf-8?Q?Re=3A_8241952=3A_=28fs=29_FileChannel=2Ewrite=E2=80=8B?= =?utf-8?Q?=28ByteBuffer_src=2C_long_position=29_does_not_check_for_the_Fi?= =?utf-8?Q?leChannel_being_closed_first?= In-Reply-To: References: Message-ID: <04B65881-9033-44B9-9ACC-899BF915008E@oracle.com> Hi Brian, I am wondering if the main javadoc for FileChannel should indicate that a check to see that the channel is open is the first thing done. Thoughts? Best Lance > On Apr 8, 2020, at 4:28 PM, Brian Burkhalter wrote: > > To address [1] please see [2]. The same thing applies to FileChannel.read(ByteBuffer,long). It is not clear whether the exceptions should be expected in the order documented. If so, an alternative would be to change the documentation. > > I noticed that the documentation of the FileChannel methods implementing {Scattering,Gathering}ByteChannel does not inherit all the throws cases documented in these latter interfaces. > > Thanks, > > Brian > > [1] https://bugs.openjdk.java.net/browse/JDK-8241952 > [2] http://cr.openjdk.java.net/~bpb/8241952/webrev.00/ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From brian.burkhalter at oracle.com Wed Apr 8 21:20:05 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 8 Apr 2020 14:20:05 -0700 Subject: =?utf-8?Q?Re=3A_8241952=3A_=28fs=29_FileChannel=2Ewrite=E2=80=8B?= =?utf-8?Q?=28ByteBuffer_src=2C_long_position=29_does_not_check_for_the_Fi?= =?utf-8?Q?leChannel_being_closed_first?= In-Reply-To: <04B65881-9033-44B9-9ACC-899BF915008E@oracle.com> References: <04B65881-9033-44B9-9ACC-899BF915008E@oracle.com> Message-ID: <5D0A2A7C-7E4A-4422-8C3D-1D0D3569AFB3@oracle.com> Hi Lance, If we indeed change the location of the ensureOpen() calls in the methods in question then this might be a good idea. Thanks, Brian > On Apr 8, 2020, at 2:02 PM, Lance Andersen wrote: > > I am wondering if the main javadoc for FileChannel should indicate that a check to see that the channel is open is the first thing done. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Thu Apr 9 05:50:20 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 9 Apr 2020 05:50:20 +0000 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> Message-ID: Hey Lance, I believe the call to if(!isOpen()) return; should be placed before pulling the writeLock, that is, before beginWrite(). Then not every close call necessarily needs to synchronize. Please also note that there should be a space between ?if? and ?(?. ?? Furthermore, we should build on the already existing locking facilities of super class jdk.nio.zipfs.ByteArrayChannel. In your patch, please remove the new rwlock Object and the new methods beginWrite() and endWrite(). Make their implementations in ByteArrayChannel package private. To make it completely safe that no lock of the whole zipfs is pulled, prefix the new calls to beginWrite() and endWrite() with ?super.?. Otherwise there?s a danger that ZIPFS? beginWrite and endWrite methods are called inadvertently here when no superclass implements them. Best regards Christoph From: nio-dev On Behalf Of Lance Andersen Sent: Mittwoch, 8. April 2020 22:41 To: Alan Bateman Cc: nio-dev Subject: Re: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE On Apr 8, 2020, at 4:26 PM, Alan Bateman > wrote: On 08/04/2020 20:36, Lance Andersen wrote: : ?????????? $ hg diff diff -r f4c174bf0276 src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java Tue Apr 07 09:03:05 2020 -0400 +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java Wed Apr 08 11:07:30 2020 -0400 @@ -876,6 +876,7 @@ // channel is closed private class EntryOutputChannel extends ByteArrayChannel { final Entry e; + final ReadWriteLock rwlock = new ReentrantReadWriteLock(); EntryOutputChannel(Entry e) { super(e.size > 0? (int)e.size : 8192, false); @@ -892,11 +893,24 @@ @Override public void close() throws IOException { - // will update the entry - try (OutputStream os = getOutputStream(e)) { - os.write(toByteArray()); + beginWrite(); + try { + if(!isOpen()) + return; + // will update the entry + try (OutputStream os = getOutputStream(e)) { + os.write(toByteArray()); + } + super.close(); + } finally { + endWrite(); } - super.close(); + } + private final void beginWrite() { + rwlock.writeLock().lock(); + } + private final void endWrite() { + rwlock.writeLock().unlock(); } That is the equivalent of close synchronizing on any object. Are there any code paths use the readLock? In the case of the close method above, ByteArrayChannel::toByteArray uses a readLock ByteArrayChannel::close uses a writeLock Best Lance -Alan [cid:image001.gif at 01D60E42.446B2030] Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 658 bytes Desc: image001.gif URL: From Alan.Bateman at oracle.com Thu Apr 9 07:03:24 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 9 Apr 2020 00:03:24 -0700 (PDT) Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> Message-ID: <4f211665-eb4c-a4dc-f4d9-a3784e6f8830@oracle.com> On 08/04/2020 21:40, Lance Andersen wrote: > > >> On Apr 8, 2020, at 4:26 PM, Alan Bateman > > wrote: >> >> >> >> On 08/04/2020 20:36, Lance Andersen wrote: >>> : >>> >>> ?????????? >>> $ hg diff >>> *diff -r f4c174bf0276 >>> src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java* >>> *--- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java >>> ? ? Tue Apr 07 09:03:05 2020 -0400* >>> *+++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java >>> ? ? Wed Apr 08 11:07:30 2020 -0400* >>> @@ -876,6 +876,7 @@ >>> ?? ? // channel is closed >>> ?? ? private class EntryOutputChannel extends ByteArrayChannel { >>> ?? ? ? ? final Entry e; >>> +? ? ? ? final ReadWriteLock rwlock = new ReentrantReadWriteLock(); >>> >>> EntryOutputChannel(Entry e) { >>> ?? ? ? ? ? ? super(e.size > 0? (int)e.size : 8192, false); >>> @@ -892,11 +893,24 @@ >>> >>> ?? ? ? ? @Override >>> ?? ? ? ? public void close() throws IOException { >>> -? ? ? ? ? ? // will update the entry >>> -? ? ? ? ? ? try (OutputStream os = getOutputStream(e)) { >>> - os.write(toByteArray()); >>> +? ? ? ? ? ? beginWrite(); >>> +? ? ? ? ? ? try { >>> + if(!isOpen()) >>> + return; >>> +? ? ? ? ? ? ? ? // will update the entry >>> +? ? ? ? ? ? ? ? try (OutputStream os = getOutputStream(e)) { >>> + os.write(toByteArray()); >>> +? ? ? ? ? ? ? ? } >>> + super.close(); >>> +? ? ? ? ? ? } finally { >>> + endWrite(); >>> ?? ? ? ? ? ? } >>> - super.close(); >>> +? ? ? ? } >>> +? ? ? ? private final void beginWrite() { >>> + rwlock.writeLock().lock(); >>> +? ? ? ? } >>> +? ? ? ? private final void endWrite() { >>> + rwlock.writeLock().unlock(); >>> ?? ? ? ? } >>> >> That is the equivalent of close synchronizing on any object. Are >> there any code paths use the readLock? > > In the case of the close method above, ByteArrayChannel::toByteArray > uses a readLock ?ByteArrayChannel::close uses a writeLock > The patch adds a new lock and doesn't appear use the readLock. Would it be possible to generate a webrev with the full patch as it's not possible to see the fully implications of what is proposed here. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Apr 9 12:00:46 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 9 Apr 2020 13:00:46 +0100 Subject: =?UTF-8?B?UmU6IDgyNDE5NTI6IChmcykgRmlsZUNoYW5uZWwud3JpdGXigIsoQnl0?= =?UTF-8?Q?eBuffer_src=2c_long_position=29_does_not_check_for_the_FileChanne?= =?UTF-8?Q?l_being_closed_first?= In-Reply-To: References: Message-ID: On 08/04/2020 21:28, Brian Burkhalter wrote: > To address [1] please see [2]. The same thing applies to FileChannel.read(ByteBuffer,long). It is not clear whether the exceptions should be expected in the order documented. If so, an alternative would be to change the documentation. > > I noticed that the documentation of the FileChannel methods implementing {Scattering,Gathering}ByteChannel does not inherit all the throws cases documented in these latter interfaces. > Keeping it consistent is good and the changes in the webrev look okay. The ordering of the exceptions in the javadoc should never be interpreted as the ordering of checks when there are several possible error conditions. I don't see anything in the webrev to inherit the text from super interfaces, are doing that with a separate issue? -Alan From brian.burkhalter at oracle.com Thu Apr 9 16:10:32 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 9 Apr 2020 09:10:32 -0700 Subject: =?utf-8?Q?Re=3A_8241952=3A_=28fs=29_FileChannel=2Ewrite=E2=80=8B?= =?utf-8?Q?=28ByteBuffer_src=2C_long_position=29_does_not_check_for_the_Fi?= =?utf-8?Q?leChannel_being_closed_first?= In-Reply-To: References: Message-ID: <3531DEF8-ED9B-4EE2-9F67-9C7095DF8BD8@oracle.com> > On Apr 9, 2020, at 5:00 AM, Alan Bateman wrote: > > I don't see anything in the webrev to inherit the text from super interfaces, are doing that with a separate issue? There is no separate issue yet. An example of what I am talking about is FileChannel.read(ByteBuffer[] dsts,int,int). Its javadoc [1] does not have the same exceptions as the specification in ScatteringByteChannel [2]. It does have a subset of the exceptions in the doc but there are none in the FileChannel code itself [3] nor anything about inheriting doc. Brian [1] https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/nio/channels/FileChannel.html#read(java.nio.ByteBuffer[],int,int) [2] https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/nio/channels/ScatteringByteChannel.html#read(java.nio.ByteBuffer%5B%5D,int,int) [3] http://hg.openjdk.java.net/jdk/jdk14/file/6c954123ee8d/src/java.base/share/classes/java/nio/channels/FileChannel.java#l360 /** * Reads a sequence of bytes from this channel into a subsequence of the * given buffers. * *

Bytes are read starting at this channel's current file position, and * then the file position is updated with the number of bytes actually * read. Otherwise this method behaves exactly as specified in the {@link * ScatteringByteChannel} interface.

*/ public abstract long read(ByteBuffer[] dsts, int offset, int length) throws IOException; -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Thu Apr 9 18:19:43 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 9 Apr 2020 14:19:43 -0400 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> Message-ID: Hi Christoph, Thank you for the feedback as always! > On Apr 9, 2020, at 1:50 AM, Langer, Christoph wrote: > > Hey Lance, > > I believe the call to > > if(!isOpen()) > return; > > should be placed before pulling the writeLock, that is, before beginWrite(). I kept this within the beginWrite() to keep it consistent with how ensureOpen is used. If the final consensus is to move it, I will. > Then not every close call necessarily needs to synchronize. Please also note that there should be a space between ?if? and ?(?. ?? Done. > > Furthermore, we should build on the already existing locking facilities of super class jdk.nio.zipfs.ByteArrayChannel. In your patch, please remove the new rwlock Object and the new methods beginWrite() and endWrite(). Make their implementations in ByteArrayChannel package private. To make it completely safe that no lock of the whole zipfs is pulled, prefix the new calls to beginWrite() and endWrite() with ?super.?. Otherwise there?s a danger that ZIPFS? beginWrite and endWrite methods are called inadvertently here when no superclass implements them. I had planned to do that originally, but for some reason I talked myself out of it but per your suggestion I reversed my original decision :-) The webrev can be found at: http://cr.openjdk.java.net/~lancea/8241883/webrev.00/index.html Best Lance > > Best regards > Christoph > > From: nio-dev > On Behalf Of Lance Andersen > Sent: Mittwoch, 8. April 2020 22:41 > To: Alan Bateman > > Cc: nio-dev > > Subject: Re: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE > > > > > On Apr 8, 2020, at 4:26 PM, Alan Bateman > wrote: > > > > On 08/04/2020 20:36, Lance Andersen wrote: > : > > ?????????? > $ hg diff > diff -r f4c174bf0276 src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java > --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java Tue Apr 07 09:03:05 2020 -0400 > +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java Wed Apr 08 11:07:30 2020 -0400 > @@ -876,6 +876,7 @@ > // channel is closed > private class EntryOutputChannel extends ByteArrayChannel { > final Entry e; > + final ReadWriteLock rwlock = new ReentrantReadWriteLock(); > > EntryOutputChannel(Entry e) { > super(e.size > 0? (int)e.size : 8192, false); > @@ -892,11 +893,24 @@ > > @Override > public void close() throws IOException { > - // will update the entry > - try (OutputStream os = getOutputStream(e)) { > - os.write(toByteArray()); > + beginWrite(); > + try { > + if(!isOpen()) > + return; > + // will update the entry > + try (OutputStream os = getOutputStream(e)) { > + os.write(toByteArray()); > + } > + super.close(); > + } finally { > + endWrite(); > } > - super.close(); > + } > + private final void beginWrite() { > + rwlock.writeLock().lock(); > + } > + private final void endWrite() { > + rwlock.writeLock().unlock(); > } > > That is the equivalent of close synchronizing on any object. Are there any code paths use the readLock? > > In the case of the close method above, ByteArrayChannel::toByteArray uses a readLock ByteArrayChannel::close uses a writeLock > > Best > Lance > > > -Alan > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From lance.andersen at oracle.com Thu Apr 9 18:20:00 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 9 Apr 2020 14:20:00 -0400 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: <4f211665-eb4c-a4dc-f4d9-a3784e6f8830@oracle.com> References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> <4f211665-eb4c-a4dc-f4d9-a3784e6f8830@oracle.com> Message-ID: > On Apr 9, 2020, at 3:03 AM, Alan Bateman wrote: > >> >> In the case of the close method above, ByteArrayChannel::toByteArray uses a readLock ByteArrayChannel::close uses a writeLock >> > The patch adds a new lock and doesn't appear use the readLock. Would it be possible to generate a webrev with the full patch as it's not possible to see the fully implications of what is proposed here. Of course, and can be found at: http://cr.openjdk.java.net/~lancea/8241883/webrev.00/index.html I modified the patch to use the beginWrite() from ByteArrayChannel per Christoph?s suggestion (I had done that originally but talked myself out of that for some reason ?) > > -Alan Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From christoph.langer at sap.com Thu Apr 9 20:20:01 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 9 Apr 2020 20:20:01 +0000 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> Message-ID: Hi Lance, From: Lance Andersen Sent: Donnerstag, 9. April 2020 20:20 To: Langer, Christoph Cc: Alan Bateman ; nio-dev Subject: Re: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE Hi Christoph, Thank you for the feedback as always! You?re welcome, as always ?? On Apr 9, 2020, at 1:50 AM, Langer, Christoph > wrote: Hey Lance, I believe the call to if(!isOpen()) return; should be placed before pulling the writeLock, that is, before beginWrite(). I kept this within the beginWrite() to keep it consistent with how ensureOpen is used. If the final consensus is to move it, I will. TL;DR: You?re right, keep it! Full story: Yes that?s safer. Only checking isOpen() before the lock is not correct. It would probably be a scenario for double checked locking. You could check for isOpen() once outside the lock but you have to as well check for it again after acquiring the lock. Otherwise it can happen that the actual close logic runs more than once. On the other hand, maybe that?s a bit overkill, assuming that a scenario of having multiple threads close an EntryOutputChannel at the same time occurs rather seldom. BTW: ByteArrayChannel::close does the ?mistake? of only checking for closed before pulling the lock, too. But there closing only means nulling some fields. So if that runs multiple times it?s not an issue. Then not every close call necessarily needs to synchronize. Please also note that there should be a space between ?if? and ?(?. ?? Done. Perfect. Furthermore, we should build on the already existing locking facilities of super class jdk.nio.zipfs.ByteArrayChannel. In your patch, please remove the new rwlock Object and the new methods beginWrite() and endWrite(). Make their implementations in ByteArrayChannel package private. To make it completely safe that no lock of the whole zipfs is pulled, prefix the new calls to beginWrite() and endWrite() with ?super.?. Otherwise there?s a danger that ZIPFS? beginWrite and endWrite methods are called inadvertently here when no superclass implements them. I had planned to do that originally, but for some reason I talked myself out of it but per your suggestion I reversed my original decision :-) Great. The webrev can be found at: http://cr.openjdk.java.net/~lancea/8241883/webrev.00/index.html I?m fine with it. Best regards Christoph -------------- next part -------------- An HTML attachment was scrubbed... URL: From macanaoire at hotmail.com Thu Apr 9 20:51:29 2020 From: macanaoire at hotmail.com (mark sheppard) Date: Thu, 9 Apr 2020 20:51:29 +0000 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> <4f211665-eb4c-a4dc-f4d9-a3784e6f8830@oracle.com>, Message-ID: Hi Lance, In the ByteArrayChannel is there any merit in making the readonly member variable final to exhibit its immutable characteristic i.e. it appears to be set during construction only, and the closed member variable volatile to ensure consistency of reads for concurrent isOpen and closed method invocations ? regards Mark ________________________________ From: nio-dev on behalf of Lance Andersen Sent: Thursday 9 April 2020 18:20 To: Alan Bateman Cc: nio-dev Subject: Re: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE On Apr 9, 2020, at 3:03 AM, Alan Bateman > wrote: In the case of the close method above, ByteArrayChannel::toByteArray uses a readLock ByteArrayChannel::close uses a writeLock The patch adds a new lock and doesn't appear use the readLock. Would it be possible to generate a webrev with the full patch as it's not possible to see the fully implications of what is proposed here. Of course, and can be found at: http://cr.openjdk.java.net/~lancea/8241883/webrev.00/index.html I modified the patch to use the beginWrite() from ByteArrayChannel per Christoph?s suggestion (I had done that originally but talked myself out of that for some reason ?) -Alan [cid:E1C4E2F0-ECD0-4C9D-ADB4-B16CA7BCB7FC at home] Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: oracle_sig_logo.gif URL: From lance.andersen at oracle.com Thu Apr 9 22:08:53 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 9 Apr 2020 18:08:53 -0400 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> <4f211665-eb4c-a4dc-f4d9-a3784e6f8830@oracle.com> Message-ID: Hi Mark, Thank you for the feedback. > On Apr 9, 2020, at 4:51 PM, mark sheppard wrote: > > > Hi Lance, > In the ByteArrayChannel is there any merit in making > the readonly member variable final to exhibit its immutable characteristic Yes, I would agree and Intellij analyze code also suggests that along with removing final from begin/endRead as the methods are final. This might be better to do in a separate cleanup > i.e. it appears to be set during construction only, > and the closed member variable volatile to ensure consistency of reads > for concurrent isOpen and closed method invocations ? It might be better to use begin/endRead calls in the isOpen method. Best Lance > > regards > Mark > > > > From: nio-dev on behalf of Lance Andersen > Sent: Thursday 9 April 2020 18:20 > To: Alan Bateman > Cc: nio-dev > Subject: Re: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE > > > >> On Apr 9, 2020, at 3:03 AM, Alan Bateman > wrote: >> >>> >>> In the case of the close method above, ByteArrayChannel::toByteArray uses a readLock ByteArrayChannel::close uses a writeLock >>> >> The patch adds a new lock and doesn't appear use the readLock. Would it be possible to generate a webrev with the full patch as it's not possible to see the fully implications of what is proposed here. > > Of course, and can be found at: http://cr.openjdk.java.net/~lancea/8241883/webrev.00/index.html > > I modified the patch to use the beginWrite() from ByteArrayChannel per Christoph?s suggestion (I had done that originally but talked myself out of that for some reason ?) >> >> -Alan > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From lance.andersen at oracle.com Thu Apr 9 22:09:19 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 9 Apr 2020 18:09:19 -0400 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> Message-ID: <7D8486BF-6C6C-4FC6-8623-711B1B3E49B6@oracle.com> Hi Christoph, Thank you for the review. Best Lance > On Apr 9, 2020, at 4:20 PM, Langer, Christoph wrote: > > Hi Lance, > > From: Lance Andersen > > Sent: Donnerstag, 9. April 2020 20:20 > To: Langer, Christoph > > Cc: Alan Bateman >; nio-dev > > Subject: Re: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE > > Hi Christoph, > > > Thank you for the feedback as always! > > > You?re welcome, as always ?? > > > On Apr 9, 2020, at 1:50 AM, Langer, Christoph > wrote: > > Hey Lance, > > I believe the call to > > if(!isOpen()) > return; > > should be placed before pulling the writeLock, that is, before beginWrite(). > > I kept this within the beginWrite() to keep it consistent with how ensureOpen is used. If the final consensus is to move it, I will. > > TL;DR: You?re right, keep it! > Full story: Yes that?s safer. Only checking isOpen() before the lock is not correct. It would probably be a scenario for double checked locking. You could check for isOpen() once outside the lock but you have to as well check for it again after acquiring the lock. Otherwise it can happen that the actual close logic runs more than once. On the other hand, maybe that?s a bit overkill, assuming that a scenario of having multiple threads close an EntryOutputChannel at the same time occurs rather seldom. > BTW: ByteArrayChannel::close does the ?mistake? of only checking for closed before pulling the lock, too. But there closing only means nulling some fields. So if that runs multiple times it?s not an issue. > > > > Then not every close call necessarily needs to synchronize. Please also note that there should be a space between ?if? and ?(?. ?? > > Done. > > > Perfect. > > > Furthermore, we should build on the already existing locking facilities of super class jdk.nio.zipfs.ByteArrayChannel. In your patch, please remove the new rwlock Object and the new methods beginWrite() and endWrite(). Make their implementations in ByteArrayChannel package private. To make it completely safe that no lock of the whole zipfs is pulled, prefix the new calls to beginWrite() and endWrite() with ?super.?. Otherwise there?s a danger that ZIPFS? beginWrite and endWrite methods are called inadvertently here when no superclass implements them. > > I had planned to do that originally, but for some reason I talked myself out of it but per your suggestion I reversed my original decision :-) > > Great. > > > The webrev can be found at: http://cr.openjdk.java.net/~lancea/8241883/webrev.00/index.html > > I?m fine with it. > > Best regards > Christoph > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From brian.burkhalter at oracle.com Thu Apr 9 22:26:23 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 9 Apr 2020 15:26:23 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings Message-ID: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> To fix [1] this additional verbiage is proposed for the class specification of MappedByteBuffer: @@ -58,6 +58,9 @@ * avoid the manipulation of a mapped file by this program, or by a * concurrently running program, except to read or write the file's content. * + *

The result of a bulk copy between two {@code MappedByteBuffer}s is + * unspecified if they are mapped from overlapping regions of the same file. + * *

Mapped byte buffers otherwise behave no differently than ordinary direct * byte buffers.

Please refer also to [2]. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8242477 [2] https://bugs.openjdk.java.net/browse/JDK-8242253 -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Fri Apr 10 00:44:37 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 9 Apr 2020 17:44:37 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> Message-ID: <392EC22A-6A91-475E-A7E9-389A5880009C@oracle.com> +1 Paul. > On Apr 9, 2020, at 3:26 PM, Brian Burkhalter wrote: > > To fix [1] this additional verbiage is proposed for the class specification of MappedByteBuffer: > > @@ -58,6 +58,9 @@ > * avoid the manipulation of a mapped file by this program, or by a > * concurrently running program, except to read or write the file's content. > * > + *

The result of a bulk copy between two {@code MappedByteBuffer}s is > + * unspecified if they are mapped from overlapping regions of the same file. > + * > *

Mapped byte buffers otherwise behave no differently than ordinary direct > * byte buffers.

> > Please refer also to [2]. > > Thanks, > > Brian > > [1] https://bugs.openjdk.java.net/browse/JDK-8242477 > [2] https://bugs.openjdk.java.net/browse/JDK-8242253 -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Apr 10 07:23:02 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 10 Apr 2020 08:23:02 +0100 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> <4f211665-eb4c-a4dc-f4d9-a3784e6f8830@oracle.com> Message-ID: <83af2472-1763-3a07-e830-7a3544b64d24@oracle.com> On 09/04/2020 19:20, Lance Andersen wrote: > > Of course, and can be found at: > http://cr.openjdk.java.net/~lancea/8241883/webrev.00/index.html > > I modified the patch to use the beginWrite() from ByteArrayChannel per > Christoph?s suggestion (I had done that originally but talked myself > out of that for some reason ?) Thanks for this, the inline patch didn't look right because it didn't coordinate the locking with the super class but this version seems to do it right. So I think it looks good. -Alan. From Alan.Bateman at oracle.com Fri Apr 10 11:10:31 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 10 Apr 2020 12:10:31 +0100 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> Message-ID: On 09/04/2020 23:26, Brian Burkhalter wrote: > To fix [1] this additional verbiage is proposed for the class > specification of MappedByteBuffer: > > @@ -58,6 +58,9 @@ > ? * avoid the manipulation of a mapped file by this program, or by a > ? * concurrently running program, except to read or write the file's > content. > ? * > + *

The result of a bulk copy between two {@code MappedByteBuffer}s is > + * unspecified if they are mapped from overlapping regions of the > same file. > + * > ? *

Mapped byte buffers otherwise behave no differently than > ordinary direct > ? * byte buffers.

> Looks good, I think this is the right place in the class description for this. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Fri Apr 10 13:44:44 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 10 Apr 2020 09:44:44 -0400 Subject: RFR: JDK-8241883: (zipfs) SeekableByteChannel:close followed by SeekableByteChannel:close will throw an NPE In-Reply-To: <83af2472-1763-3a07-e830-7a3544b64d24@oracle.com> References: <65baa1d1-b99b-8e93-1395-96a82b529fe8@oracle.com> <3edd3a33-7dba-b47f-9538-22654e7a8e12@oracle.com> <190F55AD-7BBE-4173-BDB5-73BF22FB31B7@oracle.com> <89b2a258-877e-ab01-e247-d574fbce595b@oracle.com> <349AC75D-0449-4C51-B147-C938498F4C34@oracle.com> <4f211665-eb4c-a4dc-f4d9-a3784e6f8830@oracle.com> <83af2472-1763-3a07-e830-7a3544b64d24@oracle.com> Message-ID: <5ADBFE37-60EB-476E-A8FF-6AA1829D00B5@oracle.com> Thank you Alan. I will push this back today. Best Lance > On Apr 10, 2020, at 3:23 AM, Alan Bateman wrote: > > On 09/04/2020 19:20, Lance Andersen wrote: >> >> Of course, and can be found at: http://cr.openjdk.java.net/~lancea/8241883/webrev.00/index.html >> >> I modified the patch to use the beginWrite() from ByteArrayChannel per Christoph?s suggestion (I had done that originally but talked myself out of that for some reason ?) > Thanks for this, the inline patch didn't look right because it didn't coordinate the locking with the super class but this version seems to do it right. So I think it looks good. > > -Alan. Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From brian.burkhalter at oracle.com Fri Apr 10 16:47:59 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 10 Apr 2020 09:47:59 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> Message-ID: <1387EAA0-AA5B-463C-B4B0-69BC68BCF9B2@oracle.com> > On Apr 10, 2020, at 4:10 AM, Alan Bateman wrote: > > On 09/04/2020 23:26, Brian Burkhalter wrote: >> To fix [1] this additional verbiage is proposed for the class specification of MappedByteBuffer: >> >> @@ -58,6 +58,9 @@ >> * avoid the manipulation of a mapped file by this program, or by a >> * concurrently running program, except to read or write the file's content. >> * >> + *

The result of a bulk copy between two {@code MappedByteBuffer}s is >> + * unspecified if they are mapped from overlapping regions of the same file. >> + * >> *

Mapped byte buffers otherwise behave no differently than ordinary direct >> * byte buffers.

>> > Looks good, I think this is the right place in the class description for this. CSR filed: https://bugs.openjdk.java.net/browse/JDK-8242500. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From fw at deneb.enyo.de Sat Apr 11 11:23:13 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Sat, 11 Apr 2020 13:23:13 +0200 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> (Brian Burkhalter's message of "Thu, 9 Apr 2020 15:26:23 -0700") References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> Message-ID: <874ktq9sgu.fsf@mid.deneb.enyo.de> * Brian Burkhalter: > To fix [1] this additional verbiage is proposed for the class specification of MappedByteBuffer: > > @@ -58,6 +58,9 @@ > * avoid the manipulation of a mapped file by this program, or by a > * concurrently running program, except to read or write the file's content. > * > + *

The result of a bulk copy between two {@code MappedByteBuffer}s is > + * unspecified if they are mapped from overlapping regions of the same file. > + * > *

Mapped byte buffers otherwise behave no differently than ordinary direct > * byte buffers.

> > Please refer also to [2]. In principle, this also applies to direct buffer created via JNI's NewDirectByteBuffer. Such direct buffers could alias with each other or even MappedByteBuffer instances. From brian.burkhalter at oracle.com Mon Apr 13 15:42:40 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 13 Apr 2020 08:42:40 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: <874ktq9sgu.fsf@mid.deneb.enyo.de> References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> <874ktq9sgu.fsf@mid.deneb.enyo.de> Message-ID: > On Apr 11, 2020, at 4:23 AM, Florian Weimer wrote: > > In principle, this also applies to direct buffer created via JNI's > NewDirectByteBuffer. Such direct buffers could alias with each other > or even MappedByteBuffer instances. Indeed. And in an even simpler case, two different views of the same direct buffer could produce unexpected results with a relative bulk put from one into the other. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon Apr 13 15:49:25 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 13 Apr 2020 16:49:25 +0100 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> <874ktq9sgu.fsf@mid.deneb.enyo.de> Message-ID: <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> On 13/04/2020 16:42, Brian Burkhalter wrote: > >> On Apr 11, 2020, at 4:23 AM, Florian Weimer > > wrote: >> >> In principle, this also applies to direct buffer created via JNI's >> NewDirectByteBuffer. ?Such direct buffers could alias with each other >> or even MappedByteBuffer instances. > > Indeed. And in an even simpler case, two different views of the same > direct buffer could produce unexpected results with a relative bulk > put from one into the other. > Yes, it's a good point. Can you check out whether the statement could be added to the BB "Direct vs. non-direct buffer" sections. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Mon Apr 13 15:50:27 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 13 Apr 2020 08:50:27 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> <874ktq9sgu.fsf@mid.deneb.enyo.de> <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> Message-ID: <815DB032-A519-4E62-BBE7-A3665C3A751A@oracle.com> > On Apr 13, 2020, at 8:49 AM, Alan Bateman wrote: > > On 13/04/2020 16:42, Brian Burkhalter wrote: >> >>> On Apr 11, 2020, at 4:23 AM, Florian Weimer > wrote: >>> >>> In principle, this also applies to direct buffer created via JNI's >>> NewDirectByteBuffer. Such direct buffers could alias with each other >>> or even MappedByteBuffer instances. >> >> Indeed. And in an even simpler case, two different views of the same direct buffer could produce unexpected results with a relative bulk put from one into the other. >> > Yes, it's a good point. I verified it with a small test. > Can you check out whether the statement could be added to the BB "Direct vs. non-direct buffer" sections. Sure. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Mon Apr 13 16:14:19 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 13 Apr 2020 09:14:19 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: <815DB032-A519-4E62-BBE7-A3665C3A751A@oracle.com> References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> <874ktq9sgu.fsf@mid.deneb.enyo.de> <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> <815DB032-A519-4E62-BBE7-A3665C3A751A@oracle.com> Message-ID: <6E2AE464-9950-4601-9E36-3976D07CE88B@oracle.com> > On Apr 13, 2020, at 8:50 AM, Brian Burkhalter wrote: > > > >> On Apr 13, 2020, at 8:49 AM, Alan Bateman > wrote: >> >> On 13/04/2020 16:42, Brian Burkhalter wrote: >>> >>>> On Apr 11, 2020, at 4:23 AM, Florian Weimer > wrote: >>>> >>>> In principle, this also applies to direct buffer created via JNI's >>>> NewDirectByteBuffer. Such direct buffers could alias with each other >>>> or even MappedByteBuffer instances. >>> >>> Indeed. And in an even simpler case, two different views of the same direct buffer could produce unexpected results with a relative bulk put from one into the other. >>> >> Yes, it's a good point. > > I verified it with a small test. > Is that due to the buffer implementation (in some cases not using the Unsafe mem-copy) rather than mapping two sources at different virtual memory addresses? If so it may be possible to fix it. Paul. >> Can you check out whether the statement could be added to the BB "Direct vs. non-direct buffer" sections. > > Sure. > > Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Mon Apr 13 16:19:39 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 13 Apr 2020 09:19:39 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: <6E2AE464-9950-4601-9E36-3976D07CE88B@oracle.com> References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> <874ktq9sgu.fsf@mid.deneb.enyo.de> <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> <815DB032-A519-4E62-BBE7-A3665C3A751A@oracle.com> <6E2AE464-9950-4601-9E36-3976D07CE88B@oracle.com> Message-ID: <9CC519BA-244A-46D6-9A7B-C944FAF6A8C5@oracle.com> > On Apr 13, 2020, at 9:14 AM, Paul Sandoz wrote: > >>>> Indeed. And in an even simpler case, two different views of the same direct buffer could produce unexpected results with a relative bulk put from one into the other. >>>> >>> Yes, it's a good point. >> >> I verified it with a small test. >> > > Is that due to the buffer implementation (in some cases not using the Unsafe mem-copy) rather than mapping two sources at different virtual memory addresses? > > If so it may be possible to fix it. The test used two float views, one little endian and one big endian, of the same direct ByteBuffer and did a relative put (ByteBuffer.put(ByteBuffer)) from one into the other. This devolves into the loopy default code more or less as while (src.hasRemaining()) dst.put(src.get()); Since the two are views there is no IAE although both refer to the same storage. As there is currently no way to check whether these refer eventually to the same storage I don?t think there?s a way to get to using Unsafe. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Mon Apr 13 16:51:10 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 13 Apr 2020 09:51:10 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: <9CC519BA-244A-46D6-9A7B-C944FAF6A8C5@oracle.com> References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> <874ktq9sgu.fsf@mid.deneb.enyo.de> <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> <815DB032-A519-4E62-BBE7-A3665C3A751A@oracle.com> <6E2AE464-9950-4601-9E36-3976D07CE88B@oracle.com> <9CC519BA-244A-46D6-9A7B-C944FAF6A8C5@oracle.com> Message-ID: Ok. I was wondering about the behavior of Unsafe.copySwapMemory. Buffers have a base + address that should be accessible for all kinds of buffer. Paul. > On Apr 13, 2020, at 9:19 AM, Brian Burkhalter wrote: > > >> On Apr 13, 2020, at 9:14 AM, Paul Sandoz > wrote: >> >>>>> Indeed. And in an even simpler case, two different views of the same direct buffer could produce unexpected results with a relative bulk put from one into the other. >>>>> >>>> Yes, it's a good point. >>> >>> I verified it with a small test. >>> >> >> Is that due to the buffer implementation (in some cases not using the Unsafe mem-copy) rather than mapping two sources at different virtual memory addresses? >> >> If so it may be possible to fix it. > > The test used two float views, one little endian and one big endian, of the same direct ByteBuffer and did a relative put (ByteBuffer.put(ByteBuffer)) from one into the other. This devolves into the loopy default code more or less as > while (src.hasRemaining()) > dst.put(src.get()); > Since the two are views there is no IAE although both refer to the same storage. As there is currently no way to check whether these refer eventually to the same storage I don?t think there?s a way to get to using Unsafe. > > Brian > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Mon Apr 13 16:54:21 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 13 Apr 2020 09:54:21 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> <874ktq9sgu.fsf@mid.deneb.enyo.de> <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> <815DB032-A519-4E62-BBE7-A3665C3A751A@oracle.com> <6E2AE464-9950-4601-9E36-3976D07CE88B@oracle.com> <9CC519BA-244A-46D6-9A7B-C944FAF6A8C5@oracle.com> Message-ID: <1C3958DD-572C-4D22-9D16-4573C803DB4C@oracle.com> > On Apr 13, 2020, at 9:51 AM, Paul Sandoz wrote: > > Ok. I was wondering about the behavior of Unsafe.copySwapMemory. I have tried this in a prototype patch for absolute bulk put and it does work but does not handle the view case described at least not without further effort. > Buffers have a base + address that should be accessible for all kinds of buffer. I?ll take another look. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Mon Apr 13 19:23:58 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 13 Apr 2020 12:23:58 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> <874ktq9sgu.fsf@mid.deneb.enyo.de> <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> Message-ID: <8D93D1C2-3C3D-4384-9368-C284EB18E05B@oracle.com> > On Apr 13, 2020, at 8:49 AM, Alan Bateman wrote: > > On 13/04/2020 16:42, Brian Burkhalter wrote: >> >>> On Apr 11, 2020, at 4:23 AM, Florian Weimer > wrote: >>> >>> In principle, this also applies to direct buffer created via JNI's >>> NewDirectByteBuffer. Such direct buffers could alias with each other >>> or even MappedByteBuffer instances. >> >> Indeed. And in an even simpler case, two different views of the same direct buffer could produce unexpected results with a relative bulk put from one into the other. >> > Yes, it's a good point. Can you check out whether the statement could be added to the BB "Direct vs. non-direct buffer" sections. Something like this maybe could be appended to the penultimate paragraph in ByteBuffer's direct v. indirect section: "The result of a bulk copy between two direct buffers is unspecified if they correspond to overlapping regions of the same native storage.? Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Apr 23 02:08:24 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 22 Apr 2020 19:08:24 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> <874ktq9sgu.fsf@mid.deneb.enyo.de> <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> Message-ID: <36BCDCB6-027E-4DB0-87B0-E2A1CC7E7D62@oracle.com> > On Apr 13, 2020, at 8:49 AM, Alan Bateman wrote: > > On 13/04/2020 16:42, Brian Burkhalter wrote: >> >>> On Apr 11, 2020, at 4:23 AM, Florian Weimer > wrote: >>> >>> In principle, this also applies to direct buffer created via JNI's >>> NewDirectByteBuffer. Such direct buffers could alias with each other >>> or even MappedByteBuffer instances. >> >> Indeed. And in an even simpler case, two different views of the same direct buffer could produce unexpected results with a relative bulk put from one into the other. >> > Yes, it's a good point. Can you check out whether the statement could be added to the BB "Direct vs. non-direct buffer" sections. I updated the proposed change to include such a statement which is included below. I?ll update the CSR [1] accordingly if this looks good. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8242500 --- a/src/java.base/share/classes/java/nio/MappedByteBuffer.java +++ b/src/java.base/share/classes/java/nio/MappedByteBuffer.java @@ -58,6 +58,9 @@ * avoid the manipulation of a mapped file by this program, or by a * concurrently running program, except to read or write the file's content. * + *

The result of a bulk copy between two {@code MappedByteBuffer}s is + * unspecified if they are mapped from overlapping regions of the same file. + * *

Mapped byte buffers otherwise behave no differently than ordinary direct * byte buffers.

* --- a/src/java.base/share/classes/java/nio/X-Buffer.java.template +++ b/src/java.base/share/classes/java/nio/X-Buffer.java.template @@ -125,7 +125,9 @@ * instance of one of these kinds of buffers refers to an inaccessible region * of memory then an attempt to access that region will not change the buffer's * content and will cause an unspecified exception to be thrown either at the - * time of the access or at some later time. + * time of the access or at some later time. The result of a bulk copy between + * two direct buffers created via JNI is unspecified if they refer to + * overlapping regions of memory. * *

Whether a byte buffer is direct or non-direct may be determined by * invoking its {@link #isDirect isDirect} method. This method is provided so -------------- next part -------------- An HTML attachment was scrubbed... URL: From luke.hutch at gmail.com Thu Apr 23 15:16:13 2020 From: luke.hutch at gmail.com (Luke Hutchison) Date: Thu, 23 Apr 2020 09:16:13 -0600 Subject: ByteBuffer cannot be extended because constructors have default visibility Message-ID: The ByteBuffer class cannot be extended, because its two constructors both have default visibility. This is problematic, since there are numerous situations where it would be useful to be able to extend ByteBuffer. For example, I needed to create a ByteBuffer that was backed by a RandomAccessFile (without actually mapping the file to a MappedByteBuffer), so that I could use a RandomAccessFile with a ByteBuffer API. As far as I can tell, there is no way to do this. This led to a huge amount of unnecessary work and duplicated code, since methods that accepted ByteBuffer now needed to have a duplicated alternative that accepted RandomAccessFile. I just came across another example of this -- Google's Jimfs (in-memory filesystem) has run into the same problem. Currently if you call FileChannel.open(path) on a Jimfs path, then call map() on the file channel to mmap it, Jimfs throws an UnsupportedOperationException. This is due to ByteBuffer not being extendable: https://github.com/google/jimfs/issues/54 This seems like an unnecessary restriction, based on the source of ByteBuffer. Can the ByteBuffer constructors please be made public? -------------- next part -------------- An HTML attachment was scrubbed... URL: From philippe.marschall at gmail.com Thu Apr 23 15:55:41 2020 From: philippe.marschall at gmail.com (Philippe Marschall) Date: Thu, 23 Apr 2020 17:55:41 +0200 Subject: ByteBuffer cannot be extended because constructors have default visibility In-Reply-To: References: Message-ID: Hello Luke I was told this would lead to massive performance degradation due to C2 deoptimization events. Cheers Philippe On Thu, Apr 23, 2020 at 5:19 PM Luke Hutchison wrote: > > The ByteBuffer class cannot be extended, because its two constructors both have default visibility. This is problematic, since there are numerous situations where it would be useful to be able to extend ByteBuffer. > > For example, I needed to create a ByteBuffer that was backed by a RandomAccessFile (without actually mapping the file to a MappedByteBuffer), so that I could use a RandomAccessFile with a ByteBuffer API. As far as I can tell, there is no way to do this. This led to a huge amount of unnecessary work and duplicated code, since methods that accepted ByteBuffer now needed to have a duplicated alternative that accepted RandomAccessFile. > > I just came across another example of this -- Google's Jimfs (in-memory filesystem) has run into the same problem. Currently if you call FileChannel.open(path) on a Jimfs path, then call map() on the file channel to mmap it, Jimfs throws an UnsupportedOperationException. This is due to ByteBuffer not being extendable: https://github.com/google/jimfs/issues/54 > > This seems like an unnecessary restriction, based on the source of ByteBuffer. Can the ByteBuffer constructors please be made public? > From Alan.Bateman at oracle.com Fri Apr 24 09:24:29 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 24 Apr 2020 10:24:29 +0100 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: <36BCDCB6-027E-4DB0-87B0-E2A1CC7E7D62@oracle.com> References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> <874ktq9sgu.fsf@mid.deneb.enyo.de> <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> <36BCDCB6-027E-4DB0-87B0-E2A1CC7E7D62@oracle.com> Message-ID: On 23/04/2020 03:08, Brian Burkhalter wrote: > : > > --- a/src/java.base/share/classes/java/nio/X-Buffer.java.template > +++ b/src/java.base/share/classes/java/nio/X-Buffer.java.template > > @@ -125,7 +125,9 @@ > ? * instance of one of these kinds of buffers refers to an > inaccessible region > ? * of memory then an attempt to access that region will not change > the buffer's > ? * content and will cause an unspecified exception to be thrown > either at the > - * time of the access or at some later time. > + * time of the access or at some later time.? The result of a bulk > copy between > + * two direct buffers created via JNI is unspecified if they refer to > + * overlapping regions of memory. > ? * > ? *

Whether a byte buffer is direct or non-direct may be determined by > ? * invoking its {@link #isDirect isDirect} method.? This method is > provided so > I think this opens the door to questions about bulk copy operations on direct buffers by allocateDirect too because they can be sliced and duplicated in interesting ways. So easy to create scenarios where there is copying where the source and target overlaps. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Apr 24 15:14:28 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 24 Apr 2020 08:14:28 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> <874ktq9sgu.fsf@mid.deneb.enyo.de> <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> <36BCDCB6-027E-4DB0-87B0-E2A1CC7E7D62@oracle.com> Message-ID: <8A25DCDC-73DE-4F74-BA7A-C342D1C14D54@oracle.com> > On Apr 24, 2020, at 2:24 AM, Alan Bateman wrote: > >> The result of a bulk copy between >> + * two direct buffers created via JNI is unspecified if they refer to >> + * overlapping regions of memory. >> * >> *

Whether a byte buffer is direct or non-direct may be determined by >> * invoking its {@link #isDirect isDirect} method. This method is provided so >> > I think this opens the door to questions about bulk copy operations on direct buffers by allocateDirect too because they can be sliced and duplicated in interesting ways. So easy to create scenarios where there is copying where the source and target overlaps. I concur and have been able to test this. I think it might be possible to fix that case though. This inclines me to suggest perhaps constraining this issue to the MappedByteBuffer doc change only as first proposed and addressing the other overlap situations separately. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From fw at deneb.enyo.de Fri Apr 24 17:46:26 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Fri, 24 Apr 2020 19:46:26 +0200 Subject: ByteBuffer cannot be extended because constructors have default visibility In-Reply-To: (Luke Hutchison's message of "Thu, 23 Apr 2020 09:16:13 -0600") References: Message-ID: <87mu7093pp.fsf@mid.deneb.enyo.de> * Luke Hutchison: > This seems like an unnecessary restriction, based on the source of > ByteBuffer. Can the ByteBuffer constructors please be made public? Do you want to actually override any ByteBuffer methods? It's currently part of the ByteBuffer interface contract that the methods do not retain references to passed-in argument or, for the reading operations, do not modify the objects that are passed in. This avoids the need for defensive copies in callers. With the security manager fading away, it may eventually be possible to allow subclasses without introducing vulnerabilities (because all code is trusted), but I'm not sure if this is a good idea. I expect that Panama will allow creation of direct byte buffers directly from Java code (without a detour via JNI), but that may not fit your use case because it still requires actual memory as a backing storage. From brian.burkhalter at oracle.com Fri Apr 24 17:52:20 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 24 Apr 2020 10:52:20 -0700 Subject: ByteBuffer cannot be extended because constructors have default visibility In-Reply-To: <87mu7093pp.fsf@mid.deneb.enyo.de> References: <87mu7093pp.fsf@mid.deneb.enyo.de> Message-ID: > On Apr 24, 2020, at 10:46 AM, Florian Weimer wrote: > > It's currently part of the ByteBuffer interface contract that the > methods do not retain references to passed-in argument FWIW, not true for wrap(byte[]). Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From fw at deneb.enyo.de Fri Apr 24 17:59:23 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Fri, 24 Apr 2020 19:59:23 +0200 Subject: ByteBuffer cannot be extended because constructors have default visibility In-Reply-To: (Brian Burkhalter's message of "Fri, 24 Apr 2020 10:52:20 -0700") References: <87mu7093pp.fsf@mid.deneb.enyo.de> Message-ID: <87h7x89344.fsf@mid.deneb.enyo.de> * Brian Burkhalter: >> On Apr 24, 2020, at 10:46 AM, Florian Weimer wrote: >> >> It's currently part of the ByteBuffer interface contract that the >> methods do not retain references to passed-in argument > > FWIW, not true for wrap(byte[]). Indeed. Sorry, I meant instance methods. 8-)