From christoph.langer at sap.com Sun Jun 2 21:35:33 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Sun, 2 Jun 2019 21:35:33 +0000 Subject: RFR 8213031: (zipfs) Add support for POSIX file permissions In-Reply-To: <33E52896-E491-4660-A932-46D5A270A63E@oracle.com> References: <46af4f9d-60f9-c60d-e6f1-8fb97df3ba2e@oracle.com> <5d28b0715d2041ff892a3c44e024f120@sap.com> <8e9231ff-b7d5-bc2f-2643-713f3c670a1d@oracle.com> <3aeba9a64a434a968fc1d82a44077745@sap.com> <953449f0913340f6a94ae3d83611fd92@sap.com> <9b3c9cfe-63e9-94ea-1af0-7ba9e2db52fd@oracle.com> <62a26037-a991-dc31-a972-a82386f63b92@oracle.com> <9806f4b1-9b55-d1d8-4511-5db0ef6786a5@oracle.com> <33E52896-E491-4660-A932-46D5A270A63E@oracle.com> Message-ID: Hi Alan, Lance, thanks for the updated wording in module-info, that really looks good. I incorporated it into my change, no we'd be here: http://cr.openjdk.java.net/~clanger/webrevs/8213031.13/ To be honest, I was hoping it would still make it into JDK13. I guess now I shall update the CSR to get it reviewed, correct? Thanks Christoph From: Lance Andersen Sent: Freitag, 31. Mai 2019 20:01 To: Alan Bateman Cc: Langer, Christoph ; nio-dev ; Java Core Libs Subject: Re: RFR 8213031: (zipfs) Add support for POSIX file permissions On May 31, 2019, at 12:32 PM, Alan Bateman > wrote: On 29/05/2019 13:16, Langer, Christoph wrote: Hi Alan, The table items in L119-150 look fine, we just need to avoid really long lines One minor comment on L123 is that it might be clearer if you drop "created" from the sentence. L48-78 is a "wall of text" and links that I don't think will be easy for most developers to read. Can I provide suggested wording for this part of the spec? I'm just thinking that an alternative wording might help avoid too much iteration on this. I have created a new webrev to add some linebreaks and pick up your suggestion to drop the word "created" in L123. http://cr.openjdk.java.net/~clanger/webrevs/8213031.12/ Waiting on your update for the other part. Attached is alternative wording for the "Support for POSIX file permissions" section. My concern with the proposed in webrev.12 is that it's dense and not easy to read. It also misses a few things - one important one is that access permissions aren't enforced. I think the wording below is looking good. A few minor suggestions below. So overall I think you've got this feature into reasonable shape (I realize it has taken 7 months to get here, this is perhaps a good example of something that needs a lot of up front discussion before going near the code). Once we finalize the CSR, we should look towards getting the changes in early in the JDK 14 cycle so that we have time to vet/catch potential issues. -Alan import java.nio.file.Files; import java.nio.file.LinkOption; import java.nio.file.Path; import java.nio.file.attribute.FileAttributeView; import java.nio.file.attribute.PosixFileAttributes; import java.nio.file.attribute.PosixFilePermission; import java.nio.file.attribute.PosixFileAttributeView; import java.util.Set; *

POSIX file attributes

* *

A Zip file system supports a file attribute {@link FileAttributeView view} * named "{@code zip}" that defines the following file attribute: * *

* * * * * * * * * * * * * * *
Supported attributes
Name Type
permissions {@link Set}<{@link PosixFilePermission}>
*
* * The "permissions" attribute is the set of access permissions that are optionally * stored for entries in a Zip file. The value of the attribute is {@code null} * for entries that do not have access permissions. Zip file systems do not * enforce access permissions. * *

The "permissions" attribute can be read and set using the can-> may * {@linkplain Files#getAttribute(Path, String, LinkOption...) Files.getAttribute} and * {@linkplain Files#setAttribute(Path, String, Object, LinkOption...) Files.setAttribute} * methods. The following example uses these methods to read and set the attribute: *

 {@code
 *     Set perms = Files.getAttribute(entry, "zip:permissions");
 *     if (perms == null) {
 *         perms = PosixFilePermissions.fromString("rw-rw-rw-");
 *         Files.setAttribute(entry, "zip:permissions", perms);
 *     }
 * } 
* *

In addition to the "{@code zip}" view, a Zip file system optionally supports * the {@link PosixFileAttributeView POSIX file attribute view} ("{@code posix}"). * This view extends the "{@code basic}" view with type safe access to the * {@link PosixFileAttributes#owner() owner}, {@link PosixFileAttributes#group() group-owner}, * and {@link PosixFileAttributes#permissions() permissions} attributes. The * "{@code posix}" view is only supported when the Zip file system is created with * the provider property "{@code enablePosixFileAttributes}" set to "{@code true}". * The following creates a file system with this property and reads the access * permissions of a file: *

 {@code
 *     var env = Map.of("enablePosixFileAttributes", "true");
 *     try (FileSystem fs = FileSystems.newFileSystem(file, env) {
 *         Path entry = fs.getPath("entry");
 *         Set perms = Files.getPosixFilePermissions(entry);
 *     }
 * } 
* *

The file owner and group owner attributes are not persisted, meaning they are * not stored in the zip file. The "{@code defaultOwner}" and "{@code defaultGroup}" * provider properties (listed below) can be used to configure the default values * for these attributes. If these properties are not set then the file owner * defaults to the owner of the zip file, and the group owner defaults to the * zip file's group owner (or the file owner on platforms that don't support a * group owner). * *

The "{@code permissions}" attribute is not optional in the "{@code posix}" * view so a default of set of permissions are used for entries that do not have ^^^ of seems out of place * access permissions stored in the Zip file. The default set of permissions * is {@link PosixFilePermission#OWNER_READ OWNER_READ}, {@link PosixFilePermission#OWNER_WRITE is -> are * OWNER_WRITE} and {@link PosixFilePermission#GROUP_READ GROUP_READ}. perhaps consider using a

    when listing the permissions? The default * permissions can be configured with the "{@code defaultPermissions}" property * described below. Best Lance [cid:image001.gif at 01D5199B.DED73A10] Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 658 bytes Desc: image001.gif URL: From adinn at redhat.com Mon Jun 3 13:17:25 2019 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 3 Jun 2019 14:17:25 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> Message-ID: <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> Hi Vladimir, Thanks for the follow-up and for checking the submit failure. I have addressed all the points you raised (point by point comments are included below). A new webrev which passes a submit run is here: http://cr.openjdk.java.net/~adinn/8224974/webrev.04 n.b. This webrev also includes changes to the javadoc for ExtendedMapMode suggested by Alan Bateman. regards, Andrew Dinn ----------- On 30/05/2019 17:58, Vladimir Kozlov wrote: > On 5/30/19 9:08 AM, Andrew Dinn wrote: >> I have uploaded a new webrev which attempts to address the problem. >> >> ?? http://cr.openjdk.java.net/~adinn/8224974/webrev.03 > > I looked only on HotSpot code. Sure, thank you. > stubGenerator_x86_64.cpp - in generate_data_cache_writeback() next are > not used: > > +??? bool optimized = VM_Version::supports_clflushopt(); > +??? bool no_evict = VM_Version::supports_clwb(); Yes. I have removed them. > vm_version_x86.hpp it should check CPUID flag in 32-bit: > > +#else > +? static bool supports_clflush() { return true; } I have added a new feature flag CPU_FLUSH, which is set conditional on the cpuid1 edx bit being set. The 32-bit version of supports_clflush() test for the presence of this feature. The 64-bit version always returns true. On 64-bit I still set the feature flag (unconditionally) even though it is not used. I prefer the loss of 1-2 cycles to the risk of surprising someone looking at the feature bits in a debugger. > We don't check has_match_rule() in LibraryCallKit any more. > In .ad files you need to add predicate to new insrtructions: > > ? predicate(VM_Version::supports_data_cache_line_flush()); Ok, done. > Also add this check to Matcher::match_rule_supported() which you can use > then in C2Compiler::is_intrinsic_supported(). Yes, this is much better. Done. > DISABLE_UNSAFE_WRITEBACK_INTRINSIC should be checked much early may be > together with os::supports_map_sync() when you set > _data_cache_line_flush_size. Doing that that would change the behaviour. If I push the check down into the VM_Version code as you suggest then that will disable map and writeback operations as well as the JIT intrinsic. This test was used to allow the benefit of the intrinsic to be measured. However, I don't think it is worth worrying about retaining this test. It was useful during development but is probably not going to be needed any more. So, I have removed it. >> Unfortunately, this latest webrev still fails when uploaded to the >> submit repo. ... > > ?t:/workspace/open/src/java.base/windows/native/libnio/ch/FileChannelImpl.c(64): > error C2220: warning treated as error - no 'object' file generated > ?t:/workspace/open/src/java.base/windows/native/libnio/ch/FileChannelImpl.c(64): > warning C4029: declared formal parameter list different from definition Ah, got it. The change to handle the isSync argument to map0 needs propagating to the Windows native implementation. Also, I tweaked the Windows C code to throw InternalError if mapSync is ever passed as true. That /should never happen/ in the current implementation. If/when this gets ported to Windows then the code that does the throw can be replaced with case handling to call mmap with MAP_SYNC. From adinn at redhat.com Mon Jun 3 14:37:58 2019 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 3 Jun 2019 15:37:58 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> Message-ID: On 03/06/2019 14:17, Andrew Dinn wrote: > Hi Vladimir, > > Thanks for the follow-up and for checking the submit failure. I have > addressed all the points you raised (point by point comments are > included below). A new webrev which passes a submit run is here: > > http://cr.openjdk.java.net/~adinn/8224974/webrev.04 > > n.b. This webrev also includes changes to the javadoc for > ExtendedMapMode suggested by Alan Bateman. . . . I have made one further change to locate the new ExtendedMapMode enum in module jdk.nio.mapmode and package jdk.nio.mapmode as suggested offline by Alan Bateman and the OpenJDK lead. Please refer to this webrev instead: webrev: http://cr.openjdk.java.net/~adinn/8224974/webrev.05 The CSR and JEP have been updated accordingly CSR: https://bugs.openjdk.java.net/browse/JDK-8224975 JEP: https://bugs.openjdk.java.net/browse/JDK-8207851 regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From aeubanks at google.com Mon Jun 3 15:17:33 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Mon, 3 Jun 2019 08:17:33 -0700 Subject: [RFR] 8224645: Test java/nio/channels/DatagramChannel/BasicMulticastTests.java fails with NoSuchElementException In-Reply-To: References: <8fa9d55a-520e-df06-1d5d-566ef39fa992@oracle.com> <27fa2523-2f81-10c9-0688-3de42af96b02@oracle.com> <7e2ead05-f49c-8ca3-9bd5-b6d1e3e6ec21@oracle.com> <548abdef-3f06-0947-2e00-e05cc59f626f@oracle.com> Message-ID: Ping on the updated webrev. On Wed, May 29, 2019 at 1:08 PM Arthur Eubanks wrote: > Moved NetworkConfiguration.printSystemConfiguration() to the beginning, >> removed counting and printing the number of interfaces/throwing >> SkippedException. >> new webrev: http://cr.openjdk.java.net/~aeubanks/8224645/webrev.02/ >> >> This looks okay to me, the only thing is that the NetworkConfiguration is >> sent to System.err whereas the test proper sends its output to System.out. >> Can you check the .jtr file to see how it looks? I suspect it may be easier >> to read if you send the network configuration to System.out. >> >> -Alan >> > I don't think it's a big difference either way, but changed it to > System.out. Also added some extra info to the "Exception Tests" log to show > the addresses it's testing. Now it looks like: > > ----------System.out:(33/801)---------- > *** all system network interface configuration *** > Display name: eth0 > Name: eth0 > InetAddress: /2620:15c:2ce:200:fb70:ba8c:5c39:dd96%eth0 > InetAddress: /fe80:0:0:0:3745:855d:754c:c782%eth0 > InetAddress: /100.117.216.36 > Up? true > Loopback? false > PointToPoint? false > Supports multicast? true > Virtual? false > Hardware address: [112, 90, 15, 47, 36, 82] > MTU: 1500 > Index: 2 > > Display name: lo > Name: lo > InetAddress: /0:0:0:0:0:0:0:1%lo > InetAddress: /127.0.0.1 > Up? true > Loopback? true > PointToPoint? false > Supports multicast? false > Virtual? false > Hardware address: null > MTU: 65536 > Index: 1 > > *** end *** > MembershipKey test using 225.4.5.6 @ eth0 > Exception Tests using 225.4.5.6, 1.2.3.4 @ eth0 > MembershipKey test using ff02:0:0:0:0:0:0:a @ eth0 > Exception Tests using ff02:0:0:0:0:0:0:a, fe80:0:0:0:0:0:0:1234 @ eth0 > ----------System.err:(1/15)---------- > STATUS:Passed. > > http://cr.openjdk.java.net/~aeubanks/8224645/webrev.03 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon Jun 3 18:42:29 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 3 Jun 2019 19:42:29 +0100 Subject: RFR 8213031: (zipfs) Add support for POSIX file permissions In-Reply-To: References: <8e9231ff-b7d5-bc2f-2643-713f3c670a1d@oracle.com> <3aeba9a64a434a968fc1d82a44077745@sap.com> <953449f0913340f6a94ae3d83611fd92@sap.com> <9b3c9cfe-63e9-94ea-1af0-7ba9e2db52fd@oracle.com> <62a26037-a991-dc31-a972-a82386f63b92@oracle.com> <9806f4b1-9b55-d1d8-4511-5db0ef6786a5@oracle.com> <33E52896-E491-4660-A932-46D5A270A63E@oracle.com> Message-ID: On 02/06/2019 22:35, Langer, Christoph wrote: > > Hi Alan, Lance, > > thanks for the updated wording in module-info, that really looks good. > I incorporated it into my change, no we?d be here: > > http://cr.openjdk.java.net/~clanger/webrevs/8213031.13/ > > To be honest, I was hoping it would still make it into JDK13. > > I guess now I shall update the CSR to get it reviewed, correct? > > The spec looks good. The CSR is the next step, it can happen while the changes are being reviewed here. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Jun 4 10:27:53 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 4 Jun 2019 11:27:53 +0100 Subject: [RFR] 8224645: Test java/nio/channels/DatagramChannel/BasicMulticastTests.java fails with NoSuchElementException In-Reply-To: References: <8fa9d55a-520e-df06-1d5d-566ef39fa992@oracle.com> <27fa2523-2f81-10c9-0688-3de42af96b02@oracle.com> <7e2ead05-f49c-8ca3-9bd5-b6d1e3e6ec21@oracle.com> <548abdef-3f06-0947-2e00-e05cc59f626f@oracle.com> Message-ID: <6dca339e-69e1-328b-faf5-9b2cc9a71168@oracle.com> On 03/06/2019 16:17, Arthur Eubanks wrote: > > I don't think it's a big difference either way, but changed it to > System.out. Also added some extra info to the "Exception Tests" > log to show the addresses it's testing. Now it looks like: > webrev.03 looks okay to me. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Jun 4 10:40:11 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 4 Jun 2019 11:40:11 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> Message-ID: <42909e1f-e235-8f4b-d881-cff556fb7bc2@oracle.com> On 03/06/2019 15:37, Andrew Dinn wrote: > : > > The CSR and JEP have been updated accordingly > > CSR: https://bugs.openjdk.java.net/browse/JDK-8224975 > JEP: https://bugs.openjdk.java.net/browse/JDK-8207851 > The specification section in the CSR was missing the module definition so I added that. The rest looks okay so I've added myself as Reviewer so you can finalize it. -Alan. From christoph.langer at sap.com Tue Jun 4 14:25:08 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 4 Jun 2019 14:25:08 +0000 Subject: RFR 8213031: (zipfs) Add support for POSIX file permissions In-Reply-To: References: <8e9231ff-b7d5-bc2f-2643-713f3c670a1d@oracle.com> <3aeba9a64a434a968fc1d82a44077745@sap.com> <953449f0913340f6a94ae3d83611fd92@sap.com> <9b3c9cfe-63e9-94ea-1af0-7ba9e2db52fd@oracle.com> <62a26037-a991-dc31-a972-a82386f63b92@oracle.com> <9806f4b1-9b55-d1d8-4511-5db0ef6786a5@oracle.com> <33E52896-E491-4660-A932-46D5A270A63E@oracle.com> Message-ID: Hi Alan, I made some updates to the CSR. The main update was that I transferred the new documentation part from module-info.java to the CSR's Specification section. Can you please review the CSR? Thanks Christoph From: Alan Bateman Sent: Montag, 3. Juni 2019 20:42 To: Langer, Christoph ; Lance Andersen Cc: nio-dev ; Java Core Libs Subject: Re: RFR 8213031: (zipfs) Add support for POSIX file permissions On 02/06/2019 22:35, Langer, Christoph wrote: Hi Alan, Lance, thanks for the updated wording in module-info, that really looks good. I incorporated it into my change, no we'd be here: http://cr.openjdk.java.net/~clanger/webrevs/8213031.13/ To be honest, I was hoping it would still make it into JDK13. I guess now I shall update the CSR to get it reviewed, correct? The spec looks good. The CSR is the next step, it can happen while the changes are being reviewed here. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From k.mehta at salesforce.com Tue Jun 4 17:13:05 2019 From: k.mehta at salesforce.com (Karan Mehta) Date: Tue, 4 Jun 2019 10:13:05 -0700 Subject: Invalid argument exception when trying to seek FilePosition to LONG.MAX_VALUE Message-ID: Hello everyone, Apache bookkeeper project uses Java NIO to read journal files from disk. Recently a change was made that would seek the file to LONG.MAX_VALUE. When we tried running this on a test inside a docker container with openjdk:8-jdk image, it results in IOException. The patch fix PR is here . Furthermore, running it inside the same container on different hardware throws IOException either while seeking the file to LONG.MAX_VALUE or when reading the file at that value as per the link of the interface here . Checkout the comment here . I am suspecting this to be a JNI related issue or bug. Let me know what you all think and if there's any better way to figure this out. -- Karan Mehta -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Jun 4 18:15:28 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 4 Jun 2019 19:15:28 +0100 Subject: Invalid argument exception when trying to seek FilePosition to LONG.MAX_VALUE In-Reply-To: References: Message-ID: <3c8a1d34-37f7-6598-ad6d-7f2648173ac9@oracle.com> On 04/06/2019 18:13, Karan Mehta wrote: > Hello everyone, > > Apache bookkeeper project uses Java NIO to read journal files from > disk. Recently a change was made that would seek the file to > LONG.MAX_VALUE. When we tried running this on a test inside a docker > container with openjdk:8-jdk image, it results in IOException. The > patch fix PR is here . > Furthermore, running it inside the same container on different > hardware throws IOException either while seeking the file to > LONG.MAX_VALUE or when reading the file at that value as per the link > of the interface here > . > Checkout the comment here > . > I am suspecting this to be a JNI related issue or bug. Let me?know > what you all think and if there's any better way to figure this out. I just checked this on macOS and Linux. macOS seems happy with lseek'ing to LONG.MAX_VALUE, Linux does not: [pid 34168] lseek(4, 9223372036854775807, SEEK_SET) = -1 EINVAL (Invalid argument) I see the man page on Linux includes "or beyond the end of a seekable device" in the possible reasons for EINVAL and I assume this is why it fails. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From k.mehta at salesforce.com Tue Jun 4 18:31:42 2019 From: k.mehta at salesforce.com (Karan Mehta) Date: Tue, 4 Jun 2019 11:31:42 -0700 Subject: Invalid argument exception when trying to seek FilePosition to LONG.MAX_VALUE In-Reply-To: <3c8a1d34-37f7-6598-ad6d-7f2648173ac9@oracle.com> References: <3c8a1d34-37f7-6598-ad6d-7f2648173ac9@oracle.com> Message-ID: Thanks for your quick response. Appreciate it. The link for man page seems a bit contradictory. At the top, it says, "lseek() allows the file offset to be set beyond the end of the file (but this does not change the size of the file)." but it is followed by the text you mentioned. The function declaration is `off_t lseek(int fd, off_t offset, int whence);` where the offset is defined with off_t type. I was trying to reason its size based on stackoverflow here . The link suggests that they would be signed integers and hence this is a potential overflow and might result in negative value, causing it to be a reason for exception. Should the JNI call be using this instead since the Java interface offers the option to pass long variable for the offset. What should be the correct fix here? Of course, if the underlying OS is 32 bit, even the SO link above won't help but it would make it work on almost all modern hardware. On Tue, Jun 4, 2019 at 11:17 AM Alan Bateman wrote: > On 04/06/2019 18:13, Karan Mehta wrote: > > Hello everyone, > > Apache bookkeeper project uses Java NIO to read journal files from disk. > Recently a change was made that would seek the file to LONG.MAX_VALUE. When > we tried running this on a test inside a docker container with > openjdk:8-jdk image, it results in IOException. The patch fix PR is here > . Furthermore, running it > inside the same container on different hardware throws IOException either > while seeking the file to LONG.MAX_VALUE or when reading the file at that > value as per the link of the interface here > . > Checkout the comment here > . > I am suspecting this to be a JNI related issue or bug. Let me know what you > all think and if there's any better way to figure this out. > > I just checked this on macOS and Linux. macOS seems happy with lseek'ing > to LONG.MAX_VALUE, Linux does not: > > [pid 34168] lseek(4, 9223372036854775807, SEEK_SET) = -1 EINVAL (Invalid > argument) > > I see the man page on Linux includes "or beyond the end of a seekable > device" in the possible reasons for EINVAL and I assume this is why it > fails. > > -Alan > -- Karan Mehta -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Jun 4 18:45:36 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 4 Jun 2019 19:45:36 +0100 Subject: Invalid argument exception when trying to seek FilePosition to LONG.MAX_VALUE In-Reply-To: References: <3c8a1d34-37f7-6598-ad6d-7f2648173ac9@oracle.com> Message-ID: <0ab48751-2a62-c887-5e60-72cbc219197c@oracle.com> On 04/06/2019 19:31, Karan Mehta wrote: > Thanks for your quick response. Appreciate it. > > The link for man page seems a bit contradictory. At the top, it says, > "lseek() allows the file offset to be set beyond the end of the file > (but this does not change the size of the file)." but it is followed > by the text you mentioned. > The function declaration is `off_t lseek(int fd, off_t offset, int > whence);` where the offset is defined with off_t type. I was trying to > reason its size based on stackoverflow here > . > The link > ?suggests > that they would be signed integers and hence this is a potential > overflow and might result in negative value, causing it to be a reason > for exception. Should the JNI call be using this > ?instead since the Java > interface offers the option to pass long variable for the offset. The code uses lseek64 and off64_t so I don't think this is the issue. You can verify this by running with strace and you should see output similar to what I had in the first mail. > What should be the correct fix here? Of?course, if the underlying OS > is 32 bit, even the SO link above won't help but it would make it work > on almost all modern hardware. > The IOException is correct, it's just the "Illegal argument" from the syscall is confusing. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From k.mehta at salesforce.com Tue Jun 4 20:07:04 2019 From: k.mehta at salesforce.com (Karan Mehta) Date: Tue, 4 Jun 2019 13:07:04 -0700 Subject: Invalid argument exception when trying to seek FilePosition to LONG.MAX_VALUE In-Reply-To: <0ab48751-2a62-c887-5e60-72cbc219197c@oracle.com> References: <3c8a1d34-37f7-6598-ad6d-7f2648173ac9@oracle.com> <0ab48751-2a62-c887-5e60-72cbc219197c@oracle.com> Message-ID: > The code uses lseek64 and off64_t so I don't think this is the issue. You can verify this by running with strace and you should see output similar to what I had in the first mail. I ran strace on a 64 bit Linux box but I see calls to only lseek instead of lseek64. Am I missing something here? What does it show up on mac? > The IOException is correct, it's just the "Illegal argument" from the syscall is confusing. Should the IOException be swallowed here according to the FileChannel interface contract and the resulting operation be no-op for the caller instead? Does this mean that any file size longer than INT.MAX_VALUE can potentially cause a problem here, either when trying to read from it or seeking it? On Tue, Jun 4, 2019 at 11:47 AM Alan Bateman wrote: > On 04/06/2019 19:31, Karan Mehta wrote: > > Thanks for your quick response. Appreciate it. > > The link for man page seems a bit contradictory. At the top, it says, > "lseek() allows the file offset to be set beyond the end of the file (but > this does not change the size of the file)." but it is followed by the text > you mentioned. > The function declaration is `off_t lseek(int fd, off_t offset, int > whence);` where the offset is defined with off_t type. I was trying to > reason its size based on stackoverflow here > . > The link > suggests > that they would be signed integers and hence this is a potential overflow > and might result in negative value, causing it to be a reason for > exception. Should the JNI call be using this > instead since the Java > interface offers the option to pass long variable for the offset. > > The code uses lseek64 and off64_t so I don't think this is the issue. You > can verify this by running with strace and you should see output similar to > what I had in the first mail. > > What should be the correct fix here? Of course, if the underlying OS is 32 > bit, even the SO link above won't help but it would make it work on almost > all modern hardware. > > The IOException is correct, it's just the "Illegal argument" from the > syscall is confusing. > > -Alan > -- Karan Mehta -------------- next part -------------- An HTML attachment was scrubbed... URL: From adinn at redhat.com Wed Jun 5 08:43:35 2019 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 5 Jun 2019 09:43:35 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <42909e1f-e235-8f4b-d881-cff556fb7bc2@oracle.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <42909e1f-e235-8f4b-d881-cff556fb7bc2@oracle.com> Message-ID: <7adbabb2-c804-90aa-ee58-b97b7aed52eb@redhat.com> On 04/06/2019 11:40, Alan Bateman wrote: > On 03/06/2019 15:37, Andrew Dinn wrote: >> : >> >> The CSR and JEP have been updated accordingly >> >> CSR:? https://bugs.openjdk.java.net/browse/JDK-8224975 >> JEP:? https://bugs.openjdk.java.net/browse/JDK-8207851 >> > The specification section in the CSR was missing the module definition > so I added that. The rest looks okay so I've added myself as Reviewer so > you can finalize it. Thanks, Alan. Done. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Wed Jun 5 09:52:51 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 5 Jun 2019 10:52:51 +0100 Subject: RFR 8213031: (zipfs) Add support for POSIX file permissions In-Reply-To: References: <953449f0913340f6a94ae3d83611fd92@sap.com> <9b3c9cfe-63e9-94ea-1af0-7ba9e2db52fd@oracle.com> <62a26037-a991-dc31-a972-a82386f63b92@oracle.com> <9806f4b1-9b55-d1d8-4511-5db0ef6786a5@oracle.com> <33E52896-E491-4660-A932-46D5A270A63E@oracle.com> Message-ID: <458d4ce8-d669-3117-908d-687dae2ad8d2@oracle.com> On 04/06/2019 15:25, Langer, Christoph wrote: > > Hi Alan, > > I made some updates to the CSR. The main update was that I transferred > the new documentation part from module-info.java to the CSR?s > Specification section. > > Can you please review the CSR? > The CSR mostly looks good. One thing is that the Specification section doesn't have the changes to the "Zip File System Properties" part of the spec so I think that should be added. I'll add myself as Reviewer to the CSR so you can finalize when you are done. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Jun 5 10:48:19 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 5 Jun 2019 11:48:19 +0100 Subject: Invalid argument exception when trying to seek FilePosition to LONG.MAX_VALUE In-Reply-To: References: <3c8a1d34-37f7-6598-ad6d-7f2648173ac9@oracle.com> <0ab48751-2a62-c887-5e60-72cbc219197c@oracle.com> Message-ID: <9bf5f8d0-e86f-033a-0d8e-d160cdab8859@oracle.com> On 04/06/2019 21:07, Karan Mehta wrote: > > The code uses lseek64 and off64_t so I don't think this is the > issue. You can verify this by running with strace and you should see > output similar to what I had in the first mail. > > I ran strace on a 64 bit Linux box but I see calls to only lseek > instead of lseek64. Am I missing something here? It's just an alias for llseek. You should be able to see the value of the offset specified to the syscall, it will be 9223372036854775807 when you use fc.position(Long.MAX_VALUE). > What does it show up on mac? > > > The IOException is correct, it's just the "Illegal argument" from > the syscall is confusing. > > Should the IOException be swallowed here according to the FileChannel > interface contract and the resulting operation be no-op for the caller > instead? Does this mean that any file size longer than INT.MAX_VALUE > can potentially cause a problem here, either when trying to read from > it or seeking it? > I'm not sure what you mean by "swallowing" the exception here. FileChannel.position(long) is implemented to use lseek64 and I think is failing with values larger than the file system. I don't think Integer.MAX_VALUE is an issue. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Wed Jun 5 14:07:31 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 5 Jun 2019 14:07:31 +0000 Subject: RFR 8213031: (zipfs) Add support for POSIX file permissions In-Reply-To: <458d4ce8-d669-3117-908d-687dae2ad8d2@oracle.com> References: <953449f0913340f6a94ae3d83611fd92@sap.com> <9b3c9cfe-63e9-94ea-1af0-7ba9e2db52fd@oracle.com> <62a26037-a991-dc31-a972-a82386f63b92@oracle.com> <9806f4b1-9b55-d1d8-4511-5db0ef6786a5@oracle.com> <33E52896-E491-4660-A932-46D5A270A63E@oracle.com> <458d4ce8-d669-3117-908d-687dae2ad8d2@oracle.com> Message-ID: Hi Alan, I've added the properties to the Specification section and made a few additional updates to get the document better structured and readable. Maybe you want to double check one more time. I've set the CSR to status "Finalized". Thanks for reviewing. /Christoph From: Alan Bateman Sent: Mittwoch, 5. Juni 2019 11:53 To: Langer, Christoph Cc: nio-dev ; Java Core Libs ; Lance Andersen Subject: Re: RFR 8213031: (zipfs) Add support for POSIX file permissions On 04/06/2019 15:25, Langer, Christoph wrote: Hi Alan, I made some updates to the CSR. The main update was that I transferred the new documentation part from module-info.java to the CSR's Specification section. Can you please review the CSR? The CSR mostly looks good. One thing is that the Specification section doesn't have the changes to the "Zip File System Properties" part of the spec so I think that should be added. I'll add myself as Reviewer to the CSR so you can finalize when you are done. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jun 5 14:36:09 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 5 Jun 2019 07:36:09 -0700 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> Message-ID: HI Andrew, I have some minor comments on webrev.05. (webrev.06 looks rather strange.) Thanks, Brian Note: Hotspot changes not reviewed here. General Check copyrights. Some headers are missing altogether, some have incorrect years. MappedByteBuffer.java 84: ?any of other modes? -> ?any of the other modes?, ?this? -> ?This? 85: nit: usually use American spelling: ?behaviour? -> ?behavior? (but it does not matter so much here as it is not public javadoc. 172: ?pasing? -> ?passing? 175: delete ?using? 181: comma or semicolon before ?otherwise? 355-365: Collapse to? Unsafe.getUnsafe().writebackMemory(address + index, length); Unsafe.java 924-945: Capitalize first words of sentences. 959: Is there a possibility of overflow, e.g., use Math.addExact(address, length) ? 1290: Insert blank line after. FileChannelImpl.c (Unix) 85: ?||! map_sync? -> ??|| !map_sync? 98-110: Put symbolic name definitions at head of file? FileChannelImpl.c (Windows) 91: Use same message as at Unix version line 135? ExtendedMapMode 13: Constant name should be MAP_MODE_CONSTRUCTOR. 13: Insert blank line after 34-36: Move to before line 13. 24: Move constructor to end of class. > On Jun 3, 2019, at 7:37 AM, Andrew Dinn wrote: > > > > On 03/06/2019 14:17, Andrew Dinn wrote: >> Hi Vladimir, >> >> Thanks for the follow-up and for checking the submit failure. I have >> addressed all the points you raised (point by point comments are >> included below). A new webrev which passes a submit run is here: >> >> http://cr.openjdk.java.net/~adinn/8224974/webrev.04 >> >> n.b. This webrev also includes changes to the javadoc for >> ExtendedMapMode suggested by Alan Bateman. > . . . > > I have made one further change to locate the new ExtendedMapMode enum in > module jdk.nio.mapmode and package jdk.nio.mapmode as suggested offline > by Alan Bateman and the OpenJDK lead. Please refer to this webrev instead: > > webrev: http://cr.openjdk.java.net/~adinn/8224974/webrev.05 > > The CSR and JEP have been updated accordingly > > CSR: https://bugs.openjdk.java.net/browse/JDK-8224975 > JEP: https://bugs.openjdk.java.net/browse/JDK-8207851 > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Wed Jun 5 14:44:47 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 5 Jun 2019 14:44:47 +0000 Subject: RFR 8213031: (zipfs) Add support for POSIX file permissions In-Reply-To: <458d4ce8-d669-3117-908d-687dae2ad8d2@oracle.com> References: <953449f0913340f6a94ae3d83611fd92@sap.com> <9b3c9cfe-63e9-94ea-1af0-7ba9e2db52fd@oracle.com> <62a26037-a991-dc31-a972-a82386f63b92@oracle.com> <9806f4b1-9b55-d1d8-4511-5db0ef6786a5@oracle.com> <33E52896-E491-4660-A932-46D5A270A63E@oracle.com> <458d4ce8-d669-3117-908d-687dae2ad8d2@oracle.com> Message-ID: Hi Lance, Alan, et al., as it seems we are getting close to CSR approval, the changeset should also be finally reviewed. Here is the most current version: http://cr.openjdk.java.net/~clanger/webrevs/8213031.14/ I have made some updates to ZipConstants as suggested by Lance in a private mail. Best regards Christoph -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jun 5 14:46:22 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 5 Jun 2019 07:46:22 -0700 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> Message-ID: <53712EB1-D1C1-4174-BA69-9C7D85DCE9A8@oracle.com> To clarify, the following comments were for the _internal_ class. These along with a number of my other comments are just stylistic and not absolutely necessary to change. Thanks, Brian > On Jun 5, 2019, at 7:36 AM, Brian Burkhalter wrote: > > ExtendedMapMode > 13: Constant name should be MAP_MODE_CONSTRUCTOR. > 13: Insert blank line after > 34-36: Move to before line 13. > 24: Move constructor to end of class. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Jun 5 15:15:58 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 5 Jun 2019 16:15:58 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <7adbabb2-c804-90aa-ee58-b97b7aed52eb@redhat.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <42909e1f-e235-8f4b-d881-cff556fb7bc2@oracle.com> <7adbabb2-c804-90aa-ee58-b97b7aed52eb@redhat.com> Message-ID: <8c1c2dbe-0dc4-609e-4cf5-093a2f0734b7@oracle.com> On 05/06/2019 09:43, Andrew Dinn wrote: > On 04/06/2019 11:40, Alan Bateman wrote: >> On 03/06/2019 15:37, Andrew Dinn wrote: >>> : >>> >>> The CSR and JEP have been updated accordingly >>> >>> CSR:? https://bugs.openjdk.java.net/browse/JDK-8224975 >>> JEP:? https://bugs.openjdk.java.net/browse/JDK-8207851 >>> >> The specification section in the CSR was missing the module definition >> so I added that. The rest looks okay so I've added myself as Reviewer so >> you can finalize it. > Thanks, Alan. Done. > I see the CSR is finalized now and I'm sure Joe will look at this in the coming days. I re-read the JEP and I think it needs a few edits before seeking Brian's endorsement. One thing is that the scope is "JDK" but the Description still has a "Proposed Java SE API Changes" sub-section. The supported interfaces are the new jdk.nio.mapmode module and probably the name of the new buffer pool, everything else is describing the implementation. I think the "Proposal Internal JDK API changes" section is okay. I think the "Alternatives" section should list waiting for Project Panama as an alternative. We've had at two discussions here about buffers not being the long term API for this and I think we should at least acknowledge that in this section. This goes partly with the reference in the "Risks and Assumptions" section to the the absolute bulk put/get methods - these methods have been added for Java SE 13. -Alan From fweimer at redhat.com Wed Jun 5 15:31:09 2019 From: fweimer at redhat.com (Florian Weimer) Date: Wed, 05 Jun 2019 17:31:09 +0200 Subject: Invalid argument exception when trying to seek FilePosition to LONG.MAX_VALUE In-Reply-To: <3c8a1d34-37f7-6598-ad6d-7f2648173ac9@oracle.com> (Alan Bateman's message of "Tue, 4 Jun 2019 19:15:28 +0100") References: <3c8a1d34-37f7-6598-ad6d-7f2648173ac9@oracle.com> Message-ID: <87r288t5fm.fsf@oldenburg2.str.redhat.com> * Alan Bateman: > On 04/06/2019 18:13, Karan Mehta wrote: > > Hello everyone, > > Apache bookkeeper project uses Java NIO to read journal files from disk. Recently a change was made that would seek the file to > LONG.MAX_VALUE. When we tried running this on a test inside a docker container with openjdk:8-jdk image, it results in > IOException. The patch fix PR is here. Furthermore, running it inside the same container on different hardware throws IOException > either while seeking the file to LONG.MAX_VALUE or when reading the file at that value as per the link of the interface here. > Checkout the comment here. I am suspecting this to be a JNI related issue or bug. Let me know what you all think and if there's > any better way to figure this out. > > I just checked this on macOS and Linux. macOS seems happy with lseek'ing to LONG.MAX_VALUE, Linux does not: > > [pid 34168] lseek(4, 9223372036854775807, SEEK_SET) = -1 EINVAL (Invalid argument) This is with ext4, right? XFS is fine with it and can even write there: info: maximum writable file offset: 9223372036854775806 (7ffffffffffffffe) info: writing out of range fails with Invalid argument (22) Thanks, Florian From k.mehta at salesforce.com Wed Jun 5 17:06:42 2019 From: k.mehta at salesforce.com (Karan Mehta) Date: Wed, 5 Jun 2019 10:06:42 -0700 Subject: Invalid argument exception when trying to seek FilePosition to LONG.MAX_VALUE In-Reply-To: <87r288t5fm.fsf@oldenburg2.str.redhat.com> References: <3c8a1d34-37f7-6598-ad6d-7f2648173ac9@oracle.com> <87r288t5fm.fsf@oldenburg2.str.redhat.com> Message-ID: > It's just an alias for llseek. You should be able to see the value of the offset specified to the syscall, it will be 9223372036854775807 when you use fc.position(Long.MAX_VALUE). Since the value passed to syscall is LONG.MAX_VALUE, we can confirm that it is indeed lseek64, otherwise it would have been some negative number. > I'm not sure what you mean by "swallowing" the exception here. FileChannel.position(long) is implemented to use lseek64 and I think is failing with values larger than the file system. I don't think Integer.MAX_VALUE is an issue. Let me clarify this a bit further. If the underlying FS throws an exception while seeking to LONG.MAX_VALUE, the JNI call should catch it and return a No-op to the Java function, instead of throwing it back further. > This is with ext4, right? XFS is fine with it and can even write there: Can't speak for XFS, but yeah with ext4 is where I am seeing the issue. On Wed, Jun 5, 2019 at 8:31 AM Florian Weimer wrote: > * Alan Bateman: > > > On 04/06/2019 18:13, Karan Mehta wrote: > > > > Hello everyone, > > > > Apache bookkeeper project uses Java NIO to read journal files from > disk. Recently a change was made that would seek the file to > > LONG.MAX_VALUE. When we tried running this on a test inside a docker > container with openjdk:8-jdk image, it results in > > IOException. The patch fix PR is here. Furthermore, running it inside > the same container on different hardware throws IOException > > either while seeking the file to LONG.MAX_VALUE or when reading the > file at that value as per the link of the interface here. > > Checkout the comment here. I am suspecting this to be a JNI related > issue or bug. Let me know what you all think and if there's > > any better way to figure this out. > > > > I just checked this on macOS and Linux. macOS seems happy with lseek'ing > to LONG.MAX_VALUE, Linux does not: > > > > [pid 34168] lseek(4, 9223372036854775807, SEEK_SET) = -1 EINVAL (Invalid > argument) > > This is with ext4, right? XFS is fine with it and can even write there: > > info: maximum writable file offset: 9223372036854775806 (7ffffffffffffffe) > info: writing out of range fails with Invalid argument (22) > > Thanks, > Florian > -- Karan Mehta -------------- next part -------------- An HTML attachment was scrubbed... URL: From adinn at redhat.com Wed Jun 5 17:10:16 2019 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 5 Jun 2019 18:10:16 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> Message-ID: Hi Brian, Thanks very much for reviewing the patch. Also, commendations on your eagle-eyed thoroughness. I have happily accepted most corrections. I have commented (inline below) in a few places where I think things might need to be discussed further. I will upload a new webrev after these points are resolved. On 05/06/2019 15:36, Brian Burkhalter wrote: > I have some minor comments on webrev.05. (webrev.06 looks rather strange.) That is the next working version being queued up -- you may well have seen an intermediate version while it was uploading. > Note: Hotspot changes not reviewed here. Sure. > General > Check copyrights. Some headers are missing altogether, some have > incorrect years. Yes, all of them are now fixed. > MappedByteBuffer.java > 84:?any of other modes? -> ?any of the other modes?, ?this? -> ?This? > 85:nit: usually use American spelling: ?behaviour? -> ?behavior? (but it > does not matter so much here as it is not public javadoc. > 172:?pasing? -> ?passing? > 175:delete ?using? > 181:comma or semicolon before ?otherwise? All done. I even gritted my teeth and spelled behaviour 'US-wize'. > 355-365: Collapse to? > > Unsafe.getUnsafe().writebackMemory(address + index, length); Sure, done. Although personally I'm 'not much into that whole brevity thing'. > Unsafe.java > 924-945:Capitalize first words of sentences. Done. > 959:Is there a possibility of overflow, e.g., use Math.addExact(address, > length) ? I think it assumed that all OSes currently ported (certainly Linux) will never map a vmem region so that addition of a valid internal offset might overflow. If it did we would be in big trouble as regards null pointer checking. So, modulo input from those wiser than me in operating system lore, I think this is not needed. > 1290:Insert blank line after. Done. > FileChannelImpl.c (Unix) > 85:?||! map_sync? -> ??|| !map_sync? Done. > 98-110:Put symbolic name definitions at head of file? I thought these definitions would be better placed at the point where the defines are needed so as to make it clear why this is being done. If you are strongly motivated to move them to the more normal location I will do so. > FileChannelImpl.c (Windows) > 91:Use same message as at Unix version line 135? Perhaps but I don't think they are the same error and thougt it beter to reinforce that with different messages. Note the different exception types. In the case of Windows we should never reach this point -- at least not until a Windows port comes along and requires the message to be changed (effectively removing here and replacing it with somewhere else). So, an InternalError is thrown. In the Unix case the error might conceivably happen (because the target OS mmap implementation does not have proper MAP_SYNC support. Hence the use of IOException. > ExtendedMapMode > 13:Constant name should be MAP_MODE_CONSTRUCTOR. > 13:Insert blank line after > 34-36:Move to before line 13. > 24:Move constructor to end of class. Yes, all good. Thanks! regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From brian.burkhalter at oracle.com Wed Jun 5 17:25:54 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 5 Jun 2019 10:25:54 -0700 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> Message-ID: Hi Andrew, > On Jun 5, 2019, at 10:10 AM, Andrew Dinn wrote: > > Thanks very much for reviewing the patch. Also, commendations on your > eagle-eyed thoroughness. Thanks. :-) > I have happily accepted most corrections. I have commented (inline > below) Likewise. > in a few places where I think things might need to be discussed > further. I will upload a new webrev after these points are resolved. OK > On 05/06/2019 15:36, Brian Burkhalter wrote: >> I have some minor comments on webrev.05. (webrev.06 looks rather strange.) > > That is the next working version being queued up -- you may well have > seen an intermediate version while it was uploading. Must have done as it looks OK now. >> MappedByteBuffer.java >> 84:?any of other modes? -> ?any of the other modes?, ?this? -> ?This? >> 85:nit: usually use American spelling: ?behaviour? -> ?behavior? (but it >> does not matter so much here as it is not public javadoc. >> 172:?pasing? -> ?passing? >> 175:delete ?using? >> 181:comma or semicolon before ?otherwise? > > All done. I even gritted my teeth and spelled behaviour 'US-wize?. ;-) >> 959:Is there a possibility of overflow, e.g., use Math.addExact(address, >> length) ? > > I think it assumed that all OSes currently ported (certainly Linux) will > never map a vmem region so that addition of a valid internal offset > might overflow. If it did we would be in big trouble as regards null > pointer checking. So, modulo input from those wiser than me in operating > system lore, I think this is not needed. It sounds reasonable to leave it as-is in that case. >> 98-110:Put symbolic name definitions at head of file? > > I thought these definitions would be better placed at the point where > the defines are needed so as to make it clear why this is being done. If > you are strongly motivated to move them to the more normal location I > will do so. I think that is reasonable as well so no need to move them. >> FileChannelImpl.c (Windows) >> 91:Use same message as at Unix version line 135? > > Perhaps but I don't think they are the same error and thougt it beter to > reinforce that with different messages. Note the different exception > types. In the case of Windows we should never reach this point -- at > least not until a Windows port comes along and requires the message to > be changed (effectively removing here and replacing it with somewhere > else). So, an InternalError is thrown. In the Unix case the error might > conceivably happen (because the target OS mmap implementation does not > have proper MAP_SYNC support. Hence the use of IOException. All right, then leave it as it is. >> ExtendedMapMode >> 13:Constant name should be MAP_MODE_CONSTRUCTOR. >> 13:Insert blank line after >> 34-36:Move to before line 13. >> 24:Move constructor to end of class. > Yes, all good. Thanks! Good. I suppose these changes will be in version 7. If it is likely that comments will be forthcoming from others then it might be worth waiting to incorporate any further changes in version 7. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jun 5 17:47:37 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 5 Jun 2019 10:47:37 -0700 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> Message-ID: <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> Hi Andrew, Please ignore the content below: I see the changes are in version 6 which I had not looked through before posting my previous message. Thanks, Brian > On Jun 5, 2019, at 10:25 AM, Brian Burkhalter wrote: > > I suppose these changes will be in version 7. If it is likely that comments will be forthcoming from others then it might be worth waiting to incorporate any further changes in version 7. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jun 5 18:17:48 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 5 Jun 2019 11:17:48 -0700 Subject: RFR: 8207851: Implement JEP 352 In-Reply-To: <3b0f0bcb-4ea0-fe71-acdb-6ca9e44a6d8d@linux.vnet.ibm.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <67ad071d-376b-8b0d-9b2f-42dca17a1041@redhat.com> <3b0f0bcb-4ea0-fe71-acdb-6ca9e44a6d8d@linux.vnet.ibm.com> Message-ID: <1B979240-1C4F-417A-AB82-82A91C0FCA12@oracle.com> Yes, jcheck [1] is a good thing to run for this. Then normalizer [2] can be used to clean them up. Thanks, Brian [1] https://openjdk.java.net/projects/code-tools/jcheck/ [2] /make/scripts/normalizer.pl > On Jun 5, 2019, at 11:13 AM, Gustavo Romero wrote: > > I found some trailing space in v5 and it seems they are in v6 as well. > > You might want to check the followings: -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jun 5 23:04:49 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 5 Jun 2019 16:04:49 -0700 Subject: 8224617: (fs) java/nio/file/FileStore/Basic.java found filesystem twice In-Reply-To: References: <4d6ab69c-a7cd-b3ca-eae7-3b4def6f0d1a@oracle.com> <9EBF8F97-67A3-4F77-91CC-8A0BC898DF7C@oracle.com> Message-ID: <3F4F32B5-2D81-4034-927B-E91389D6F292@oracle.com> > On May 28, 2019, at 11:36 PM, Alan Bateman wrote: > > On 28/05/2019 20:45, Brian Burkhalter wrote: >> >> Yes I changed the name more than once while writing this: > areAllMountPointsAccessible seems okay. > >> >> It is not specified whether it does or does not as far as I can see. The test FileStore/Basic historically only flags duplicates if they are successors in the FileStore iteration. The order of the elements in the iteration is undefined according to the specification however. I would think that either duplicates should be allowed in the iteration or if they are not allowed then the error should be independent of their order. In either of these cases the test has been wrong and still is. >> > I think we'll need to do a bit more investigation on this as FileSystem::getFileStores shouldn't have duplicates (and I think the test was checking for that). So I think we need to dig into how we are ending up with duplicates in the mount table and then decide if they need to be filtered or not. Perhaps we can resolve the issue at hand with [1] and file another issue to investigate whether to filter dups in FilesSystems::getFileStores? Thanks, Brian [1] http://cr.openjdk.java.net/~bpb/8224617/webrev.01 -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Jun 6 00:06:47 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 5 Jun 2019 17:06:47 -0700 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> Message-ID: <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> Hi Andrew, With version 6 [1] of the patch applied, tests throw this exception: Mapped java.lang.InternalError: java.lang.NullPointerException at java.base/jdk.internal.misc.ExtendedMapMode.newMapMode(ExtendedMapMode.java:58) at java.base/jdk.internal.misc.ExtendedMapMode.(ExtendedMapMode.java:40) at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1007) Looks like a problem in the initialization order of the static final constants: MAP_MODE_CONSTRUCTOR needs to be initialized first so I think the *_SYNC constants need to be after the static initializer [2]. Sorry for suggesting the change that was the proximate cause of this. :-( Thanks, Brian [1] http://cr.openjdk.java.net/~adinn/8224974/webrev.06/ [2] diff --- a/src/java.base/share/classes/jdk/internal/misc/ExtendedMapMode.java +++ b/src/java.base/share/classes/jdk/internal/misc/ExtendedMapMode.java @@ -37,10 +37,6 @@ static final MethodHandle MAP_MODE_CONSTRUCTOR; - public static final MapMode READ_ONLY_SYNC = newMapMode("READ_ONLY_SYNC"); - - public static final MapMode READ_WRITE_SYNC = newMapMode("READ_WRITE_SYNC"); - static { try { var lookup = MethodHandles.privateLookupIn(MapMode.class, MethodHandles.lookup()); @@ -51,6 +47,10 @@ } } + public static final MapMode READ_ONLY_SYNC = newMapMode("READ_ONLY_SYNC"); + + public static final MapMode READ_WRITE_SYNC = newMapMode("READ_WRITE_SYNC"); + private static MapMode newMapMode(String name) { try { return (MapMode) MAP_MODE_CONSTRUCTOR.invoke(name); > On Jun 5, 2019, at 10:47 AM, Brian Burkhalter wrote: > > Hi Andrew, > > Please ignore the content below: I see the changes are in version 6 which I had not looked through before posting my previous message. > > Thanks, > > Brian > >> On Jun 5, 2019, at 10:25 AM, Brian Burkhalter > wrote: >> >> I suppose these changes will be in version 7. If it is likely that comments will be forthcoming from others then it might be worth waiting to incorporate any further changes in version 7. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adinn at redhat.com Thu Jun 6 08:53:20 2019 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 6 Jun 2019 09:53:20 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> Message-ID: <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> Hi Brian, On 06/06/2019 01:06, Brian Burkhalter wrote: > With version 6 [1] of the patch applied, tests throw this exception: > > Mapped java.lang.InternalError: java.lang.NullPointerException > at > java.base/jdk.internal.misc.ExtendedMapMode.newMapMode(ExtendedMapMode.java:58) > at > java.base/jdk.internal.misc.ExtendedMapMode.(ExtendedMapMode.java:40) > at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1007) > > Looks like a problem in the initialization order of the static final > constants: MAP_MODE_CONSTRUCTOR needs to be initialized first so I think > the *_SYNC constants need to be after the static initializer [2]. Doh! Yes, you are right. Sorry for adding confusion. webrev.06 was not meant to be intended for public consumption. I uploaded it last night when I set the rebuild off but I didn't intend to advertise it until I had retested and got your confirmation on the suggestions I made. Anyway, thanks all the same for reviewing it, trying to run it and also finding/fixing this issue. I have updated webrev.06 in place and PmemTest now runs ok. I still need to have Jonathan Halliday test the patch with our Transaction and Data Grid software before it can be properly signed off. Another submit run will also be needed. However, PmemTest is a good enough sanity check to validate the changes you requested. So, here is webrev.06 as the latest official patch: http://cr.openjdk.java.net/~adinn/8224974/webrev.06/ It should address changes requested on this list by Alan and Vladimir all the changes you requested in your review (modulo those you agreed to defer on) the change to correct this static init order bug a change to graal test CheckGraalIntrinsics to declare the new jdk13 intrinsics in alphanum sort order (this was requested offline by Alan -- without it the test refuses to run on jdk13) The last of those changes still leaves one issue unanswered (although it may not actually be an issue). Vladimir is probably best able to comment. When I run CheckGraalIntrinsics (which is actually run indirectly by compiler/graalunit/HotspotTest) with my patched tree it fails, claiming that Graal does not know about Unsafe.writeback0: java.lang.AssertionError: missing Graal intrinsics for: jdk/internal/misc/Unsafe.writeback0(J)V I am assuming this is what is expected to happen? Or perhaps the test is supposed to be added to the problem or exclude lists? > Sorry for suggesting the change that was the proximate cause of this. :-( No problem :-) Thanks for reviewing and checking! regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Thu Jun 6 09:13:41 2019 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 6 Jun 2019 10:13:41 +0100 Subject: RFR: 8207851: Implement JEP 352 In-Reply-To: <3b0f0bcb-4ea0-fe71-acdb-6ca9e44a6d8d@linux.vnet.ibm.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <67ad071d-376b-8b0d-9b2f-42dca17a1041@redhat.com> <3b0f0bcb-4ea0-fe71-acdb-6ca9e44a6d8d@linux.vnet.ibm.com> Message-ID: Hi Gustavo, On 05/06/2019 19:13, Gustavo Romero wrote: > I found some trailing space in v5 and it seems they are in v6 as well. Thanks for the heads up. I will fix these in the next webrev (07). regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Thu Jun 6 09:16:43 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 6 Jun 2019 10:16:43 +0100 Subject: 8224617: (fs) java/nio/file/FileStore/Basic.java found filesystem twice In-Reply-To: <3F4F32B5-2D81-4034-927B-E91389D6F292@oracle.com> References: <4d6ab69c-a7cd-b3ca-eae7-3b4def6f0d1a@oracle.com> <9EBF8F97-67A3-4F77-91CC-8A0BC898DF7C@oracle.com> <3F4F32B5-2D81-4034-927B-E91389D6F292@oracle.com> Message-ID: On 06/06/2019 00:04, Brian Burkhalter wrote: > > > Perhaps we can resolve the issue at hand with [1] and file another > issue to investigate whether to filter dups in > FilesSystems::getFileStores? > That should be okay. The duplicate check in the FileUtils test library can be removed if the implementation is updated. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Thu Jun 6 11:13:38 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 6 Jun 2019 12:13:38 +0100 Subject: [RFR] 8224645: Test java/nio/channels/DatagramChannel/BasicMulticastTests.java fails with NoSuchElementException In-Reply-To: <6dca339e-69e1-328b-faf5-9b2cc9a71168@oracle.com> References: <8fa9d55a-520e-df06-1d5d-566ef39fa992@oracle.com> <27fa2523-2f81-10c9-0688-3de42af96b02@oracle.com> <7e2ead05-f49c-8ca3-9bd5-b6d1e3e6ec21@oracle.com> <548abdef-3f06-0947-2e00-e05cc59f626f@oracle.com> <6dca339e-69e1-328b-faf5-9b2cc9a71168@oracle.com> Message-ID: Hi Arthur, Looks good to me too. I have run it through our test system and saw no issues. best regards, -- daniel On 04/06/2019 11:27, Alan Bateman wrote: > On 03/06/2019 16:17, Arthur Eubanks wrote: >> >> I don't think it's a big difference either way, but changed it to >> System.out. Also added some extra info to the "Exception Tests" >> log to show the addresses it's testing. Now it looks like: >> > webrev.03 looks okay to me. > From chris.hegarty at oracle.com Thu Jun 6 11:23:57 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 6 Jun 2019 12:23:57 +0100 Subject: [RFR] 8224645: Test java/nio/channels/DatagramChannel/BasicMulticastTests.java fails with NoSuchElementException In-Reply-To: References: <8fa9d55a-520e-df06-1d5d-566ef39fa992@oracle.com> <27fa2523-2f81-10c9-0688-3de42af96b02@oracle.com> <7e2ead05-f49c-8ca3-9bd5-b6d1e3e6ec21@oracle.com> <548abdef-3f06-0947-2e00-e05cc59f626f@oracle.com> <6dca339e-69e1-328b-faf5-9b2cc9a71168@oracle.com> Message-ID: +1 from me too. Arthur, This test is failing quite frequently in our build and test system. If not pushed by this time tomorrow, I will push it on your behalf. -Chris. > On 6 Jun 2019, at 12:13, Daniel Fuchs wrote: > > Hi Arthur, > > Looks good to me too. > I have run it through our test system and saw no issues. > > best regards, > > -- daniel > > On 04/06/2019 11:27, Alan Bateman wrote: >> On 03/06/2019 16:17, Arthur Eubanks wrote: >>> >>> I don't think it's a big difference either way, but changed it to >>> System.out. Also added some extra info to the "Exception Tests" >>> log to show the addresses it's testing. Now it looks like: >>> >> webrev.03 looks okay to me. > From adinn at redhat.com Thu Jun 6 12:15:15 2019 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 6 Jun 2019 13:15:15 +0100 Subject: RFR: 8207851: Implement JEP 352 In-Reply-To: <5c3af4bf-32f8-2f73-2263-b3aa77e84c66@redhat.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <67ad071d-376b-8b0d-9b2f-42dca17a1041@redhat.com> <3b0f0bcb-4ea0-fe71-acdb-6ca9e44a6d8d@linux.vnet.ibm.com> <5c3af4bf-32f8-2f73-2263-b3aa77e84c66@redhat.com> Message-ID: On 06/06/2019 10:46, Andrew Haley wrote: > Handy hint. In your .emacs, add this: > > ;;; goodbye trailing whitespace blues!!! > > (add-hook 'java-mode-hook > (lambda () > (progn > (set-variable 'show-trailing-whitespace t) > ))) > > (add-hook 'c-mode-hook > (lambda () > (progn > (set-variable 'show-trailing-whitespace t) > ))) > > > (add-hook 'c++-mode-hook > (lambda () > (progn > (set-variable 'show-trailing-whitespace t) > ))) There, fixed that for ya! > ...and you'll never have any trouble with trailing whitespace again. Now working beautifully, thank you! regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Thu Jun 6 14:22:12 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 6 Jun 2019 15:22:12 +0100 Subject: Invalid argument exception when trying to seek FilePosition to LONG.MAX_VALUE In-Reply-To: References: <3c8a1d34-37f7-6598-ad6d-7f2648173ac9@oracle.com> <87r288t5fm.fsf@oldenburg2.str.redhat.com> Message-ID: <884093d3-4c9a-2c70-cacc-81010f6c6a0d@oracle.com> On 05/06/2019 18:06, Karan Mehta wrote: > : > > Let me clarify this a bit further. If the underlying FS throws an > exception while seeking to LONG.MAX_VALUE, the JNI call should catch > it and return a No-op to the Java function, instead of throwing it > back further. I don't think this make sense. The position(long) method is specified to throw IOException when it fails so I'm wondering what you expect the file position to be if it silently fails? > > > This is with ext4, right?? XFS is fine with it and can even write there: > > Can't speak for XFS, but yeah with ext4 is where I am seeing the issue. > Thanks Florian for the additional information on this. -Alan. From brian.burkhalter at oracle.com Thu Jun 6 16:24:42 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 6 Jun 2019 09:24:42 -0700 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> Message-ID: <13D82172-FEA3-4F9C-8887-974A770E3FFA@oracle.com> Hi Andrew, > On Jun 6, 2019, at 1:53 AM, Andrew Dinn wrote: > > [?] > > I have updated webrev.06 in place and PmemTest now runs ok. I still need > to have Jonathan Halliday test the patch with our Transaction and Data > Grid software before it can be properly signed off. Another submit run > will also be needed. However, PmemTest is a good enough sanity check to > validate the changes you requested. > > So, here is webrev.06 as the latest official patch: > > http://cr.openjdk.java.net/~adinn/8224974/webrev.06/ I am running it through our tests now. > It should address > > [?] > > The last of those changes still leaves one issue unanswered (although it > may not actually be an issue). Vladimir is probably best able to comment. > > When I run CheckGraalIntrinsics (which is actually run indirectly by > compiler/graalunit/HotspotTest) with my patched tree it fails, claiming > that Graal does not know about Unsafe.writeback0: > > java.lang.AssertionError: missing Graal intrinsics for: > jdk/internal/misc/Unsafe.writeback0(J)V > > I am assuming this is what is expected to happen? Or perhaps the test is > supposed to be added to the problem or exclude lists? I?ll leave it to Vladimir or someone else more knowledgeable than I to comment on this area. >> Sorry for suggesting the change that was the proximate cause of this. :-( > No problem :-) Thanks for reviewing and checking! You are welcome! Cheers, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From aeubanks at google.com Thu Jun 6 17:02:51 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Thu, 6 Jun 2019 10:02:51 -0700 Subject: [RFR] 8224645: Test java/nio/channels/DatagramChannel/BasicMulticastTests.java fails with NoSuchElementException In-Reply-To: References: <8fa9d55a-520e-df06-1d5d-566ef39fa992@oracle.com> <27fa2523-2f81-10c9-0688-3de42af96b02@oracle.com> <7e2ead05-f49c-8ca3-9bd5-b6d1e3e6ec21@oracle.com> <548abdef-3f06-0947-2e00-e05cc59f626f@oracle.com> <6dca339e-69e1-328b-faf5-9b2cc9a71168@oracle.com> Message-ID: Pushed, thanks for the reviews. On Thu, Jun 6, 2019 at 4:23 AM Chris Hegarty wrote: > +1 from me too. > > Arthur, > > This test is failing quite frequently in our build and test system. If not > pushed by this time tomorrow, I will push it on your behalf. > > -Chris. > > > On 6 Jun 2019, at 12:13, Daniel Fuchs wrote: > > > > Hi Arthur, > > > > Looks good to me too. > > I have run it through our test system and saw no issues. > > > > best regards, > > > > -- daniel > > > > On 04/06/2019 11:27, Alan Bateman wrote: > >> On 03/06/2019 16:17, Arthur Eubanks wrote: > >>> > >>> I don't think it's a big difference either way, but changed it to > >>> System.out. Also added some extra info to the "Exception Tests" > >>> log to show the addresses it's testing. Now it looks like: > >>> > >> webrev.03 looks okay to me. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From k.mehta at salesforce.com Thu Jun 6 18:24:09 2019 From: k.mehta at salesforce.com (Karan Mehta) Date: Thu, 6 Jun 2019 11:24:09 -0700 Subject: Invalid argument exception when trying to seek FilePosition to LONG.MAX_VALUE In-Reply-To: <884093d3-4c9a-2c70-cacc-81010f6c6a0d@oracle.com> References: <3c8a1d34-37f7-6598-ad6d-7f2648173ac9@oracle.com> <87r288t5fm.fsf@oldenburg2.str.redhat.com> <884093d3-4c9a-2c70-cacc-81010f6c6a0d@oracle.com> Message-ID: > I don't think this make sense. The position(long) method is specified to throw IOException when it fails so I'm wondering what you expect the file position to be if it silently fails? In hindsight, I realize that your suggestion seems right. By a quick search, I didn't find a way of figuring the underlying file system in Java or not, so I guess IOException handling is the way to go. The application should determine this appropriately then. I hope we can update the javadoc here to reflect that things can change based on underlying FS. Thanks Alan and Florian for the help. On Thu, Jun 6, 2019 at 7:22 AM Alan Bateman wrote: > > > On 05/06/2019 18:06, Karan Mehta wrote: > > : > > > > Let me clarify this a bit further. If the underlying FS throws an > > exception while seeking to LONG.MAX_VALUE, the JNI call should catch > > it and return a No-op to the Java function, instead of throwing it > > back further. > I don't think this make sense. The position(long) method is specified to > throw IOException when it fails so I'm wondering what you expect the > file position to be if it silently fails? > > > > > > > This is with ext4, right? XFS is fine with it and can even write > there: > > > > Can't speak for XFS, but yeah with ext4 is where I am seeing the issue. > > > Thanks Florian for the additional information on this. > > -Alan. > -- Karan Mehta -------------- next part -------------- An HTML attachment was scrubbed... URL: From fweimer at redhat.com Thu Jun 6 18:30:07 2019 From: fweimer at redhat.com (Florian Weimer) Date: Thu, 06 Jun 2019 20:30:07 +0200 Subject: Invalid argument exception when trying to seek FilePosition to LONG.MAX_VALUE In-Reply-To: (Karan Mehta's message of "Wed, 5 Jun 2019 10:06:42 -0700") References: <3c8a1d34-37f7-6598-ad6d-7f2648173ac9@oracle.com> <87r288t5fm.fsf@oldenburg2.str.redhat.com> Message-ID: <87r286mus0.fsf@oldenburg2.str.redhat.com> * Karan Mehta: > Let me clarify this a bit further. If the underlying FS throws an > exception while seeking to LONG.MAX_VALUE, the JNI call should catch > it and return a No-op to the Java function, instead of throwing it > back further. That would cause subsequent operations use the previous file offset, possibly leading to data corruption. I don't think that's a good idea. Thanks, Florian From brian.burkhalter at oracle.com Thu Jun 6 18:59:56 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 6 Jun 2019 11:59:56 -0700 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <13D82172-FEA3-4F9C-8887-974A770E3FFA@oracle.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <13D82172-FEA3-4F9C-8887-974A770E3FFA@oracle.com> Message-ID: <9F355689-D204-4C28-9EA3-602930CA07D1@oracle.com> Hi Andrew, > On Jun 6, 2019, at 9:24 AM, Brian Burkhalter wrote: > >> So, here is webrev.06 as the latest official patch: >> >> http://cr.openjdk.java.net/~adinn/8224974/webrev.06/ I created a delta webrev versus version 5 and this looks good modulo the trailing whitespace which I think you wrote you were going to fix in version 7. > I am running it through our tests now. All tier 1-3 tests passed on our four usual platforms. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Jun 6 21:49:25 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 6 Jun 2019 14:49:25 -0700 Subject: 8224617: (fs) java/nio/file/FileStore/Basic.java found filesystem twice In-Reply-To: References: <4d6ab69c-a7cd-b3ca-eae7-3b4def6f0d1a@oracle.com> <9EBF8F97-67A3-4F77-91CC-8A0BC898DF7C@oracle.com> <3F4F32B5-2D81-4034-927B-E91389D6F292@oracle.com> Message-ID: <1EE7EBB0-03C7-42AD-A469-8731E63C650C@oracle.com> > On Jun 6, 2019, at 2:16 AM, Alan Bateman wrote: > > On 06/06/2019 00:04, Brian Burkhalter wrote: >> >> >> Perhaps we can resolve the issue at hand with [1] and file another issue to investigate whether to filter dups in FilesSystems::getFileStores? >> > That should be okay. The duplicate check in the FileUtils test library can be removed if the implementation is updated. OK I pushed the patch for the present issue and filed https://bugs.openjdk.java.net/browse/JDK-8225461 to track possibly filtering dups out in FileSystem::getFileStores. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From adinn at redhat.com Fri Jun 7 11:24:33 2019 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 7 Jun 2019 12:24:33 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> Message-ID: <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> On 06/06/2019 18:40, Vladimir Kozlov wrote: > Hotspot changes looks good. > > CheckGraalIntrinsics failed because of typo - you should use 'J' instead > of 'L': > > +? "jdk/internal/misc/Unsafe.writeback0(L)V", Doh! Thanks, Vladimir :-) With that correction the test now passes. > One nitpick in vmSymbols.hpp spacing is off in next line: > > +? do_intrinsic(_writebackPostSync0,??????? > jdk_internal_misc_Unsafe,???? writebackPostSync0_name, > void_method_signature , F_RN)?? \ That line and the one two lines further on were both misaligned. I have uploaded a new webrev to fix the bove problems. This version also removes all the extra extraneous whitespace found by Brian and Gustavo. webrev: http://cr.openjdk.java.net/~adinn/8224974/webrev.07/ I have posted the changes to the submit repo to re-verify that all builds pass. I have also asked Jonathan Halliday to re-test this version against the Red Hat middleware clients to ensure there are no functional or performance changes. Modulo confirmation of those two checks this looks like it is a complete implementation. Unless anyone has more changes to recommend? regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Fri Jun 7 14:10:49 2019 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 7 Jun 2019 15:10:49 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> Message-ID: <08aef662-69e2-eb70-ef83-043837dbdd69@redhat.com> On 07/06/2019 12:24, Andrew Dinn wrote: > On 06/06/2019 18:40, Vladimir Kozlov wrote: >> Hotspot changes looks good. >> >> CheckGraalIntrinsics failed because of typo - you should use 'J' instead >> of 'L': >> >> +? "jdk/internal/misc/Unsafe.writeback0(L)V", > > Doh! Thanks, Vladimir :-) > > With that correction the test now passes. > >> One nitpick in vmSymbols.hpp spacing is off in next line: >> >> +? do_intrinsic(_writebackPostSync0,??????? >> jdk_internal_misc_Unsafe,???? writebackPostSync0_name, >> void_method_signature , F_RN)?? \ > That line and the one two lines further on were both misaligned. > > I have uploaded a new webrev to fix the bove problems. This version also > removes all the extra extraneous whitespace found by Brian and Gustavo. > > webrev: http://cr.openjdk.java.net/~adinn/8224974/webrev.07/ > > I have posted the changes to the submit repo to re-verify that all > builds pass. I have also asked Jonathan Halliday to re-test this version > against the Red Hat middleware clients to ensure there are no functional > or performance changes. > > Modulo confirmation of those two checks this looks like it is a complete > implementation. Unless anyone has more changes to recommend? I just want to confirm that the submit job was clean and the middleware clients are performing as expected. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Sun Jun 9 15:53:55 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 9 Jun 2019 16:53:55 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> Message-ID: <7b39f68d-5698-4079-425c-c86ec161e361@oracle.com> On 07/06/2019 12:24, Andrew Dinn wrote: > : > Modulo confirmation of those two checks this looks like it is a complete > implementation. Unless anyone has more changes to recommend? > I read through the recent mails and I think you are nearly done with the code review. There are still a few updates to be done to the JEP and it will need to be endorsed before propose-to-target. Given that JDK 13 enters RDP1 on June 13 then I assume it's best to re-target the CSR to 14 and for the JEP to become a candidate for that release. -Alan. From adinn at redhat.com Mon Jun 10 07:56:59 2019 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 10 Jun 2019 08:56:59 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <7b39f68d-5698-4079-425c-c86ec161e361@oracle.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> <7b39f68d-5698-4079-425c-c86ec161e361@oracle.com> Message-ID: <72810832-eb88-3418-f336-b95af95d9dcc@redhat.com> On 09/06/2019 16:53, Alan Bateman wrote: > On 07/06/2019 12:24, Andrew Dinn wrote: >> : >> Modulo confirmation of those two checks this looks like it is a complete >> implementation. Unless anyone has more changes to recommend? >> > I read through the recent mails and I think you are nearly done with the > code review. There are still a few updates to be done to the JEP and it > will need to be endorsed before propose-to-target. Given that JDK 13 > enters RDP1 on June 13 then I assume it's best to re-target the CSR to > 14 and for the JEP to become a candidate for that release. Yes, I agree it is too late to squeeze this into 13. I have updated the fix version for the JEP and CSR to 14: JEP: https://bugs.openjdk.java.net/browse/JDK-8207851 CSR: https://bugs.openjdk.java.net/browse/JDK-8224975 I'll wait for any further feedback on the JEP and implementation. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Mon Jun 10 08:18:06 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 10 Jun 2019 09:18:06 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <72810832-eb88-3418-f336-b95af95d9dcc@redhat.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> <7b39f68d-5698-4079-425c-c86ec161e361@oracle.com> <72810832-eb88-3418-f336-b95af95d9dcc@redhat.com> Message-ID: <20c8bbcb-2cd3-77af-cd40-29f7bd752166@oracle.com> On 10/06/2019 08:56, Andrew Dinn wrote: > : > Yes, I agree it is too late to squeeze this into 13. I have updated the > fix version for the JEP and CSR to 14: > > JEP: https://bugs.openjdk.java.net/browse/JDK-8207851 > CSR: https://bugs.openjdk.java.net/browse/JDK-8224975 > > I'll wait for any further feedback on the JEP and implementation. > The new release cadence is wonderful for situations like this as JDK 14 is only 6 months after 13. I don't have any other feedback on the JEP beyond my last mail about removing the "Proposed Java API Changes" section from the Description. Once we you ready then Brian (as area lead) should be contacted to seek his endorsement. -Alan [1] https://mail.openjdk.java.net/pipermail/nio-dev/2019-June/006238.html From adinn at redhat.com Mon Jun 10 10:09:03 2019 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 10 Jun 2019 11:09:03 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <20c8bbcb-2cd3-77af-cd40-29f7bd752166@oracle.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> <7b39f68d-5698-4079-425c-c86ec161e361@oracle.com> <72810832-eb88-3418-f336-b95af95d9dcc@redhat.com> <20c8bbcb-2cd3-77af-cd40-29f7bd752166@oracle.com> Message-ID: <68566847-de80-82af-a928-d30c75f2b1b2@redhat.com> Hi Alan, On 10/06/2019 09:18, Alan Bateman wrote: > I don't have any other feedback on the JEP beyond my last mail about > removing the "Proposed Java API Changes" section from the Description. > Once we you ready then Brian (as area lead) should be contacted to seek > his endorsement. I have updated the Proposed Java API Changes to remove the changes to map exception signature and force region specification. They were covered in the prior enabling JIRAs/CSRs. So, the remaining two sections mention 1) the new module + package + enum and 2) the bean counting changes. The new section 1 clarifies when UnsupportedOperationException is thrown vs when IOException is thrown as part of the explanation of what the new modes are for. I also added a paragraph to the alternatives section explaining that Panama was and still is an alternative option and why we decided to proceed with this model despite it being still under consideration. Finally, I am unclear whether the presence of the new module + package means this is an SE JEP or a JDK JEP? In the latest update I have assumed the JEP type is still JDK and changed the title for the public API changes to "Proposed Java API Changes". Is that right? Or do you want the JEP type to be SE and have the title remain "Proposed Java SE API Changes" regards, Andrew Dinn ----------- From michael.x.mcmahon at oracle.com Mon Jun 10 14:18:19 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Mon, 10 Jun 2019 15:18:19 +0100 Subject: RFR 8216417: cleanup of IPv6 scope-id handling Message-ID: <5CFE66AB.9060105@oracle.com> Hi, Could I get the following change to net/nio reviewed please? It is a general cleanup of IPv6 scope_id handling which removes a lot of native code trickery (mostly in Linux) and simplifies the handling of scope_ids such that: a) when binding/connecting/sending to a link-local address on the same machine, it is allowed to use addresses without a scope_id. In this case, we check local NetworkInterfaces for the correct scope_id to use. b) when connecting/sending to a remote link-local address, then a scope_id must be specified at all times. Scope_ids should be handled consistently between net and NIO and across all platforms now. http://cr.openjdk.java.net/~michaelm/8216417/webrev.2/index.html Thanks, Michael From Alan.Bateman at oracle.com Mon Jun 10 15:49:32 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 10 Jun 2019 16:49:32 +0100 Subject: RFR 8216417: cleanup of IPv6 scope-id handling In-Reply-To: <5CFE66AB.9060105@oracle.com> References: <5CFE66AB.9060105@oracle.com> Message-ID: <97b2327f-9b2d-65c5-7d3b-cddc2be787d4@oracle.com> On 10/06/2019 15:18, Michael McMahon wrote: > Hi, > > Could I get the following change to net/nio reviewed please? > > It is a general cleanup of IPv6 scope_id handling which removes > a lot of native code trickery (mostly in Linux) and simplifies the > handling > of scope_ids such that: > > a) when binding/connecting/sending to a link-local address on the same > machine, > ?? it is allowed to use addresses without a scope_id. In this case, we > check local NetworkInterfaces > ?? for the correct scope_id to use. > > b) when connecting/sending to a remote link-local address, then a > scope_id must be specified > ??? at all times. > > Scope_ids should be handled consistently between net and NIO and > across all platforms now. > > http://cr.openjdk.java.net/~michaelm/8216417/webrev.2/index.html This looks like a really good cleanup. One thing I notice is that AbstractPlainSocketImpl.connect sets this.address before converting it to the scoped address. This impacts the address returned by Socket::getInetAddress so I think it needs to set this.address first. DatagramChannel.send will need to use toScopeAddress(InetSocketAddress) for the "not connected" case, probably after the SM check. There is probably a slightly bigger question here on on SM checks and? whether it's interesting to have the scope ID in the address or not. Long standing behavior seems to be to call the SM with the address that the user has specified and this may or may not include the scope ID. -Alan From Alan.Bateman at oracle.com Mon Jun 10 17:44:04 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 10 Jun 2019 18:44:04 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <68566847-de80-82af-a928-d30c75f2b1b2@redhat.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> <7b39f68d-5698-4079-425c-c86ec161e361@oracle.com> <72810832-eb88-3418-f336-b95af95d9dcc@redhat.com> <20c8bbcb-2cd3-77af-cd40-29f7bd752166@oracle.com> <68566847-de80-82af-a928-d30c75f2b1b2@redhat.com> Message-ID: <072ce1bb-ad2c-a2cd-42fc-b4076dac99c7@oracle.com> On 10/06/2019 11:09, Andrew Dinn wrote: > : > Finally, I am unclear whether the presence of the new module + package > means this is an SE JEP or a JDK JEP? In the latest update I have > assumed the JEP type is still JDK and changed the title for the public > API changes to "Proposed Java API Changes". Is that right? Or do you > want the JEP type to be SE and have the title remain "Proposed Java SE > API Changes" > The module is JDK-specific so keeping the scope "JDK" is right. -Alan From daniel.fuchs at oracle.com Mon Jun 10 18:48:59 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 10 Jun 2019 19:48:59 +0100 Subject: RFR 8216417: cleanup of IPv6 scope-id handling In-Reply-To: <5CFE66AB.9060105@oracle.com> References: <5CFE66AB.9060105@oracle.com> Message-ID: <02a3a922-9ed9-0f6f-e505-b138988f3115@oracle.com> Hi Michael, AbstractPlainDatagramSocketImpl.java: in `send` maybe the packet should only be recreated if the address is different. best regards, -- daniel On 10/06/2019 15:18, Michael McMahon wrote: > Hi, > > Could I get the following change to net/nio reviewed please? > > It is a general cleanup of IPv6 scope_id handling which removes > a lot of native code trickery (mostly in Linux) and simplifies the handling > of scope_ids such that: > > a) when binding/connecting/sending to a link-local address on the same > machine, > ?? it is allowed to use addresses without a scope_id. In this case, we > check local NetworkInterfaces > ?? for the correct scope_id to use. > > b) when connecting/sending to a remote link-local address, then a > scope_id must be specified > ??? at all times. > > Scope_ids should be handled consistently between net and NIO and across > all platforms now. > > http://cr.openjdk.java.net/~michaelm/8216417/webrev.2/index.html > > Thanks, > Michael From brian.burkhalter at oracle.com Mon Jun 10 23:26:43 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 10 Jun 2019 16:26:43 -0700 Subject: 8225461: (fs) Investigate filtering duplicates out of Iterable returned by java.nio.file.FileSystem::getFileStores Message-ID: https://bugs.openjdk.java.net/browse/JDK-8225461 http://cr.openjdk.java.net/~bpb/8225461/webrev.00/ This implements, at least for Unix file systems, filtering out any duplicate mount entries so that FileSystem::getFileStores() does not contain any duplicates. It also modifies the test java/nio/file/FileStore/Basic.java to check for duplicate FileStores instead of just two successive stores being equal. In the test the call to FileUtils.areAllMountPointsAccessible() is replaced with FileUtils.areFileSystemsAccessible() and the former method is removed from jdk.test.lib.util.FileUtils as this was its only use. I don?t know whether the specification of FileSystem::getFileStores() should be updated to indicate that no duplicate stores are returned, which would necessitate a CSR. Thanks, Brian From brian.burkhalter at oracle.com Mon Jun 10 23:30:14 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 10 Jun 2019 16:30:14 -0700 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <08aef662-69e2-eb70-ef83-043837dbdd69@redhat.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <506d2bee-9376-52f7-731c-4d872c944847@oracle.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> <08aef662-69e2-eb70-ef83-043837dbdd69@redhat.com> Message-ID: Hi Andrew, > On Jun 7, 2019, at 7:10 AM, Andrew Dinn wrote: > >> I have uploaded a new webrev to fix the bove problems. This version also >> removes all the extra extraneous whitespace found by Brian and Gustavo. >> >> webrev: http://cr.openjdk.java.net/~adinn/8224974/webrev.07/ >> >> I have posted the changes to the submit repo to re-verify that all >> builds pass. I have also asked Jonathan Halliday to re-test this version >> against the Red Hat middleware clients to ensure there are no functional >> or performance changes. >> >> Modulo confirmation of those two checks this looks like it is a complete >> implementation. Unless anyone has more changes to recommend? > I just want to confirm that the submit job was clean and the middleware > clients are performing as expected. I wanted to let you know that I ran version 7 above through our tests and did not see any problems related to your changes. I think the copyright years in PmemTest.java are wrong however: 2002, 2011. I suppose it should be just 2019. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.x.mcmahon at oracle.com Tue Jun 11 09:04:02 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Tue, 11 Jun 2019 10:04:02 +0100 Subject: RFR 8216417: cleanup of IPv6 scope-id handling In-Reply-To: <02a3a922-9ed9-0f6f-e505-b138988f3115@oracle.com> References: <5CFE66AB.9060105@oracle.com> <02a3a922-9ed9-0f6f-e505-b138988f3115@oracle.com> Message-ID: <5CFF6E82.2020201@oracle.com> Good point Daniel. Thanks for the review. - Michael. On 10/06/2019, 19:48, Daniel Fuchs wrote: > Hi Michael, > > AbstractPlainDatagramSocketImpl.java: > > in `send` maybe the packet should only be recreated if the address > is different. > > best regards, > > -- daniel > > On 10/06/2019 15:18, Michael McMahon wrote: >> Hi, >> >> Could I get the following change to net/nio reviewed please? >> >> It is a general cleanup of IPv6 scope_id handling which removes >> a lot of native code trickery (mostly in Linux) and simplifies the >> handling >> of scope_ids such that: >> >> a) when binding/connecting/sending to a link-local address on the >> same machine, >> it is allowed to use addresses without a scope_id. In this case, >> we check local NetworkInterfaces >> for the correct scope_id to use. >> >> b) when connecting/sending to a remote link-local address, then a >> scope_id must be specified >> at all times. >> >> Scope_ids should be handled consistently between net and NIO and >> across all platforms now. >> >> http://cr.openjdk.java.net/~michaelm/8216417/webrev.2/index.html >> >> Thanks, >> Michael > From adinn at redhat.com Tue Jun 11 09:21:43 2019 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 11 Jun 2019 10:21:43 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <607e2f32-1f01-e45f-42e8-a645fa374cf7@oracle.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> <08aef662-69e2-eb70-ef83-043837dbdd69@redhat.com> Message-ID: <7a742f60-4b47-407b-4667-d007cc130a7f@redhat.com> On 11/06/2019 00:30, Brian Burkhalter wrote: > I wanted to let you know that I ran version 7 above through our tests > and did not see any problems related to your changes. Excellent. Thanks very much for running that check. > I think the copyright years in PmemTest.java are wrong however: 2002, > 2011. I suppose it should be just 2019. Well spotted! Let's hope that is the final escapee. I have tweaked my latest patch accordingly. I have also updated the patch to specify @since 14 instead of @since 13 in the two places where new elements are added. I'll post a final webrev once I am sure there are no other changes needed. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Tue Jun 11 09:23:32 2019 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 11 Jun 2019 10:23:32 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <072ce1bb-ad2c-a2cd-42fc-b4076dac99c7@oracle.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> <7b39f68d-5698-4079-425c-c86ec161e361@oracle.com> <72810832-eb88-3418-f336-b95af95d9dcc@redhat.com> <20c8bbcb-2cd3-77af-cd40-29f7bd752166@oracle.com> <68566847-de80-82af-a928-d30c75f2b1b2@redhat.com> <072ce1bb-ad2c-a2cd-42fc-b4076dac99c7@oracle.com> Message-ID: On 10/06/2019 18:44, Alan Bateman wrote: > On 10/06/2019 11:09, Andrew Dinn wrote: >> : >> Finally, I am unclear whether the presence of the new module + package >> means this is an SE JEP or a JDK JEP? In the latest update I have >> assumed the JEP type is still JDK and changed the title for the public >> API changes to "Proposed Java API Changes". Is that right? Or do you >> want the JEP type to be SE and have the title remain "Proposed Java SE >> API Changes" >> > The module is JDK-specific so keeping the scope "JDK" is right. Ok, in that case I'll ask Brian if he can endorse the JEP and then target for JDK14. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Tue Jun 11 10:46:42 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Jun 2019 11:46:42 +0100 Subject: 8225461: (fs) Investigate filtering duplicates out of Iterable returned by java.nio.file.FileSystem::getFileStores In-Reply-To: References: Message-ID: <9d73d427-f83a-cedc-45cb-1c6fa1299306@oracle.com> On 11/06/2019 00:26, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8225461 > http://cr.openjdk.java.net/~bpb/8225461/webrev.00/ > > This implements, at least for Unix file systems, filtering out any duplicate mount entries so that FileSystem::getFileStores() does not contain any duplicates. It also modifies the test java/nio/file/FileStore/Basic.java to check for duplicate FileStores instead of just two successive stores being equal. In the test the call to FileUtils.areAllMountPointsAccessible() is replaced with FileUtils.areFileSystemsAccessible() and the former method is removed from jdk.test.lib.util.FileUtils as this was its only use. > > I don?t know whether the specification of FileSystem::getFileStores() should be updated to indicate that no duplicate stores are returned, which would necessitate a CSR. > This issue is going a bit of time to review as it changes the meaning of equals/hashCode and has several other implications that will take time to work through. I will try to get to it soon, but maybe not this week. -Alan From michael.x.mcmahon at oracle.com Tue Jun 11 11:11:34 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Tue, 11 Jun 2019 12:11:34 +0100 Subject: RFR 8216417: cleanup of IPv6 scope-id handling In-Reply-To: <97b2327f-9b2d-65c5-7d3b-cddc2be787d4@oracle.com> References: <5CFE66AB.9060105@oracle.com> <97b2327f-9b2d-65c5-7d3b-cddc2be787d4@oracle.com> Message-ID: <5CFF8C66.8060309@oracle.com> Thanks for the review Alan. I've made the changes suggested by you and Daniel and added a test to Scoping.java for checking the connected IP address. http://cr.openjdk.java.net/~michaelm/8216417/webrev.3/index.html - Michael On 10/06/2019, 16:49, Alan Bateman wrote: > On 10/06/2019 15:18, Michael McMahon wrote: >> Hi, >> >> Could I get the following change to net/nio reviewed please? >> >> It is a general cleanup of IPv6 scope_id handling which removes >> a lot of native code trickery (mostly in Linux) and simplifies the >> handling >> of scope_ids such that: >> >> a) when binding/connecting/sending to a link-local address on the >> same machine, >> it is allowed to use addresses without a scope_id. In this case, >> we check local NetworkInterfaces >> for the correct scope_id to use. >> >> b) when connecting/sending to a remote link-local address, then a >> scope_id must be specified >> at all times. >> >> Scope_ids should be handled consistently between net and NIO and >> across all platforms now. >> >> http://cr.openjdk.java.net/~michaelm/8216417/webrev.2/index.html > This looks like a really good cleanup. > > One thing I notice is that AbstractPlainSocketImpl.connect sets > this.address before converting it to the scoped address. This impacts > the address returned by Socket::getInetAddress so I think it needs to > set this.address first. > > DatagramChannel.send will need to use > toScopeAddress(InetSocketAddress) for the "not connected" case, > probably after the SM check. There is probably a slightly bigger > question here on on SM checks and whether it's interesting to have > the scope ID in the address or not. Long standing behavior seems to be > to call the SM with the address that the user has specified and this > may or may not include the scope ID. > > -Alan > From Alan.Bateman at oracle.com Tue Jun 11 12:19:08 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Jun 2019 13:19:08 +0100 Subject: RFR 8216417: cleanup of IPv6 scope-id handling In-Reply-To: <5CFF8C66.8060309@oracle.com> References: <5CFE66AB.9060105@oracle.com> <97b2327f-9b2d-65c5-7d3b-cddc2be787d4@oracle.com> <5CFF8C66.8060309@oracle.com> Message-ID: On 11/06/2019 12:11, Michael McMahon wrote: > Thanks for the review Alan. > I've made the changes suggested by you and Daniel > and added a test to Scoping.java for checking the connected IP address. > > http://cr.openjdk.java.net/~michaelm/8216417/webrev.3/index.html One suggestion for the AbstractPlainSocketImpl connect methods is to set this.address and this.port together and include a comment to say that the socket address is recorded before attempting to connect. A few formatting nits to check: AbstractPlainDatagramSocketImpl L134 and the spurious blank line at IPAddressUtil L355. In the PromiscuousIPv6 test then I'm wondering if the check for the 3.10.0.* kernel is needed or not. I suspect this may have been in Pavel's original changes. Maybe Pavel has the history on that? I see the Scoping test has been expanded. In datagramTest you can use try-with-resources. Also shouldn't there be an equivalent for DatagramChannel. Everything else looks good. -Alan From chris.hegarty at oracle.com Tue Jun 11 13:19:33 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 11 Jun 2019 14:19:33 +0100 Subject: RFR 8216417: cleanup of IPv6 scope-id handling In-Reply-To: <5CFF8C66.8060309@oracle.com> References: <5CFE66AB.9060105@oracle.com> <97b2327f-9b2d-65c5-7d3b-cddc2be787d4@oracle.com> <5CFF8C66.8060309@oracle.com> Message-ID: <73536DC3-9F89-42F9-B5F7-F9C28713C58D@oracle.com> Michael, > On 11 Jun 2019, at 12:11, Michael McMahon wrote: > > Thanks for the review Alan. > I've made the changes suggested by you and Daniel > and added a test to Scoping.java for checking the connected IP address. > > http://cr.openjdk.java.net/~michaelm/8216417/webrev.3/index.html Looks like AbstractPlainDatagramSocketImpl::connect sets the connected address to the address including the scope. Strangely, it looks like there is a test for this particular scenario, it should catch the problem, no? -Chirs. From michael.x.mcmahon at oracle.com Tue Jun 11 13:55:34 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Tue, 11 Jun 2019 14:55:34 +0100 Subject: RFR 8216417: cleanup of IPv6 scope-id handling In-Reply-To: <73536DC3-9F89-42F9-B5F7-F9C28713C58D@oracle.com> References: <5CFE66AB.9060105@oracle.com> <97b2327f-9b2d-65c5-7d3b-cddc2be787d4@oracle.com> <5CFF8C66.8060309@oracle.com> <73536DC3-9F89-42F9-B5F7-F9C28713C58D@oracle.com> Message-ID: <5CFFB2D6.8080502@oracle.com> Hi Chris DatagramSocket has its own copies of the same fields and it stores the values supplied by the caller (rather than the scopified version), which are returned by getRemoteSocketAddress() and getInetAddress(). Michael. On 11/06/2019, 14:19, Chris Hegarty wrote: > Michael, > >> On 11 Jun 2019, at 12:11, Michael McMahon wrote: >> >> Thanks for the review Alan. >> I've made the changes suggested by you and Daniel >> and added a test to Scoping.java for checking the connected IP address. >> >> http://cr.openjdk.java.net/~michaelm/8216417/webrev.3/index.html > Looks like AbstractPlainDatagramSocketImpl::connect sets the > connected address to the address including the scope. Strangely, > it looks like there is a test for this particular scenario, it should > catch the problem, no? > > -Chirs. > > From chris.hegarty at oracle.com Tue Jun 11 15:52:52 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 11 Jun 2019 16:52:52 +0100 Subject: RFR 8216417: cleanup of IPv6 scope-id handling In-Reply-To: <5CFFB2D6.8080502@oracle.com> References: <5CFE66AB.9060105@oracle.com> <97b2327f-9b2d-65c5-7d3b-cddc2be787d4@oracle.com> <5CFF8C66.8060309@oracle.com> <73536DC3-9F89-42F9-B5F7-F9C28713C58D@oracle.com> <5CFFB2D6.8080502@oracle.com> Message-ID: <18C1091C-F839-4C03-9120-CD33224230DA@oracle.com> Michael, > On 11 Jun 2019, at 14:55, Michael McMahon wrote: > > Hi Chris > > DatagramSocket has its own copies of the same fields and it stores > the values supplied by the caller (rather than the scopified version), > which are returned by getRemoteSocketAddress() and getInetAddress(). > Ah, I see it now. The code is a bit misleading and confusing. Looks like the impl connectedAddress is never used. -Chris. From michael.x.mcmahon at oracle.com Tue Jun 11 17:14:25 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Tue, 11 Jun 2019 18:14:25 +0100 Subject: RFR 8216417: cleanup of IPv6 scope-id handling In-Reply-To: References: <5CFE66AB.9060105@oracle.com> <97b2327f-9b2d-65c5-7d3b-cddc2be787d4@oracle.com> <5CFF8C66.8060309@oracle.com> Message-ID: <5CFFE171.6040606@oracle.com> Hi Alan I've made the suggested changes. I need to confirm with Pavel what the background was for the Linux kernel check. For now, it is still there in the test. Maybe, that could be removed as part of another change later, if that turns out to be possible. Updated webrev: http://cr.openjdk.java.net/~michaelm/8216417/webrev.4/ Thanks, Michael. On 11/06/2019, 13:19, Alan Bateman wrote: > On 11/06/2019 12:11, Michael McMahon wrote: >> Thanks for the review Alan. >> I've made the changes suggested by you and Daniel >> and added a test to Scoping.java for checking the connected IP address. >> >> http://cr.openjdk.java.net/~michaelm/8216417/webrev.3/index.html > One suggestion for the AbstractPlainSocketImpl connect methods is to > set this.address and this.port together and include a comment to say > that the socket address is recorded before attempting to connect. > > A few formatting nits to check: AbstractPlainDatagramSocketImpl L134 > and the spurious blank line at IPAddressUtil L355. > > In the PromiscuousIPv6 test then I'm wondering if the check for the > 3.10.0.* kernel is needed or not. I suspect this may have been in > Pavel's original changes. Maybe Pavel has the history on that? > > I see the Scoping test has been expanded. In datagramTest you can use > try-with-resources. Also shouldn't there be an equivalent for > DatagramChannel. > > Everything else looks good. > > -Alan From brian.burkhalter at oracle.com Tue Jun 11 17:25:24 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 11 Jun 2019 10:25:24 -0700 Subject: 8225461: (fs) Investigate filtering duplicates out of Iterable returned by java.nio.file.FileSystem::getFileStores In-Reply-To: <9d73d427-f83a-cedc-45cb-1c6fa1299306@oracle.com> References: <9d73d427-f83a-cedc-45cb-1c6fa1299306@oracle.com> Message-ID: > On Jun 11, 2019, at 3:46 AM, Alan Bateman wrote: > >> I don?t know whether the specification of FileSystem::getFileStores() should be updated to indicate that no duplicate stores are returned, which would necessitate a CSR. >> > This issue is going a bit of time to review as it changes the meaning of equals/hashCode and has several other implications that will take time to work through. I will try to get to it soon, but maybe not this week. Understood. Note that the new equals() and hashCode() methods in UnixMountEntry are identical to the respective methods of UnixFileStore. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Jun 11 17:46:55 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Jun 2019 18:46:55 +0100 Subject: RFR 8216417: cleanup of IPv6 scope-id handling In-Reply-To: <5CFFE171.6040606@oracle.com> References: <5CFE66AB.9060105@oracle.com> <97b2327f-9b2d-65c5-7d3b-cddc2be787d4@oracle.com> <5CFF8C66.8060309@oracle.com> <5CFFE171.6040606@oracle.com> Message-ID: <666a5c01-249d-bdaa-afd6-6af177891e38@oracle.com> On 11/06/2019 18:14, Michael McMahon wrote: > Hi Alan > > I've made the suggested changes. I need to confirm with Pavel what the > background > was for the Linux kernel check. For now, it is still there in the > test. Maybe, that could > be removed as part of another change later, if that turns out to be > possible. Okay with me.. I tracked down the November thread on net-dev where this came up but I couldn't find anything that fully explained it. > > Updated webrev: http://cr.openjdk.java.net/~michaelm/8216417/webrev.4/ I think this looks good. Just a few minor comments on the Scoping.java test - both DatagramChannel and DatagramSocket implement AutoCloseable so the test can using try-with-resources. - I suspect socketTest leaks a connection as it establishes a connection but doesn't accept/close it. Probably not a concern as the test is /othervm. I don't need another webrev on those points. -Alan From brian.burkhalter at oracle.com Tue Jun 11 17:56:03 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 11 Jun 2019 10:56:03 -0700 Subject: 8223813: (aio) Iocp.getErrorMessage should drop trailing \r\n Message-ID: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8223813 http://cr.openjdk.java.net/~bpb/8223813/webrev.00/ FormatMessage() and FormatMessageW() occur in a number of locations: src/java.base/windows/native/libjli/java_md.c src/java.base/windows/native/libnet/Inet4AddressImpl.c src/java.base/windows/native/libjava/ProcessImpl_md.c src/java.base/windows/native/libjava/jni_util_md.c src/java.base/windows/native/libnio/ch/Iocp.c src/java.base/windows/native/libnio/fs/WindowsNativeDispatcher.c src/java.base/share/native/libzip/zlib/gzlib.c Some of these already strip the terminal CRLF (or dot + CRLF) of the string populated by FormatMessage[W](). This patch would add removing them, if present, from src/java.base/windows/native/libnet/Inet4AddressImpl.c src/java.base/windows/native/libnio/ch/Iocp.c src/java.base/windows/native/libnio/fs/WindowsNativeDispatcher.c One question is whether it would be better just to consolidate this code into two methods for example in jni_uitl and call the methods from the other locations. There are already getLastErrorString() and getErrorString() for chars here. Also, I am not sure how to test this effectively. The code passes all tiers as-is. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Roger.Riggs at oracle.com Tue Jun 11 19:14:06 2019 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 11 Jun 2019 15:14:06 -0400 Subject: 8223813: (aio) Iocp.getErrorMessage should drop trailing \r\n In-Reply-To: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> References: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> Message-ID: <7d31f4e2-256f-ec74-bc86-b9aa94251cb6@oracle.com> Hi Brian, Having the extraneous suffix consistently removed seems like a good thing. Though I'm not sure what the best form of the utility function is: ?1) return the count of characters to remove ?2) just truncate the buffer. There may be ways to force some of the errors that would lead to the truncation, but testing all uses will not be practical. $.02, Roger On 06/11/2019 01:56 PM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8223813 > http://cr.openjdk.java.net/~bpb/8223813/webrev.00/ > > > FormatMessage() and FormatMessageW() occur in a number of locations: > > src/java.base/windows/native/libjli/java_md.c > src/java.base/windows/native/libnet/Inet4AddressImpl.c > src/java.base/windows/native/libjava/ProcessImpl_md.c > src/java.base/windows/native/libjava/jni_util_md.c > src/java.base/windows/native/libnio/ch/Iocp.c > src/java.base/windows/native/libnio/fs/WindowsNativeDispatcher.c > src/java.base/share/native/libzip/zlib/gzlib.c > > Some of these already strip the terminal CRLF (or dot + CRLF) of the > string populated by FormatMessage[W](). This patch would add removing > them, if present, from > > src/java.base/windows/native/libnet/Inet4AddressImpl.c > src/java.base/windows/native/libnio/ch/Iocp.c > src/java.base/windows/native/libnio/fs/WindowsNativeDispatcher.c > > One question is whether it would be better just to consolidate this > code into two methods for example in jni_uitl and call the methods > from the other locations. There are already getLastErrorString() and > getErrorString() for chars here. > > Also, I am not sure how to test this effectively. The code passes all > tiers as-is. > > Thanks, > > Brian From brian.burkhalter at oracle.com Tue Jun 11 19:56:06 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 11 Jun 2019 12:56:06 -0700 Subject: 8223813: (aio) Iocp.getErrorMessage should drop trailing \r\n In-Reply-To: <7d31f4e2-256f-ec74-bc86-b9aa94251cb6@oracle.com> References: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> <7d31f4e2-256f-ec74-bc86-b9aa94251cb6@oracle.com> Message-ID: Hi Roger, Thanks for the $0.02. > On Jun 11, 2019, at 12:14 PM, Roger Riggs wrote: > > Having the extraneous suffix consistently removed seems like a good thing. > Though I'm not sure what the best form of the utility function is: > 1) return the count of characters to remove > 2) just truncate the buffer. I am not sure either nor am I convinced that it would even be worth the effort. As mentioned there are already getLastErrorString() and getErrorString() for chars so I suppose the equivalent for WCHARs would need to be added for at least one of these. > There may be ways to force some of the errors that would lead to the truncation, but testing all uses will not be practical. That?s what I thought. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Roger.Riggs at oracle.com Tue Jun 11 19:58:51 2019 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 11 Jun 2019 15:58:51 -0400 Subject: 8223813: (aio) Iocp.getErrorMessage should drop trailing \r\n In-Reply-To: References: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> <7d31f4e2-256f-ec74-bc86-b9aa94251cb6@oracle.com> Message-ID: <0e2c566b-ec25-f9ba-096b-93b3a33a630b@oracle.com> Hi Brian, I'm fine with the current webrev too.? it does the job and is specific. I agree about it not likely to be worth the effort. Thanks, Roger On 06/11/2019 03:56 PM, Brian Burkhalter wrote: > Hi Roger, > > Thanks for the $0.02. > >> On Jun 11, 2019, at 12:14 PM, Roger Riggs > > wrote: >> >> Having the extraneous suffix consistently removed seems like a good >> thing. >> Though I'm not sure what the best form of the utility function is: >> ?1) return the count of characters to remove >> ?2) just truncate the buffer. > > I am not sure either nor am I convinced that it would even be worth > the effort. As mentioned there are already getLastErrorString() and > getErrorString() for chars so I suppose the equivalent for WCHARs > would need to be added for at least one of these. > >> There may be ways to force some of the errors that would lead to the >> truncation, but testing all uses will not be practical. > > That?s what I thought. > > Thanks, > > Brian From brian.burkhalter at oracle.com Tue Jun 11 20:01:30 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 11 Jun 2019 13:01:30 -0700 Subject: 8223813: (aio) Iocp.getErrorMessage should drop trailing \r\n In-Reply-To: <0e2c566b-ec25-f9ba-096b-93b3a33a630b@oracle.com> References: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> <7d31f4e2-256f-ec74-bc86-b9aa94251cb6@oracle.com> <0e2c566b-ec25-f9ba-096b-93b3a33a630b@oracle.com> Message-ID: <9F83CF6E-0859-47AD-9A5E-1C4CF7566E43@oracle.com> Hi Roger, OK, good. I?ll let it hang out there for another day and proceed with it as-is if there are no objections. Thanks, Brian > On Jun 11, 2019, at 12:58 PM, Roger Riggs wrote: > > I'm fine with the current webrev too. it does the job and is specific. > > I agree about it not likely to be worth the effort. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan.gerasimov at oracle.com Wed Jun 12 00:06:58 2019 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 11 Jun 2019 17:06:58 -0700 Subject: 8223813: (aio) Iocp.getErrorMessage should drop trailing \r\n In-Reply-To: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> References: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> Message-ID: Hi Brian! Inet4AddressImpl.c: 1) There is an extra space after FormatMessage, 2) It is preexisting. The code doesn't check if FormatMessage failed to allocate the buffer. It's not clear from the MSDN documentation, if the pointer to the buffer will be set to NULL upon the failure. If it does not, then subsequent NET_ThrowNew(env, err, buf); LocalFree(buf); may hit uninitialized memory. It would be more accurate to invoke them only if (n > 0). 3) It purely optional, but you may want to use the TEXT macro to append the L prefix to the character literals, if TCHAR is defined to be WCHAR: 389 if (buf[n - 1] == TEXT('\n')) n--; 390 if (buf[n - 1] == TEXT('\r')) n--; 391 if (buf[n - 1] == TEXT('.')) n--; 392 buf[n] = TEXT('\0'); It may make the compiler just a tiny bit happier :) Everything else looks good to me. With kind regards, Ivan On 6/11/19 10:56 AM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8223813 > http://cr.openjdk.java.net/~bpb/8223813/webrev.00/ > > > FormatMessage() and FormatMessageW() occur in a number of locations: > > src/java.base/windows/native/libjli/java_md.c > src/java.base/windows/native/libnet/Inet4AddressImpl.c > src/java.base/windows/native/libjava/ProcessImpl_md.c > src/java.base/windows/native/libjava/jni_util_md.c > src/java.base/windows/native/libnio/ch/Iocp.c > src/java.base/windows/native/libnio/fs/WindowsNativeDispatcher.c > src/java.base/share/native/libzip/zlib/gzlib.c > > Some of these already strip the terminal CRLF (or dot + CRLF) of the > string populated by FormatMessage[W](). This patch would add removing > them, if present, from > > src/java.base/windows/native/libnet/Inet4AddressImpl.c > src/java.base/windows/native/libnio/ch/Iocp.c > src/java.base/windows/native/libnio/fs/WindowsNativeDispatcher.c > > One question is whether it would be better just to consolidate this > code into two methods for example in jni_uitl and call the methods > from the other locations. There are already getLastErrorString() and > getErrorString() for chars here. > > Also, I am not sure how to test this effectively. The code passes all > tiers as-is. > > Thanks, > > Brian -- With kind regards, Ivan Gerasimov -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jun 12 00:26:52 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 11 Jun 2019 17:26:52 -0700 Subject: 8223813: (aio) Iocp.getErrorMessage should drop trailing \r\n In-Reply-To: References: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> Message-ID: Hi Ivan, I updated the patch: http://cr.openjdk.java.net/~bpb/8223813/webrev.01/ Please see comments inline below. > On Jun 11, 2019, at 5:06 PM, Ivan Gerasimov wrote: > > Inet4AddressImpl.c: > > 1) There is an extra space after FormatMessage, Fixed. > 2) It is preexisting. The code doesn't check if FormatMessage failed to allocate the buffer. > It's not clear from the MSDN documentation, if the pointer to the buffer will be set to NULL upon the failure. > If it does not, then subsequent NET_ThrowNew(env, err, buf); LocalFree(buf); may hit uninitialized memory. > It would be more accurate to invoke them only if (n > 0). I put ?if (buf != NULL)? instead of ?if (n > 0)?. > 3) It purely optional, but you may want to use the TEXT macro to append the L prefix to the character literals, if TCHAR is defined to be WCHAR: > > 389 if (buf[n - 1] == TEXT('\n')) n--; > 390 if (buf[n - 1] == TEXT('\r')) n--; > 391 if (buf[n - 1] == TEXT('.')) n--; > 392 buf[n] = TEXT('\0'); > > It may make the compiler just a tiny bit happier :) So changed. > Everything else looks good to me. Thanks for the comments! Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan.gerasimov at oracle.com Wed Jun 12 02:11:52 2019 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 11 Jun 2019 19:11:52 -0700 Subject: 8223813: (aio) Iocp.getErrorMessage should drop trailing \r\n In-Reply-To: References: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> Message-ID: <40d7cdad-ae61-3b36-df46-ec904488c045@oracle.com> Thanks Brian! The webrev looks fine to me. I think that *most likely* the check if (buf != NULL) will work as expected: buf will only be set to non-NULL value upon success. However, the documentation for the function FormatMessage states: """ If the function fails, the return value is zero. """ So, my preference would be if (n > 0). With kind regards, Ivan On 6/11/19 5:26 PM, Brian Burkhalter wrote: > Hi Ivan, > > I updated the patch: > http://cr.openjdk.java.net/~bpb/8223813/webrev.01/ > > > Please see comments inline below. > >> On Jun 11, 2019, at 5:06 PM, Ivan Gerasimov >> > wrote: >> >> Inet4AddressImpl.c: >> >> 1) There is an extra space after FormatMessage, > > Fixed. > >> 2) It is preexisting. The code doesn't check if FormatMessage failed >> to allocate the buffer. >> It's not clear from the MSDN documentation, if the pointer to the >> buffer will be set to NULL upon the failure. >> If it does not, then subsequent NET_ThrowNew(env, err, buf); >> LocalFree(buf); may hit uninitialized memory. >> It would be more accurate to invoke them only if (n > 0). > > I put ?if (buf != NULL)? instead of ?if (n > 0)?. > >> 3) It purely optional, but you may want to use the TEXT macro to >> append the L prefix to the character literals, if TCHAR is defined to >> be WCHAR: >> >> 389 if (buf[n - 1] == TEXT('\n')) n--; >> 390 if (buf[n - 1] == TEXT('\r')) n--; >> 391 if (buf[n - 1] == TEXT('.')) n--; >> 392 buf[n] = TEXT('\0'); >> >> It may make the compiler just a tiny bit happier :) > > So changed. > >> Everything else looks good to me. > > Thanks for the comments! > > Brian -- With kind regards, Ivan Gerasimov -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Wed Jun 12 10:31:55 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 12 Jun 2019 11:31:55 +0100 Subject: RFR 8216417: cleanup of IPv6 scope-id handling In-Reply-To: <666a5c01-249d-bdaa-afd6-6af177891e38@oracle.com> References: <5CFE66AB.9060105@oracle.com> <97b2327f-9b2d-65c5-7d3b-cddc2be787d4@oracle.com> <5CFF8C66.8060309@oracle.com> <5CFFE171.6040606@oracle.com> <666a5c01-249d-bdaa-afd6-6af177891e38@oracle.com> Message-ID: <12d4e816-4944-a2c9-dbbb-d06d5761783f@oracle.com> Michael, > On 11/06/2019 18:14, Michael McMahon wrote: >> ... >> Updated webrev: http://cr.openjdk.java.net/~michaelm/8216417/webrev.4/ This looks very good. -Chris. From brian.burkhalter at oracle.com Wed Jun 12 16:51:19 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 12 Jun 2019 09:51:19 -0700 Subject: 8223813: (aio) Iocp.getErrorMessage should drop trailing \r\n In-Reply-To: <40d7cdad-ae61-3b36-df46-ec904488c045@oracle.com> References: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> <40d7cdad-ae61-3b36-df46-ec904488c045@oracle.com> Message-ID: <3592CD7E-511E-4795-AE7C-4A165B6D01FE@oracle.com> Hi Ivan, I am perhaps beating a dead horse here, but how about this instead? if (n > 0) { NET_ThrowNew(env, err, buf); LocalFree(buf); } else { NET_ThrowNew(env, err, "FormatMessage failed"); } After all, an error *did* occur. Thanks, Brian > On Jun 11, 2019, at 7:11 PM, Ivan Gerasimov wrote: > > The webrev looks fine to me. > > I think that *most likely* the check if (buf != NULL) will work as expected: buf will only be set to non-NULL value upon success. > > However, the documentation for the function FormatMessage states: > """ > If the function fails, the return value is zero. > """ > > So, my preference would be if (n > 0). -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jun 12 17:02:49 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 12 Jun 2019 10:02:49 -0700 Subject: 8223813: (aio) Iocp.getErrorMessage should drop trailing \r\n In-Reply-To: <3592CD7E-511E-4795-AE7C-4A165B6D01FE@oracle.com> References: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> <40d7cdad-ae61-3b36-df46-ec904488c045@oracle.com> <3592CD7E-511E-4795-AE7C-4A165B6D01FE@oracle.com> Message-ID: <69085D8F-42C3-434A-BEB9-D2E694038CE7@oracle.com> Actually, never mind, I am being completely lame here: both NET_ThrowNew() and the Windows function LocalFree() are robust to a NULL-valued buf so I think we can just remove the n > 0 or buf == NULL check altogether. Sorry for the noise: I should have checked this first. Thanks, Brian > On Jun 12, 2019, at 9:51 AM, Brian Burkhalter wrote: > > I am perhaps beating a dead horse here, but how about this instead? > > if (n > 0) { > NET_ThrowNew(env, err, buf); > LocalFree(buf); > } else { > NET_ThrowNew(env, err, "FormatMessage failed"); > } > > After all, an error *did* occur. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan.gerasimov at oracle.com Wed Jun 12 23:03:40 2019 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 12 Jun 2019 16:03:40 -0700 Subject: 8223813: (aio) Iocp.getErrorMessage should drop trailing \r\n In-Reply-To: <69085D8F-42C3-434A-BEB9-D2E694038CE7@oracle.com> References: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> <40d7cdad-ae61-3b36-df46-ec904488c045@oracle.com> <3592CD7E-511E-4795-AE7C-4A165B6D01FE@oracle.com> <69085D8F-42C3-434A-BEB9-D2E694038CE7@oracle.com> Message-ID: <534ebf7b-ff79-9109-9b14-4e061cbf3511@oracle.com> On 6/12/19 10:02 AM, Brian Burkhalter wrote: > Actually, never mind, I am being completely lame here: both > NET_ThrowNew() and the Windows function LocalFree() are robust to a > NULL-valued buf so I think we can just remove the n > 0 or buf == NULL > check altogether. > That's true, assuming that you initialize buf = NULL and hopping that FormatMessage won't change buf upon failure. I wish MSDN were a little bit more specific here. I am fine with 1) 362 TCHAR *buf = NULL; 2) unconditional 395 NET_ThrowNew(env, err, buf); 396 LocalFree(buf); With kind regards, Ivan > Sorry for the noise: I should have checked this first. > > Thanks, > > Brian > >> On Jun 12, 2019, at 9:51 AM, Brian Burkhalter >> > wrote: >> >> I am perhaps beating a dead horse here, but how about this instead? >> >> if (n > 0) { >> NET_ThrowNew(env, err, buf); >> LocalFree(buf); >> } else { >> NET_ThrowNew(env, err, "FormatMessage failed"); >> } >> >> After all, an error *did* occur. > -- With kind regards, Ivan Gerasimov -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jun 12 23:26:52 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 12 Jun 2019 16:26:52 -0700 Subject: 8223813: (aio) Iocp.getErrorMessage should drop trailing \r\n In-Reply-To: <534ebf7b-ff79-9109-9b14-4e061cbf3511@oracle.com> References: <50CF47F6-78A6-48E2-8247-12C18A20C330@oracle.com> <40d7cdad-ae61-3b36-df46-ec904488c045@oracle.com> <3592CD7E-511E-4795-AE7C-4A165B6D01FE@oracle.com> <69085D8F-42C3-434A-BEB9-D2E694038CE7@oracle.com> <534ebf7b-ff79-9109-9b14-4e061cbf3511@oracle.com> Message-ID: <17847CCA-56C7-4494-9F78-C8B1138D3A11@oracle.com> Hi Ivan, > On Jun 12, 2019, at 4:03 PM, Ivan Gerasimov wrote: > > On 6/12/19 10:02 AM, Brian Burkhalter wrote: >> Actually, never mind, I am being completely lame here: both NET_ThrowNew() and the Windows function LocalFree() are robust to a NULL-valued buf so I think we can just remove the n > 0 or buf == NULL check altogether. >> > That's true, assuming that you initialize buf = NULL and hopping that FormatMessage won't change buf upon failure. True. > I wish MSDN were a little bit more specific here. Likewise. > I am fine with > > 1) > 362 TCHAR *buf = NULL; > > 2) > unconditional > 395 NET_ThrowNew(env, err, buf); > 396 LocalFree(buf); Barring objections I will check it in with the above code after the JDK 13 fork. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Jun 13 17:42:03 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 13 Jun 2019 10:42:03 -0700 Subject: =?utf-8?Q?=5B14=5D_8223151=3A_Document_whether_FileSystemProvider?= =?utf-8?Q?=2EnewFileSystem=E2=80=8B=28Path=2C_Map=29_always_returns_new_F?= =?utf-8?Q?ileSystem_objects_?= Message-ID: <8B61B4F9-8195-463F-8202-BD03E83ADF19@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8223151 Please see the diff below. A CSR would be in order of course. It appears that within the JDK the ZipFileSystemProvider is the only FileSystemProvider for which this method does not throw UOE. Thanks, Brian --- a/src/java.base/share/classes/java/nio/file/spi/FileSystemProvider.java +++ b/src/java.base/share/classes/java/nio/file/spi/FileSystemProvider.java @@ -268,7 +268,7 @@ * *

    This method returns a reference to a {@code FileSystem} that was * created by invoking the {@link #newFileSystem(URI,Map) newFileSystem(URI,Map)} - * method. File systems created the {@link #newFileSystem(Path,Map) + * method. File systems created by the {@link #newFileSystem(Path,Map) * newFileSystem(Path,Map)} method are not returned by this method. * The file system is identified by its {@code URI}. Its exact form * is highly provider dependent. In the case of the default provider the URI's @@ -351,6 +351,14 @@ * it throws {@code UnsupportedOperationException}. The default implementation * of this method throws {@code UnsupportedOperationException}. * + *

    Unlike {@link #newFileSystem(URI,Map)}, this method does not + * throw a {@link FileSystemAlreadyExistsException} if it is invoked + * with the same {@code path} used previously to create a file system. + * It is provider-dependent whether this method returns different + * {@link FileSystem} instances for two invocations with the same path. If + * different {@link FileSystem} instances are returned, it is likewise + * provider-dependent whether such instances can safely be used at once. + * * @param path * The path to the file * @param env -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Jun 13 17:51:57 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 13 Jun 2019 10:51:57 -0700 Subject: [14] RFC: 8219644: java/nio/file/Files/CopyAndMove.java: Test threw exception: java.lang.RuntimeException: AtomicMoveNotSupportedException expected Message-ID: https://bugs.openjdk.java.net/browse/JDK-8219644 This particular problem was due to two Windows paths beginning with ?c:? and ?C:? respectively being considered as different FileStores. This is because equals() depends on String.equals() applied to the ?root? instance variable in WindowsFileStore. Could this perhaps be changed to equalsIgnoreCase() (and hashCode() to use toLowerCase())? Is ?root? always a drive letter or do other mount types come into play? Or maybe the test could be changed to be robust to case? Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Jun 13 18:03:29 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 13 Jun 2019 19:03:29 +0100 Subject: [14] RFC: 8219644: java/nio/file/Files/CopyAndMove.java: Test threw exception: java.lang.RuntimeException: AtomicMoveNotSupportedException expected In-Reply-To: References: Message-ID: <11d21385-6d2c-bb7a-bb4e-2bc61d4e793f@oracle.com> On 13/06/2019 18:51, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8219644 > > This particular problem was due to two Windows paths beginning with > ?c:? and ?C:? respectively being considered as different FileStores. > This is because equals() depends on String.equals() applied to the > ?root? instance variable in WindowsFileStore. Could this perhaps be > changed to equalsIgnoreCase() (and hashCode() to use toLowerCase())? > Is ?root? always a drive letter or do other mount types come into play? It's also used to create a FileStore for files on UNCs so care is needed. > Or maybe the test could be changed to be robust to case? > That would be fine but worth investigating if WindowsFileStore needs changes too. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Jun 13 18:09:47 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 13 Jun 2019 11:09:47 -0700 Subject: [14] RFC: 8219644: java/nio/file/Files/CopyAndMove.java: Test threw exception: java.lang.RuntimeException: AtomicMoveNotSupportedException expected In-Reply-To: <11d21385-6d2c-bb7a-bb4e-2bc61d4e793f@oracle.com> References: <11d21385-6d2c-bb7a-bb4e-2bc61d4e793f@oracle.com> Message-ID: <9336C83A-57B3-4902-8591-F50016A889F5@oracle.com> > On Jun 13, 2019, at 11:03 AM, Alan Bateman wrote: > > On 13/06/2019 18:51, Brian Burkhalter wrote: >> https://bugs.openjdk.java.net/browse/JDK-8219644 >> >> This particular problem was due to two Windows paths beginning with ?c:? and ?C:? respectively being considered as different FileStores. This is because equals() depends on String.equals() applied to the ?root? instance variable in WindowsFileStore. Could this perhaps be changed to equalsIgnoreCase() (and hashCode() to use toLowerCase())? Is ?root? always a drive letter or do other mount types come into play? > It's also used to create a FileStore for files on UNCs so care is needed. I guess that could be detected by the first character being a ?/? or by using Character.isLetter(). >> Or maybe the test could be changed to be robust to case? >> > That would be fine but worth investigating if WindowsFileStore needs changes too. OK. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Jun 13 18:12:14 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 13 Jun 2019 19:12:14 +0100 Subject: =?UTF-8?Q?Re=3a_=5b14=5d_8223151=3a_Document_whether_FileSystemProv?= =?UTF-8?Q?ider=2enewFileSystem=e2=80=8b=28Path=2c_Map=29_always_returns_new?= =?UTF-8?Q?_FileSystem_objects?= In-Reply-To: <8B61B4F9-8195-463F-8202-BD03E83ADF19@oracle.com> References: <8B61B4F9-8195-463F-8202-BD03E83ADF19@oracle.com> Message-ID: On 13/06/2019 18:42, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8223151 > > Please see the diff below. A CSR would be in order of course. > Lance - is this related to an issue with zipfs that you are looking into? -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Thu Jun 13 18:15:04 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 13 Jun 2019 14:15:04 -0400 Subject: =?utf-8?Q?Re=3A_=5B14=5D_8223151=3A_Document_whether_FileSystemPr?= =?utf-8?Q?ovider=2EnewFileSystem=E2=80=8B=28Path=2C_Map=29_always_returns?= =?utf-8?Q?_new_FileSystem_objects?= In-Reply-To: References: <8B61B4F9-8195-463F-8202-BD03E83ADF19@oracle.com> Message-ID: <19382213-226C-4741-89FE-4FC2BB51F35D@oracle.com> No, it is slightly different. > On Jun 13, 2019, at 2:12 PM, Alan Bateman wrote: > > On 13/06/2019 18:42, Brian Burkhalter wrote: >> https://bugs.openjdk.java.net/browse/JDK-8223151 >> >> Please see the diff below. A CSR would be in order of course. >> > Lance - is this related to an issue with zipfs that you are looking into? > > -Alan Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From christoph.langer at sap.com Fri Jun 14 18:18:38 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 14 Jun 2019 18:18:38 +0000 Subject: RFR 8213031: (zipfs) Add support for POSIX file permissions In-Reply-To: References: <953449f0913340f6a94ae3d83611fd92@sap.com> <9b3c9cfe-63e9-94ea-1af0-7ba9e2db52fd@oracle.com> <62a26037-a991-dc31-a972-a82386f63b92@oracle.com> <9806f4b1-9b55-d1d8-4511-5db0ef6786a5@oracle.com> <33E52896-E491-4660-A932-46D5A270A63E@oracle.com> <458d4ce8-d669-3117-908d-687dae2ad8d2@oracle.com> Message-ID: Alan, Lance, the CSR was approved yesterday. So, is the webrev (below) also ok and reviewed from your end? Thanks Christoph From: Langer, Christoph Sent: Mittwoch, 5. Juni 2019 16:45 To: Alan Bateman ; Lance Andersen Cc: nio-dev ; Java Core Libs Subject: RE: RFR 8213031: (zipfs) Add support for POSIX file permissions Hi Lance, Alan, et al., as it seems we are getting close to CSR approval, the changeset should also be finally reviewed. Here is the most current version: http://cr.openjdk.java.net/~clanger/webrevs/8213031.14/ I have made some updates to ZipConstants as suggested by Lance in a private mail. Best regards Christoph -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Sat Jun 15 13:53:00 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Sat, 15 Jun 2019 09:53:00 -0400 Subject: RFR 8213031: (zipfs) Add support for POSIX file permissions In-Reply-To: References: <953449f0913340f6a94ae3d83611fd92@sap.com> <9b3c9cfe-63e9-94ea-1af0-7ba9e2db52fd@oracle.com> <62a26037-a991-dc31-a972-a82386f63b92@oracle.com> <9806f4b1-9b55-d1d8-4511-5db0ef6786a5@oracle.com> <33E52896-E491-4660-A932-46D5A270A63E@oracle.com> <458d4ce8-d669-3117-908d-687dae2ad8d2@oracle.com> Message-ID: Hi Christoph, I have not had a chance to completely go through the latest webrev. I hope to get back to it Monday > On Jun 14, 2019, at 2:18 PM, Langer, Christoph wrote: > > Alan, Lance, > > the CSR was approved yesterday. > > So, is the webrev (below) also ok and reviewed from your end? > > Thanks > Christoph > > From: Langer, Christoph > Sent: Mittwoch, 5. Juni 2019 16:45 > To: Alan Bateman ; Lance Andersen > Cc: nio-dev ; Java Core Libs > Subject: RE: RFR 8213031: (zipfs) Add support for POSIX file permissions > > Hi Lance, Alan, et al., > > as it seems we are getting close to CSR approval, the changeset should also be finally reviewed. > > Here is the most current version: http://cr.openjdk.java.net/~clanger/webrevs/8213031.14/ > > I have made some updates to ZipConstants as suggested by Lance in a private mail. > > Best regards > Christoph > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From Alan.Bateman at oracle.com Sun Jun 16 06:57:43 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 16 Jun 2019 07:57:43 +0100 Subject: =?UTF-8?Q?Re=3a_=5b14=5d_8223151=3a_Document_whether_FileSystemProv?= =?UTF-8?Q?ider=2enewFileSystem=e2=80=8b=28Path=2c_Map=29_always_returns_new?= =?UTF-8?Q?_FileSystem_objects?= In-Reply-To: <19382213-226C-4741-89FE-4FC2BB51F35D@oracle.com> References: <8B61B4F9-8195-463F-8202-BD03E83ADF19@oracle.com> <19382213-226C-4741-89FE-4FC2BB51F35D@oracle.com> Message-ID: <5539a39f-4ba7-224a-65a8-5a0a9c012cf9@oracle.com> On 13/06/2019 19:15, Lance Andersen wrote: > No, it is slightly different. The underlying bug here is that the zip file system provider's newFileSystem(Path, Map) is throwing FileSystemAlreadyExistsException whereas it is supposed to create a new file system each time. There are questions in the issue for the scenario where the same file is open several times to create different file systems which leads to several provider specific questions on whether the zip file is locked or whether there is any other coordination between the different instances. There is scope for additional javadoc, maybe in the FileSystems factory class or in FileSystemProvider to outline how providers that identify a file system via a URI should work. This will lead to discussion on Path.of(URI) and the observation that it is file system provider specific as to whether it creates a file system or returns a Path to a file in an existing file system. The zip file system provider was intended to support the latter approach for cases where the zip file system is created with a URI. -Alan From Alan.Bateman at oracle.com Sun Jun 16 08:47:51 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 16 Jun 2019 09:47:51 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <68566847-de80-82af-a928-d30c75f2b1b2@redhat.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <06ef3b25-9e34-f6f3-e4c4-319adea52ae7@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> <7b39f68d-5698-4079-425c-c86ec161e361@oracle.com> <72810832-eb88-3418-f336-b95af95d9dcc@redhat.com> <20c8bbcb-2cd3-77af-cd40-29f7bd752166@oracle.com> <68566847-de80-82af-a928-d30c75f2b1b2@redhat.com> Message-ID: On 10/06/2019 11:09, Andrew Dinn wrote: > : > I have updated the Proposed Java API Changes to remove the changes to > map exception signature and force region specification. They were > covered in the prior enabling JIRAs/CSRs. > > So, the remaining two sections mention 1) the new module + package + > enum and 2) the bean counting changes. The new section 1 clarifies when > UnsupportedOperationException is thrown vs when IOException is thrown as > part of the explanation of what the new modes are for. > > I also added a paragraph to the alternatives section explaining that > Panama was and still is an alternative option and why we decided to > proceed with this model despite it being still under consideration. > I re-read the JEP, trying to put myself in the position of someone reading it for the first time in 2020. Summary section: What would you think about replacing this with a sentence that makes it clear that the JEP adds new JDK-specific file mapping modes to allow the FileChannel API create MappedByteBuffers over non-volatile memory? Goals section: I think the first paragraph could be re-worded to make it clear that the goal is to use the existing MappedByteBuffer API to access NVM. Paragraphs 2-5 split this into two sub-goals. The first suggests that it extends the MBB API but this is no longer the case. The second also hints that it adds a new API. I think these two need to be re-worded. Goal 3 and 4 are okay, although I think the 4th could be summarized as allowing mapped buffers on NVM to be tracked by the existing monitoring and management APIs. Description section It might be clearer of "Proposed Java API Changes" were renamed to "Proposed JDK-specific API changes". One other thing to mention is that I re-read the javadoc for the MBB force methods and I think we need to adjust one of the sentences in the existing (and new) methods to take account of implementation specific map modes. I've created JDK-8226203 [1] to track it. As support for implementation specific map modes is in new in Java SE 13 then it might be worth trying to get that fixed now while it is fresh in our minds. -Alan [1] https://bugs.openjdk.java.net/browse/JDK-8226203 From adinn at redhat.com Tue Jun 18 11:43:26 2019 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 18 Jun 2019 12:43:26 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <2153cb8e-b4c1-d3c5-4c28-3e32d50db2ea@oracle.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> <7b39f68d-5698-4079-425c-c86ec161e361@oracle.com> <72810832-eb88-3418-f336-b95af95d9dcc@redhat.com> <20c8bbcb-2cd3-77af-cd40-29f7bd752166@oracle.com> <68566847-de80-82af-a928-d30c75f2b1b2@redhat.com> Message-ID: <892f3b0b-55f1-872d-15f4-5af907fa8437@redhat.com> Hi Alan, Thanks for reviewing the JEP one more time. On 16/06/2019 09:47, Alan Bateman wrote: > I re-read the JEP, trying to put myself in the position of someone > reading it for the first time in 2020. > > Summary section: > > What would you think about replacing this with a sentence that makes it > clear that the JEP adds new JDK-specific file mapping modes to allow the > FileChannel API create MappedByteBuffers over non-volatile memory? I would be happy with that way of describing the behaviour change except that what you suggest doesn't mention actually making it work -- which /is/ part of the JEP. How about: "Add new JDK-specific file mapping modes, allowing the FileChannel API to be used to create MappedByteBuffers over non-volatile memory, and extend the underlying implementation to support this new use case" > Goals section: > > I think the first paragraph could be re-worded to make it clear that the > goal is to use the existing MappedByteBuffer API to access NVM. Yes indeed: "This JEP proposes that class MappedByteBuffer be reworked to support access to non-volatile memory (NVM). The only API change required is a new enumeration employed by FileChannel clients to request mapping of a file located on an NVM-backed file system rather than a conventional, file storage system. Recent changes to the MappedByteBufer API mean that it supports all the behaviours needed to allow direct memory updates and provide the durability guarantees needed for higher level, Java client libraries to implement persistent data types (e.g. block file systems, journaled logs, persistent objects, etc.). The implementations of FileChannel and MappedByteBuffer need revising to be aware of this new backing type for the mapped file." > Paragraphs 2-5 split this into two sub-goals. The first suggests that it > extends the MBB API but this is no longer the case. The second also > hints that it adds a new API. I think these two need to be re-worded. Agreed. How about this: "The primary goal of this JEP is to ensure that clients can access and update NVM from a Java program efficiently and coherently. A key element of this goal is to ensure that individual writes (or small groups of contiguous writes) to a buffer region can be committed with minimal overhead i.e. to ensure that any changes which might still be in cache are written back to memory." n.b. I snuck in the word coherence because I thought it clarified the notion of committing to memory. > Goal 3 and 4 are okay, although I think the 4th could be summarized as > allowing mapped buffers on NVM to be tracked by the existing monitoring > and management APIs. Agreed. So, renumbering accordingly, how about this: "A second, subordinate goal is to implement this commit behaviour using a restricted, JDK-internal API defined in class Unsafe, allowing it to be re-used by classes other than MappedByteBuffer that may need to commit NVM. A final, related goal is to allow buffers mapped over NVM to be tracked by the existing monitoring and management APIs." > Description section > > It might be clearer of "Proposed Java API Changes" were renamed to > "Proposed JDK-specific API changes". Agreed. > One other thing to mention is that I re-read the javadoc for the MBB > force methods and I think we need to adjust one of the sentences in the > existing (and new) methods to take account of implementation specific > map modes. I've created JDK-8226203 [1] to track it. As support for > implementation specific map modes is in new in Java SE 13 then it might > be worth trying to get that fixed now while it is fresh in our minds. Sure, I'll post an RFR with the javadoc patch as a separate step. Can it go into jdk13? or is it too late for that? regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Tue Jun 18 16:15:04 2019 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 18 Jun 2019 17:15:04 +0100 Subject: RFR (T): 8226203: MappedByteBuffer.force method may have no effect on implementation specific map modes Message-ID: <77e9929b-4c35-ce0d-5159-21f9e6a12c56@redhat.com> Could I please have a review for the following trivial change to the javadoc for the two MappedByteBuffer.force methods: JIRA: https://bugs.openjdk.java.net/browse/JDK-8226203 webrev: http://cr.openjdk.java.net/~adinn/8226203/webrev.00/ It allows for the possibility of failures when extended map modes are used to create the MappedByteBuffer. The case documented here will not actually arise in JDK13 since nothing in the JDK creates, let alone exposes extended map modes. However, it would be better if the javadoc reflected that possibility. Is it still possible to push this? If that is a problem then I am happy for it to go into jdk14. regards, Andrew Dinn ----------- From Alan.Bateman at oracle.com Tue Jun 18 17:08:28 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 18 Jun 2019 18:08:28 +0100 Subject: RFR (T): 8226203: MappedByteBuffer.force method may have no effect on implementation specific map modes In-Reply-To: <77e9929b-4c35-ce0d-5159-21f9e6a12c56@redhat.com> References: <77e9929b-4c35-ce0d-5159-21f9e6a12c56@redhat.com> Message-ID: <83f9d285-a885-817a-fd33-a19bad982a4b@oracle.com> On 18/06/2019 17:15, Andrew Dinn wrote: > Could I please have a review for the following trivial change to the > javadoc for the two MappedByteBuffer.force methods: > > JIRA: https://bugs.openjdk.java.net/browse/JDK-8226203 > webrev: http://cr.openjdk.java.net/~adinn/8226203/webrev.00/ > > It allows for the possibility of failures when extended map modes are > used to create the MappedByteBuffer. > > The case documented here will not actually arise in JDK13 since nothing > in the JDK creates, let alone exposes extended map modes. However, it > would be better if the javadoc reflected that possibility. Is it still > possible to push this? If that is a problem then I am happy for it to go > into jdk14. > This looks good. Will you create a CSR for this? I think it can be fixed in jdk/jdk13 as it follows JDK-8221397 and JDK-8221696 (and there is no risk as it's javadoc only). -Alan From adinn at redhat.com Wed Jun 19 09:21:53 2019 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 19 Jun 2019 10:21:53 +0100 Subject: RFR (T): 8226203: MappedByteBuffer.force method may have no effect on implementation specific map modes In-Reply-To: <83f9d285-a885-817a-fd33-a19bad982a4b@oracle.com> References: <77e9929b-4c35-ce0d-5159-21f9e6a12c56@redhat.com> <83f9d285-a885-817a-fd33-a19bad982a4b@oracle.com> Message-ID: Hi Alan, On 18/06/2019 18:08, Alan Bateman wrote: > This looks good. Will you create a CSR for this? I think it can be fixed > in jdk/jdk13 as it follows JDK-8221397 and JDK-8221696 (and there is no > risk as it's javadoc only). I raised this CSR: CSR: https://bugs.openjdk.java.net/browse/JDK-8226385 and tagged it for jdk13. Also, I labelled it SE -- but is it, perhaps, meant to be JDK? (apologies, I am still a noob to this process). Bug: https://bugs.openjdk.java.net/browse/JDK-8226203 Webrev: http://cr.openjdk.java.net/~adinn/8226203/webrev.00/ regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Wed Jun 19 10:03:35 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Jun 2019 11:03:35 +0100 Subject: RFR (T): 8226203: MappedByteBuffer.force method may have no effect on implementation specific map modes In-Reply-To: References: <77e9929b-4c35-ce0d-5159-21f9e6a12c56@redhat.com> <83f9d285-a885-817a-fd33-a19bad982a4b@oracle.com> Message-ID: <3b2f162d-9f84-77cc-8d95-b4bd82af5bcd@oracle.com> On 19/06/2019 10:21, Andrew Dinn wrote: > : > I raised this CSR: > > CSR: https://bugs.openjdk.java.net/browse/JDK-8226385 > > and tagged it for jdk13. > > Also, I labelled it SE -- but is it, perhaps, meant to be JDK? > (apologies, I am still a noob to this process). Yes, it's "SE" as it's normative text in Java SE API spec. I added myself as Reviewer to the the CSR so you can finalize. The webrev looks good. -Alan From adinn at redhat.com Wed Jun 19 10:07:29 2019 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 19 Jun 2019 11:07:29 +0100 Subject: RFR (T): 8226203: MappedByteBuffer.force method may have no effect on implementation specific map modes In-Reply-To: <3b2f162d-9f84-77cc-8d95-b4bd82af5bcd@oracle.com> References: <77e9929b-4c35-ce0d-5159-21f9e6a12c56@redhat.com> <83f9d285-a885-817a-fd33-a19bad982a4b@oracle.com> <3b2f162d-9f84-77cc-8d95-b4bd82af5bcd@oracle.com> Message-ID: <075492eb-5cb8-8b99-57ff-13fb17894024@redhat.com> On 19/06/2019 11:03, Alan Bateman wrote: > I added myself as Reviewer to the the CSR so you can finalize. The > webrev looks good. Thanks, Alan. I have finalized the CSR. Do I still need to wait for confirmation for the CSR from Joe Darcy before pushing to the jdk13 repo? (He already knows about the CSR). regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Wed Jun 19 13:11:28 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Jun 2019 14:11:28 +0100 Subject: RFR (T): 8226203: MappedByteBuffer.force method may have no effect on implementation specific map modes In-Reply-To: <075492eb-5cb8-8b99-57ff-13fb17894024@redhat.com> References: <77e9929b-4c35-ce0d-5159-21f9e6a12c56@redhat.com> <83f9d285-a885-817a-fd33-a19bad982a4b@oracle.com> <3b2f162d-9f84-77cc-8d95-b4bd82af5bcd@oracle.com> <075492eb-5cb8-8b99-57ff-13fb17894024@redhat.com> Message-ID: <0fc47d28-a390-57dd-3d02-f10172798df3@oracle.com> On 19/06/2019 11:07, Andrew Dinn wrote: > : > Do I still need to wait for confirmation for the CSR from Joe Darcy > before pushing to the jdk13 repo? (He already knows about the CSR). > Yes, anything that has a CSR needs to wait until it is approved. -Alan From adinn at redhat.com Fri Jun 21 09:42:42 2019 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 21 Jun 2019 10:42:42 +0100 Subject: RFR (T): 8226203: MappedByteBuffer.force method may have no effect on implementation specific map modes In-Reply-To: <0fc47d28-a390-57dd-3d02-f10172798df3@oracle.com> References: <77e9929b-4c35-ce0d-5159-21f9e6a12c56@redhat.com> <83f9d285-a885-817a-fd33-a19bad982a4b@oracle.com> <3b2f162d-9f84-77cc-8d95-b4bd82af5bcd@oracle.com> <075492eb-5cb8-8b99-57ff-13fb17894024@redhat.com> <0fc47d28-a390-57dd-3d02-f10172798df3@oracle.com> Message-ID: <908a4100-1068-95de-047f-a5a3f917f396@redhat.com> Hi Alan, On 19/06/2019 14:11, Alan Bateman wrote: > On 19/06/2019 11:07, Andrew Dinn wrote: >> : >> Do I still need to wait for confirmation for the CSR from Joe Darcy >> before pushing to the jdk13 repo? (He already knows about the CSR). >> > Yes, anything that has a CSR needs to wait until it is approved. Joe Darcy has approved the CSR. However, he suggested by way of code review that the comment be tweaked to explicitly state that the force method does have an effect when the buffer was mapped with mode READ_WRITE e.g. instead of *

    This method has no effect for buffers mapped in read-only * or private mapping modes. It may also have no effect for other * implementation specific map modes.

    this + *

    If this buffer was not mapped in read/write mode + * ({@link java.nio.channels.FileChannel.MapMode#READ_WRITE}) + * then invoking this method may have no effect. In particular, + * the method has no effect for buffers mapped in read-only or + * private mapping modes. This method may or may not have an + * effect for implementation-specific mapping modes.

    Would you agree with that change or do you prefer to stick with the original? If necessary I'll amend the patch and CSR then push whichever version you prefer to JDK13. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Fri Jun 21 11:34:01 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 21 Jun 2019 12:34:01 +0100 Subject: RFR (T): 8226203: MappedByteBuffer.force method may have no effect on implementation specific map modes In-Reply-To: <908a4100-1068-95de-047f-a5a3f917f396@redhat.com> References: <77e9929b-4c35-ce0d-5159-21f9e6a12c56@redhat.com> <83f9d285-a885-817a-fd33-a19bad982a4b@oracle.com> <3b2f162d-9f84-77cc-8d95-b4bd82af5bcd@oracle.com> <075492eb-5cb8-8b99-57ff-13fb17894024@redhat.com> <0fc47d28-a390-57dd-3d02-f10172798df3@oracle.com> <908a4100-1068-95de-047f-a5a3f917f396@redhat.com> Message-ID: On 21/06/2019 10:42, Andrew Dinn wrote: > : > + *

    If this buffer was not mapped in read/write mode > + * ({@link java.nio.channels.FileChannel.MapMode#READ_WRITE}) > + * then invoking this method may have no effect. In particular, > + * the method has no effect for buffers mapped in read-only or > + * private mapping modes. This method may or may not have an > + * effect for implementation-specific mapping modes.

    > > Would you agree with that change or do you prefer to stick with the > original? If necessary I'll amend the patch and CSR then push whichever > version you prefer to JDK13. > I saw Joe's comment on the CSR. This refinement looks good to me. -Alan From GROEGES at uk.ibm.com Fri Jun 21 13:32:42 2019 From: GROEGES at uk.ibm.com (Steve Groeger) Date: Fri, 21 Jun 2019 14:32:42 +0100 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD15E56C9@ORSMSX113.amr.corp.intel.com> References: <5C96DD75-48C1-474F-9A91-5671256353F1@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15B5E36@ORSMSX113.amr.corp.intel.com> <3B5379DE-5D47-4B50-8DFC-6B4CEF93A977@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15B7562@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15BB6D5@ORSMSX113.amr.corp.intel.com> <6C2EEE6F-B9D4-478D-8CE7-E471773943F1@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15C0F33@ORSMSX113.amr.corp.intel.com> <59ecdb38-8d20-3416-602c-47198fac6e41@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15C11 <9ACD5B67AAC5594CB6268234CF29CF9AD15E56C9@ORSMSX113.amr.corp.intel.com> Message-ID: Lucy, Thanks for all the hard work getting the rsockets integrated. I have meant to take a look at this for a while but only just got around to doing so. I have tried running some of the rsocket tests using jtreg but have had some issues which I am hoping you can assist with. How should thee rsocket tests be executed? I run some of the jtreg tests directly using jtreg, using a command like: /home/test/jtreg/bin/jtreg -vf -jdk:/home/test/jdk java/lang/System/PropertyTest.java these run without any issues, however if I do the same with the rsocket tests, ie. /home/test/jtreg/bin/jtreg -vf -jdk:/home/test/jdk jdk/net/RdmaSockets/rsocket/RsocketTest.java it Thanks Steve Groeger IBM Runtime Technologies Hursley, Winchester Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129 Fax (44) 1962 816800 Lotus Notes: Steve Groeger/UK/IBM Internet: groeges at uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: From GROEGES at uk.ibm.com Fri Jun 21 13:32:42 2019 From: GROEGES at uk.ibm.com (Steve Groeger) Date: Fri, 21 Jun 2019 14:32:42 +0100 Subject: adding rsockets support into JDK In-Reply-To: References: <9ACD5B67AAC5594CB6268234CF29CF9AD15B5E36@ORSMSX113.amr.corp.intel.com> <3B5379DE-5D47-4B50-8DFC-6B4CEF93A977@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15B7562@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15BB6D5@ORSMSX113.amr.corp.intel.com> <6C2EEE6F-B9D4-478D-8CE7-E471773943F1@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15C0F33@ORSMSX113.amr.corp.intel.com> <59ecdb38-8d20-3416-602c-47198fac6e41@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15C11 <9ACD5B67AAC5594CB6268234CF29CF9AD15E56C9@ORSMSX113.amr.corp.intel.com> Message-ID: Lucy, Sorry sent previous post prematurely :-( Thanks for all the hard work getting the rsockets integrated. I have meant to take a look at this for a while but only just got around to doing so. I have tried running some of the rsocket tests using jtreg but have had some issues which I am hoping you can assist with. How should these rsocket tests be executed? I run some of the jtreg tests directly using jtreg, using a command like: /home/test/jtreg/bin/jtreg -vf -jdk:/home/test/jdk java/lang/System/PropertyTest.java these run without any issues, however if I do the same with the rsocket tests, ie. /home/test/jtreg/bin/jtreg -vf -jdk:/home/test/jdk jdk/net/RdmaSockets/rsocket/RsocketTest.java it fails with an error: TEST RESULT: Error. Use -nativepath to specify the location of native code I have looked and cant see how to specify the -nativepath as part of the jtreg command. Any help on how to run these tests would be great. Thanks Steve Groeger IBM Runtime Technologies Hursley, Winchester Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129 Fax (44) 1962 816800 Lotus Notes: Steve Groeger/UK/IBM Internet: groeges at uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: From GROEGES at uk.ibm.com Fri Jun 21 14:14:58 2019 From: GROEGES at uk.ibm.com (Steve Groeger) Date: Fri, 21 Jun 2019 15:14:58 +0100 Subject: adding rsockets support into JDK In-Reply-To: References: <3B5379DE-5D47-4B50-8DFC-6B4CEF93A977@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15B7562@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15BB6D5@ORSMSX113.amr.corp.intel.com> <6C2EEE6F-B9D4-478D-8CE7-E471773943F1@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15C0F33@ORSMSX113.amr.corp.intel.com> <59ecdb38-8d20-3416-602c-47198fac6e41@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15C11 <9ACD5B67AAC5594CB6268234CF29CF9AD15E56C9@ORSMSX113.amr.corp.intel.com> Message-ID: Hi all, Please ignore the previous question as I found out how to specify -nativepath on jtreg. I was suppling the value incorrectly -nativepath=/path/to/natives rather than -nativepath:/path/to/natives Thanks Steve Groeger IBM Runtime Technologies Hursley, Winchester Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129 Fax (44) 1962 816800 Lotus Notes: Steve Groeger/UK/IBM Internet: groeges at uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU "nio-dev" wrote on 21/06/2019 14:32:42: > From: Steve Groeger > To: "Lu, Yingqi" > Cc: nio-dev > Date: 21/06/2019 14:34 > Subject: RE: adding rsockets support into JDK > Sent by: "nio-dev" > > Lucy, > > Sorry sent previous post prematurely :-( > > Thanks for all the hard work getting the rsockets integrated. I have > meant to take a look at this for a while but only just got around todoing so. > I have tried running some of the rsocket tests using jtreg but have > had some issues which I am hoping you can assist with. > > How should these rsocket tests be executed? > > I run some of the jtreg tests directly using jtreg, using a command like: > /home/test/jtreg/bin/jtreg -vf -jdk:/home/test/jdk java/lang/ > System/PropertyTest.java > these run without any issues, however if I do the same with the > rsocket tests, ie. > /home/test/jtreg/bin/jtreg -vf -jdk:/home/test/jdk jdk/net/ > RdmaSockets/rsocket/RsocketTest.java > it fails with an error: > TEST RESULT: Error. Use -nativepath to specify the location > of native code > > I have looked and cant see how to specify the -nativepath as part of > the jtreg command. > Any help on how to run these tests would be great. > > Thanks > Steve Groeger > IBM Runtime Technologies > Hursley, Winchester > Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129 > Fax (44) 1962 816800 > Lotus Notes: Steve Groeger/UK/IBM > Internet: groeges at uk.ibm.com > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with > number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with > number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: From adinn at redhat.com Fri Jun 21 14:27:22 2019 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 21 Jun 2019 15:27:22 +0100 Subject: RFR (T): 8226203: MappedByteBuffer.force method may have no effect on implementation specific map modes In-Reply-To: References: <77e9929b-4c35-ce0d-5159-21f9e6a12c56@redhat.com> <83f9d285-a885-817a-fd33-a19bad982a4b@oracle.com> <3b2f162d-9f84-77cc-8d95-b4bd82af5bcd@oracle.com> <075492eb-5cb8-8b99-57ff-13fb17894024@redhat.com> <0fc47d28-a390-57dd-3d02-f10172798df3@oracle.com> <908a4100-1068-95de-047f-a5a3f917f396@redhat.com> Message-ID: <3ebf73a9-1f54-7c48-0597-56440092f3a8@redhat.com> Hi Alan, On 21/06/2019 12:34, Alan Bateman wrote: > I saw Joe's comment on the CSR. This refinement looks good to me. I have pushed the doc fix to jdk13. Once it percolates to jdk14 I will post a new webrev for the JEP 352 implementation which accomodates this change. That webrev will also include the changes pending after Joe's review of the implementation CSR leaving (I hope) only my recently proposed JEP updates and endorsement to finish before it can be pushed. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Mon Jun 24 11:35:34 2019 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 24 Jun 2019 12:35:34 +0100 Subject: RFR: 8224974: Implement JEP 352 In-Reply-To: <892f3b0b-55f1-872d-15f4-5af907fa8437@redhat.com> References: <80da32b2-7acb-7b94-b82c-5dcd5cf95539@redhat.com> <656f462e-c655-0c48-7c90-190c92e0bc28@redhat.com> <0a41e686-0931-5054-b314-2c015fbae62a@redhat.com> <36326F5E-12CD-487A-8FE2-1049631FE908@oracle.com> <8B1A8EDC-F9D7-4085-A34F-69100DBD7D5C@oracle.com> <5c78b5d7-dc9e-fbbd-41a9-5139ed6ee32c@redhat.com> <86d000f3-9eca-6089-5f7e-5698444d99ce@oracle.com> <7fe8d17d-6170-fcc9-b87a-eccfda2bb546@redhat.com> <7b39f68d-5698-4079-425c-c86ec161e361@oracle.com> <72810832-eb88-3418-f336-b95af95d9dcc@redhat.com> <20c8bbcb-2cd3-77af-cd40-29f7bd752166@oracle.com> <68566847-de80-82af-a928-d30c75f2b1b2@redhat.com> <892f3b0b-55f1-872d-15f4-5af907fa8437@redhat.com> Message-ID: <8f72d3b5-3a01-a7de-3b09-35571266a87d@redhat.com> Hi Alan, I have uploaded a webrev which I believe answers all outstanding review comments: JIRA: https://bugs.openjdk.java.net/browse/JDK-8224974 webrev: http://cr.openjdk.java.net/~adinn/8224974/webrev.08/ This includes the changes Joe Darcy highlighted in his code review of the implementation CSR (module header comment plus @since 14 changes in Unsafe.java). It also allows for the tweaks to the force API documentation (JDK-8226203). So, I think the only work still outstanding is to agree changes to the JEP wording (as proposed in my previous post -- see below) plus endorsement and targeting of the JEP. regards, Andrew Dinn ----------- On 18/06/2019 12:43, Andrew Dinn wrote: > Hi Alan, > > Thanks for reviewing the JEP one more time. > > On 16/06/2019 09:47, Alan Bateman wrote: >> I re-read the JEP, trying to put myself in the position of someone >> reading it for the first time in 2020. >> >> Summary section: >> >> What would you think about replacing this with a sentence that makes it >> clear that the JEP adds new JDK-specific file mapping modes to allow the >> FileChannel API create MappedByteBuffers over non-volatile memory? > > I would be happy with that way of describing the behaviour change except > that what you suggest doesn't mention actually making it work -- which > /is/ part of the JEP. How about: > > "Add new JDK-specific file mapping modes, allowing the FileChannel API > to be used to create MappedByteBuffers over non-volatile memory, and > extend the underlying implementation to support this new use case" > >> Goals section: >> >> I think the first paragraph could be re-worded to make it clear that the >> goal is to use the existing MappedByteBuffer API to access NVM. > > Yes indeed: > > "This JEP proposes that class MappedByteBuffer be reworked to support > access to non-volatile memory (NVM). The only API change required is a > new enumeration employed by FileChannel clients to request mapping of a > file located on an NVM-backed file system rather than a conventional, > file storage system. Recent changes to the MappedByteBufer API mean that > it supports all the behaviours needed to allow direct memory updates and > provide the durability guarantees needed for higher level, Java client > libraries to implement persistent data types (e.g. block file systems, > journaled logs, persistent objects, etc.). The implementations of > FileChannel and MappedByteBuffer need revising to be aware of this new > backing type for the mapped file." > >> Paragraphs 2-5 split this into two sub-goals. The first suggests that it >> extends the MBB API but this is no longer the case. The second also >> hints that it adds a new API. I think these two need to be re-worded. > > Agreed. How about this: > > "The primary goal of this JEP is to ensure that clients can access and > update NVM from a Java program efficiently and coherently. A key element > of this goal is to ensure that individual writes (or small groups of > contiguous writes) to a buffer region can be committed with minimal > overhead i.e. to ensure that any changes which might still be in cache > are written back to memory." > > n.b. I snuck in the word coherence because I thought it clarified the > notion of committing to memory. > >> Goal 3 and 4 are okay, although I think the 4th could be summarized as >> allowing mapped buffers on NVM to be tracked by the existing monitoring >> and management APIs. > > Agreed. So, renumbering accordingly, how about this: > > "A second, subordinate goal is to implement this commit behaviour using > a restricted, JDK-internal API defined in class Unsafe, allowing it to > be re-used by classes other than MappedByteBuffer that may need to > commit NVM. > > A final, related goal is to allow buffers mapped over NVM to be tracked > by the existing monitoring and management APIs." > >> Description section >> >> It might be clearer of "Proposed Java API Changes" were renamed to >> "Proposed JDK-specific API changes". > > Agreed. > >> One other thing to mention is that I re-read the javadoc for the MBB >> force methods and I think we need to adjust one of the sentences in the >> existing (and new) methods to take account of implementation specific >> map modes. I've created JDK-8226203 [1] to track it. As support for >> implementation specific map modes is in new in Java SE 13 then it might >> be worth trying to get that fixed now while it is fresh in our minds. > Sure, I'll post an RFR with the javadoc patch as a separate step. Can it > go into jdk13? or is it too late for that? > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From brian.burkhalter at oracle.com Mon Jun 24 23:03:38 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 24 Jun 2019 16:03:38 -0700 Subject: 8226706: (se) Reduce the number of outer loop iterations on Windows in java/nio/channels/Selector/RacyDeregister.java Message-ID: <95FBB4D3-E437-4DF7-B49E-28C67B5907FB@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8226706 The number of outer loop iterations of this test was increased, for Windows only, by the patch that resolved [1]. A comment was also added to the test reminding to reset the number of iterations back down to its previous smaller value once [2] was resolved. Subsequently [2] was resolved as irreproducible but the number of iterations in the test was not in fact reduced. As the higher number of iterations might be the reason for the failure in [3], it is proposed here to change the number of outer loop resolutions from 150 on Windows back to 15, the same as on the other platforms. Thanks, Brian --- a/test/jdk/java/nio/channels/Selector/RacyDeregister.java +++ b/test/jdk/java/nio/channels/Selector/RacyDeregister.java @@ -41,11 +41,6 @@ */ public class RacyDeregister { - // FIXME: NUM_OUTER_LOOP_ITERATIONS should be reverted to the hard-coded - // value 15 when JDK-8161083 is resolved as either a bug or a non-issue. - static final int NUM_OUTER_LOOP_ITERATIONS = - System.getProperty("os.name").startsWith("Windows") ? 150 : 15; - // 90% of 1200 second timeout as milliseconds static final int TIMEOUT_THRESHOLD_MILLIS = 1200*900; @@ -90,7 +85,7 @@ public void run() { try { - for (int k = 0; k < NUM_OUTER_LOOP_ITERATIONS; k++) { + for (int k = 0; k < 15; k++) { System.out.format("outer loop %3d at %7d ms%n", k, System.currentTimeMillis() - t0); -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Jun 25 17:57:02 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 25 Jun 2019 10:57:02 -0700 Subject: 8226706: (se) Reduce the number of outer loop iterations on Windows in java/nio/channels/Selector/RacyDeregister.java In-Reply-To: <95FBB4D3-E437-4DF7-B49E-28C67B5907FB@oracle.com> References: <95FBB4D3-E437-4DF7-B49E-28C67B5907FB@oracle.com> Message-ID: > On Jun 24, 2019, at 4:03 PM, Brian Burkhalter wrote: > > https://bugs.openjdk.java.net/browse/JDK-8226706 > > The number of outer loop iterations of this test was increased, for Windows only, by the patch that resolved [1]. A comment was also added to the test reminding to reset the number of iterations back down to its previous smaller value once [2] was resolved. Subsequently [2] was resolved as irreproducible but the number of iterations in the test was not in fact reduced. As the higher number of iterations might be the reason for the failure in [3], it is proposed here to change the number of outer loop resolutions from 150 on Windows back to 15, the same as on the other platforms. These references were inadvertently left out of my RFR: [1] https://bugs.openjdk.java.net/browse/JDK-8163305 [2] https://bugs.openjdk.java.net/browse/JDK-8161083 [3] https://bugs.openjdk.java.net/browse/JDK-8201765 > > Thanks, > > Brian > > --- a/test/jdk/java/nio/channels/Selector/RacyDeregister.java > +++ b/test/jdk/java/nio/channels/Selector/RacyDeregister.java > @@ -41,11 +41,6 @@ > */ > public class RacyDeregister { > > - // FIXME: NUM_OUTER_LOOP_ITERATIONS should be reverted to the hard-coded > - // value 15 when JDK-8161083 is resolved as either a bug or a non-issue. > - static final int NUM_OUTER_LOOP_ITERATIONS = > - System.getProperty("os.name").startsWith("Windows") ? 150 : 15; > - > // 90% of 1200 second timeout as milliseconds > static final int TIMEOUT_THRESHOLD_MILLIS = 1200*900; > > @@ -90,7 +85,7 @@ > > public void run() { > try { > - for (int k = 0; k < NUM_OUTER_LOOP_ITERATIONS; k++) { > + for (int k = 0; k < 15; k++) { > System.out.format("outer loop %3d at %7d ms%n", k, > System.currentTimeMillis() - t0); -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Tue Jun 25 18:22:56 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 25 Jun 2019 14:22:56 -0400 Subject: 8226706: (se) Reduce the number of outer loop iterations on Windows in java/nio/channels/Selector/RacyDeregister.java In-Reply-To: References: <95FBB4D3-E437-4DF7-B49E-28C67B5907FB@oracle.com> Message-ID: <0B6E8F83-BD3D-4F0C-8EFB-23EB033CE6D0@oracle.com> Go for it :-) > On Jun 25, 2019, at 1:57 PM, Brian Burkhalter wrote: > > >> On Jun 24, 2019, at 4:03 PM, Brian Burkhalter > wrote: >> >> https://bugs.openjdk.java.net/browse/JDK-8226706 >> >> The number of outer loop iterations of this test was increased, for Windows only, by the patch that resolved [1]. A comment was also added to the test reminding to reset the number of iterations back down to its previous smaller value once [2] was resolved. Subsequently [2] was resolved as irreproducible but the number of iterations in the test was not in fact reduced. As the higher number of iterations might be the reason for the failure in [3], it is proposed here to change the number of outer loop resolutions from 150 on Windows back to 15, the same as on the other platforms. > > These references were inadvertently left out of my RFR: > > [1] https://bugs.openjdk.java.net/browse/JDK-8163305 > [2] https://bugs.openjdk.java.net/browse/JDK-8161083 > [3] https://bugs.openjdk.java.net/browse/JDK-8201765 > >> >> Thanks, >> >> Brian >> >> --- a/test/jdk/java/nio/channels/Selector/RacyDeregister.java >> +++ b/test/jdk/java/nio/channels/Selector/RacyDeregister.java >> @@ -41,11 +41,6 @@ >> */ >> public class RacyDeregister { >> >> - // FIXME: NUM_OUTER_LOOP_ITERATIONS should be reverted to the hard-coded >> - // value 15 when JDK-8161083 is resolved as either a bug or a non-issue. >> - static final int NUM_OUTER_LOOP_ITERATIONS = >> - System.getProperty("os.name").startsWith("Windows") ? 150 : 15; >> - >> // 90% of 1200 second timeout as milliseconds >> static final int TIMEOUT_THRESHOLD_MILLIS = 1200*900; >> >> @@ -90,7 +85,7 @@ >> >> public void run() { >> try { >> - for (int k = 0; k < NUM_OUTER_LOOP_ITERATIONS; k++) { >> + for (int k = 0; k < 15; k++) { >> System.out.format("outer loop %3d at %7d ms%n", k, >> System.currentTimeMillis() - t0); > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From Alan.Bateman at oracle.com Thu Jun 27 11:22:56 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 27 Jun 2019 12:22:56 +0100 Subject: 8226706: (se) Reduce the number of outer loop iterations on Windows in java/nio/channels/Selector/RacyDeregister.java In-Reply-To: <95FBB4D3-E437-4DF7-B49E-28C67B5907FB@oracle.com> References: <95FBB4D3-E437-4DF7-B49E-28C67B5907FB@oracle.com> Message-ID: <29a09af7-237e-0364-4c18-3b6e5d1ee5d0@oracle.com> On 25/06/2019 00:03, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8226706 > > The number of outer loop iterations of this test was increased, for > Windows only, by the patch that resolved [1]. A comment was also added > to the test reminding to reset the number of iterations back down to > its previous smaller value once [2] was resolved. Subsequently [2] was > resolved as irreproducible but the number of iterations in the test > was not in fact reduced. As the higher number of iterations might be > the reason for the failure in [3], it is proposed here to change the > number of outer loop resolutions from 150 on Windows back to 15, the > same as on the other platforms. > Looks okay although at some point I think this test should be re-implemented as has the potential to be starved on Windows and timeout. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Jun 27 19:01:05 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 27 Jun 2019 12:01:05 -0700 Subject: Redux 8184157: (ch) AsynchronousFileChannel hangs with internal error when reading locked file Message-ID: <8B15EB6A-2E66-42D9-8277-C25FB9279837@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8184157 http://cr.openjdk.java.net/~bpb/8184157/webrev.01/ Reprising a thread begun in July 2017 with a significantly simpler patch but equally complicated explanation. :-) WindowsAsynchronousFileChannelImpl uses a PendingIoCache instance which internally maintains a map of longs (pointers to OVERLAPPED structures) to PendingFutures. When a read, write, or lock task is run, an entry for the task is added to this map as there is a different OVERLAPPED structure for each task. Because the channel (file) is associated with an I/O completion port, the OVERLAPPED pointer will eventually be retrieved by the Iocp long-running event task whether or not the task completes immediately or is pending. When the Iocp event task retrieves the pointer from GetQueuedCompletionStatus(), it uses it to obtain the PendingFuture from the map and if appropriate invokes one of the PendingFuture's methods. PendingIoCache tries to minimize memory allocations of the space required for OVERLAPPED structures by maintaining a cache. Addresses are placed in the cache when a map entry is removed from the PendingIoCache. When a new entry is added to the PendingIoCache, this cache is checked before allocating new memory for the OVERLAPPED pointer and if a previously allocated address is available then it is reused. The problem is that the OVERLAPPED pointer may be recycled *before* it is retrieved from the I/O completion port by the long-running Iocp event task. In this case the PendingFuture that the Iocp task retrieves from the PendingIoCache will not be the one which should be associated with the completion packet. For the failure at hand, a lock task is executed and its OVERLAPPED pointer is reused for a read task before the completion packet of the lock task is obtained by the completion port. When the completion packet of the lock task is received, the PendingFuture of the read task is therefore obtained from the map. The ?bytes transferred? value of the completion packet of the lock task is garbage but is passed to the completed() method of the read task?s PendingFuture resulting in an error. The proposed fix is *not* to remove the entry of a task from the PendingIoCache until after its pointer to OVERLAPPED key is received by the completion port. This prevents the pointer from being reused prematurely by the caching mechanism in PendingIoCache. The Iocp task itself removes the entry corresponding to the pointer so at that point it may be safely cached for reuse in the PendingIoCache. A simpler fix would be to remove the OVERLAPPED pointer caching in PendingIoCache. Without the source patch applied, the test crashes on my local machine, but succeeds with the source patch applied. The crash has yet to be reproduced in the CI system however. Otherwise, the patch passes tiers 1-3. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From uschindler at apache.org Fri Jun 28 10:32:52 2019 From: uschindler at apache.org (Uwe Schindler) Date: Fri, 28 Jun 2019 12:32:52 +0200 Subject: Any way to access fstat.st_blocks from Java (UnixFileAttributesView) Message-ID: <0afa01d52d9c$d95879e0$8c096da0$@apache.org> Hi, i browsed through the JDK source code to find a way if the allocated block size of a file can be found using any FileAttributeView. There is the hidden unix one, which allows access to most of the "st_xxxx" constants like st_size, but the st_blocks and st_blksize ones are missing. Is this intended? The use case for this is to find the actual size allocation of a file on disk (e.g., if it's sparse or for SAMFS if its currently only on band archive (in this case the block size is 0). Any hints to get the file size on disk from Java that uses the st_blocks and st_blksize behind the scenes? Thanks, Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany https://lucene.apache.org/ From Alan.Bateman at oracle.com Fri Jun 28 14:18:06 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 28 Jun 2019 15:18:06 +0100 Subject: Any way to access fstat.st_blocks from Java (UnixFileAttributesView) In-Reply-To: <0afa01d52d9c$d95879e0$8c096da0$@apache.org> References: <0afa01d52d9c$d95879e0$8c096da0$@apache.org> Message-ID: <9bc3c13e-a700-7908-4590-9e79c60fa0e2@oracle.com> On 28/06/2019 11:32, Uwe Schindler wrote: > Hi, > > i browsed through the JDK source code to find a way if the allocated block size of a file can be found using any FileAttributeView. There is the hidden unix one, which allows access to most of the "st_xxxx" constants like st_size, but the st_blocks and st_blksize ones are missing. Is this intended? > > The use case for this is to find the actual size allocation of a file on disk (e.g., if it's sparse or for SAMFS if its currently only on band archive (in this case the block size is 0). > > Any hints to get the file size on disk from Java that uses the st_blocks and st_blksize behind the scenes? > There isn't anything at this time. You are right that the "unix" view could expose a "blocks" attribute. The block size is already exposed by FileStore::getBlockSize - the motivation for that was sizing and aligning buffers for use with direct I/O. -Alan From brian.burkhalter at oracle.com Fri Jun 28 15:44:51 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 28 Jun 2019 08:44:51 -0700 Subject: Any way to access fstat.st_blocks from Java (UnixFileAttributesView) In-Reply-To: <9bc3c13e-a700-7908-4590-9e79c60fa0e2@oracle.com> References: <0afa01d52d9c$d95879e0$8c096da0$@apache.org> <9bc3c13e-a700-7908-4590-9e79c60fa0e2@oracle.com> Message-ID: > On Jun 28, 2019, at 7:18 AM, Alan Bateman wrote: > >> Any hints to get the file size on disk from Java that uses the st_blocks and st_blksize behind the scenes? >> > There isn't anything at this time. You are right that the "unix" view could expose a "blocks" attribute. The block size is already exposed by FileStore::getBlockSize - the motivation for that was sizing and aligning buffers for use with direct I/O. Maybe an enhancement should be filed? Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Jun 28 21:46:20 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 28 Jun 2019 14:46:20 -0700 Subject: 8224480: (fc) java/nio/channels/FileChannel/MapTest.java fails intermittently Message-ID: https://bugs.openjdk.java.net/browse/JDK-8224480 The failures reported in this issue were all in the recently added MapTest.testForce() method. This method currently performs 12,800 invocations of MappedByteBuffer.force(int,int). On Unix, force() calls msync(2) to flush the in-memory data to the file system. The machines on which the failures occurred were determined to be two 2012 vintage Mac Pros and a Mac Mini of similar vintage, all running HDDs, 5400 RPM on the Mini, 7200 RPM on the Pros. The hypothesis is that these configurations could be too slow for this sub-test so this patch proposes to halve the number of iterations in testForce() to account for this. An alternative would be to change testForce() to be a separate test. Thanks, Brian --- a/test/jdk/java/nio/channels/FileChannel/MapTest.java +++ b/test/jdk/java/nio/channels/FileChannel/MapTest.java @@ -195,7 +195,7 @@ * the data exercising various valid and invalid writeback ranges. */ private static void testForce() throws Exception { - for (int x=0; x<100; x++) { + for (int x=0; x<50; x++) { try (RandomAccessFile raf = new RandomAccessFile(blah, "rw")) { FileChannel fc = raf.getChannel(); final int BLOCK_SIZE = 64; -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Jun 28 21:55:30 2019 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 28 Jun 2019 14:55:30 -0700 Subject: 8224480: (fc) java/nio/channels/FileChannel/MapTest.java fails intermittently In-Reply-To: References: Message-ID: <6802ECDF-347F-456C-A34A-6D45D91019B2@oracle.com> Hi Lance, That?s probably a good idea to use the separate test approach as an alternate route if the failure still crops up. 100 test repeats with this version had no failures but that proves nothing. Thanks, Brian > On Jun 28, 2019, at 2:53 PM, Lance Andersen wrote: > > Seems reasonable attempt and I guess if all else fails go the separate test route? > >> On Jun 28, 2019, at 5:46 PM, Brian Burkhalter > wrote: >> >> https://bugs.openjdk.java.net/browse/JDK-8224480 -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Fri Jun 28 21:53:36 2019 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 28 Jun 2019 17:53:36 -0400 Subject: 8224480: (fc) java/nio/channels/FileChannel/MapTest.java fails intermittently In-Reply-To: References: Message-ID: Hi Brian, Seems reasonable attempt and I guess if all else fails go the separate test route? > On Jun 28, 2019, at 5:46 PM, Brian Burkhalter wrote: > > https://bugs.openjdk.java.net/browse/JDK-8224480 > > The failures reported in this issue were all in the recently added MapTest.testForce() method. This method currently performs 12,800 invocations of MappedByteBuffer.force(int,int). On Unix, force() calls msync(2) to flush the in-memory data to the file system. The machines on which the failures occurred were determined to be two 2012 vintage Mac Pros and a Mac Mini of similar vintage, all running HDDs, 5400 RPM on the Mini, 7200 RPM on the Pros. The hypothesis is that these configurations could be too slow for this sub-test so this patch proposes to halve the number of iterations in testForce() to account for this. An alternative would be to change testForce() to be a separate test. > > Thanks, > > Brian > > --- a/test/jdk/java/nio/channels/FileChannel/MapTest.java > +++ b/test/jdk/java/nio/channels/FileChannel/MapTest.java > @@ -195,7 +195,7 @@ > * the data exercising various valid and invalid writeback ranges. > */ > private static void testForce() throws Exception { > - for (int x=0; x<100; x++) { > + for (int x=0; x<50; x++) { > try (RandomAccessFile raf = new RandomAccessFile(blah, "rw")) { > FileChannel fc = raf.getChannel(); > final int BLOCK_SIZE = 64; > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From Alan.Bateman at oracle.com Sun Jun 30 08:37:10 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 30 Jun 2019 09:37:10 +0100 Subject: Redux 8184157: (ch) AsynchronousFileChannel hangs with internal error when reading locked file In-Reply-To: <8B15EB6A-2E66-42D9-8277-C25FB9279837@oracle.com> References: <8B15EB6A-2E66-42D9-8277-C25FB9279837@oracle.com> Message-ID: On 27/06/2019 20:01, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8184157 > http://cr.openjdk.java.net/~bpb/8184157/webrev.01/ > > I think the analysis is good and the approach to consistently leave the release of the OVERLAPPED structure to when the completion event is posted is good. One thing that isn't immediately clear from the changes is whether it introduces a memory leak when there is an error initiating the I/O operation or where there is an async close at around the time that the I/O operation is initiated. I would need to study these cases further to see if there are issues. The Windows implementations of AsynchronousSocketChannel and AsynchronousServerSocketChannel also remove and free the OVERLAPPED structure then the socket operation completes immediately. Will they also need attention? The test will need cleanup, maybe rename it to LockReadWriteStressTest or something that is closer to what it actually does. It can use try-with-resources to avoid leaking resources when it fails. I assume "f" and "l" will be renamed as they are unusual variables names in this context. Also the buffer usages is unusual and I can assume can be cleaned up too. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Sun Jun 30 08:42:24 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 30 Jun 2019 09:42:24 +0100 Subject: 8224480: (fc) java/nio/channels/FileChannel/MapTest.java fails intermittently In-Reply-To: References: Message-ID: <48c36180-3b3d-bc21-6886-04794bb53a02@oracle.com> On 28/06/2019 22:46, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8224480 > > The failures reported in this issue were all in the recently added > MapTest.testForce() method. This method currently performs 12,800 > invocations of MappedByteBuffer.force(int,int). On Unix, force() calls > msync(2) to flush the in-memory data to the file system. The machines > on which the failures occurred were determined to be two 2012 vintage > Mac Pros and a Mac Mini of similar vintage, all running HDDs, 5400 RPM > on the Mini, 7200 RPM?on?the Pros. The hypothesis is that these > configurations could be too slow for this sub-test so this patch > proposes to halve the number of iterations in testForce() to account > for this. An alternative would be to change testForce() to be a > separate test. This looks okay but I'm curious what the run-times are like on these systems (I'm just wondering if it might also be useful to add a /timeout option to the @run line so that they have more time to complete, even with the reduced number of iterations). -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: