From alanb at openjdk.java.net Sun Nov 1 09:13:55 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Sun, 1 Nov 2020 09:13:55 GMT Subject: RFR: 8255576: (fs) Files.isHidden() throws ArrayIndexOutOfBoundsException (unix) [v3] In-Reply-To: References: <4gu-AnyT-WPUIUy2sgA-kBy8RicDJ4zCr-c8i7wERpc=.18af7c2f-175f-43e2-83c4-d987b0cdcea8@github.com> Message-ID: On Sat, 31 Oct 2020 20:06:04 GMT, Christian Stein wrote: >> This PR fixes a corner case of `Files.isHidden(Path)` where the given path is an empty path: `Path.of("")`. > > Christian Stein has updated the pull request incrementally with one additional commit since the last revision: > > Wrapped call in logical assertFalse() check > > Asserting that the current directory isn't hidden. Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/952 From alanb at openjdk.java.net Sun Nov 1 09:13:55 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Sun, 1 Nov 2020 09:13:55 GMT Subject: RFR: 8255576: (fs) Files.isHidden() throws ArrayIndexOutOfBoundsException (unix) [v3] In-Reply-To: References: <4gu-AnyT-WPUIUy2sgA-kBy8RicDJ4zCr-c8i7wERpc=.18af7c2f-175f-43e2-83c4-d987b0cdcea8@github.com> <-Ns7q3kUNH-tDxy_TOTOa7Vd1pjkqr2FGtBiXUREJ5Q=.b81d6850-aae7-4e7b-bd38-65c038ad1143@github.com> Message-ID: On Sat, 31 Oct 2020 20:10:18 GMT, Christian Stein wrote: >> Thanks for the update, I think I would put an assertFalse around this because the current directory shouldn't be hidden. >> >> Creating a new VM that runs in a hidden directory is probably a stretch. I don't think it is strictly required here but don't object to reviewing any proposals if you are interested in spending time on it. > > I wrapped a "logical assertFalse" around the call -- there were already 32 cases of `assertTrue(! ...)` via https://github.com/openjdk/jdk/pull/952/commits/c27abeab3a38daf2ae5dc25894c8b966d9edfc58 > > I want to defer such a check to a different issue. If there was a feature like "chroot(Path)", or a virtual file system that could be mapped to the default one so that Path("") resolves to a hidden working directory ... or something else that's better (and faster) than spawning a new VM. This test pre-dates the use of TestNG support so had its own assertTrue. We could add an assertFalse but what you have is okay too as it's just one usage. A chroot(Path) like API would break the assumption that the current directory doesn't change. Every relative path would locate a different file. So if we add a test, and I don't think it's too important, then it would need to launch a new VM. There is test lib infrastructure to support this. Historically, multi VM tests tended to have reliability issues, also they are significantly slower that tests that run "in-process". ------------- PR: https://git.openjdk.java.net/jdk/pull/952 From cstein at openjdk.java.net Sun Nov 1 12:42:55 2020 From: cstein at openjdk.java.net (Christian Stein) Date: Sun, 1 Nov 2020 12:42:55 GMT Subject: Integrated: 8255576: (fs) Files.isHidden() throws ArrayIndexOutOfBoundsException (unix) In-Reply-To: <4gu-AnyT-WPUIUy2sgA-kBy8RicDJ4zCr-c8i7wERpc=.18af7c2f-175f-43e2-83c4-d987b0cdcea8@github.com> References: <4gu-AnyT-WPUIUy2sgA-kBy8RicDJ4zCr-c8i7wERpc=.18af7c2f-175f-43e2-83c4-d987b0cdcea8@github.com> Message-ID: On Fri, 30 Oct 2020 08:56:21 GMT, Christian Stein wrote: > This PR fixes a corner case of `Files.isHidden(Path)` where the given path is an empty path: `Path.of("")`. This pull request has now been integrated. Changeset: f61ce327 Author: Christian Stein Committer: Alan Bateman URL: https://git.openjdk.java.net/jdk/commit/f61ce327 Stats: 13 lines in 3 files changed: 10 ins; 0 del; 3 mod 8255576: (fs) Files.isHidden() throws ArrayIndexOutOfBoundsException (unix) Reviewed-by: alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/952 From lancea at openjdk.java.net Sun Nov 1 20:20:04 2020 From: lancea at openjdk.java.net (Lance Andersen) Date: Sun, 1 Nov 2020 20:20:04 GMT Subject: RFR: 8255380: (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false Message-ID: Hi, Please review the fix for JDK-8255380 which addresses an issue when the Zip file is > 4GB. Zip FS when processing the CEN extra data does not take into account the fact that there is no specific order to how the extra data fields are written. Info-ZIP writes the fields in a different order than Zip FS which presents a problem when evaluating the Info-ZIP extended timestamp and the LOC offset is 0XFFFFFFFF therefore the LOC offset needs to be read from the EXTID_ZIP64 extra data prior to attempting to read the LOC extra data field. The fix will defer reading of the LOC extra data field, if needed until all of the CEN extra data has been processed. Using jdk.nio.zipfs.ZipInfo, you can see the ordering difference of the CEND extra data fields when using Zip FS and info-zip. Info-zip is included with Mac OS so the test uses ProcessBuilder to execute zip on Mac OS and Linux. Mach5 tests jdk-tier1, jdk-tier2, and jdk-tier3 run cleanly. Best, Lance ------------- Commit messages: - (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false Changes: https://git.openjdk.java.net/jdk/pull/987/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=987&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255380 Stats: 311 lines in 2 files changed: 272 ins; 29 del; 10 mod Patch: https://git.openjdk.java.net/jdk/pull/987.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/987/head:pull/987 PR: https://git.openjdk.java.net/jdk/pull/987 From redestad at openjdk.java.net Sun Nov 1 22:23:56 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Sun, 1 Nov 2020 22:23:56 GMT Subject: RFR: 8255380: (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false In-Reply-To: References: Message-ID: <-nbhvVlcWVm1c8ZOXDHij7eG-LPTMLHYIzcNqT8MWpE=.f40672cc-4544-47ec-a3c6-f05f5dcda0c7@github.com> On Sun, 1 Nov 2020 20:09:32 GMT, Lance Andersen wrote: > Hi, > > Please review the fix for JDK-8255380 which addresses an issue when the Zip file is > 4GB. Zip FS when processing the CEN extra data does not take into account the fact that there is no specific order to how the extra data fields are written. Info-ZIP writes the fields in a different order than Zip FS which presents a problem when evaluating the Info-ZIP extended timestamp and the LOC offset is 0XFFFFFFFF therefore the LOC offset needs to be read from the EXTID_ZIP64 extra data prior to attempting to read the LOC extra data field. > > The fix will defer reading of the LOC extra data field, if needed until all of the CEN extra data has been processed. > > Using jdk.nio.zipfs.ZipInfo, you can see the ordering difference of the CEND extra data fields when using Zip FS and info-zip. > > Info-zip is included with Mac OS so the test uses ProcessBuilder to execute zip on Mac OS and Linux. > > Mach5 tests jdk-tier1, jdk-tier2, and jdk-tier3 run cleanly. > > Best, > Lance LGTM. Some nits inline. I guess the test can be cause for issues in some test systems since it needs to write out the 4Gb+ file. Is this why you've only enabled it on linux and mac? Perhaps someone might have ideas on how to improve this. src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java line 2993: > 2991: // We need to read the LOC extra data and the LOC offset was obtained > 2992: // from the EXTID_ZIP64 field. > 2993: if(hasZip64LocOffset) { Suggestion: if (hasZip64LocOffset) { src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java line 2978: > 2976: // prior to reading the LOC extra data field in order to obtain > 2977: // the Info-ZIP Extended Timestamp. > 2978: if(locoff != ZIP64_MINVAL) { Suggestion: if (locoff != ZIP64_MINVAL) { ------------- Marked as reviewed by redestad (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/987 From lancea at openjdk.java.net Sun Nov 1 22:59:08 2020 From: lancea at openjdk.java.net (Lance Andersen) Date: Sun, 1 Nov 2020 22:59:08 GMT Subject: RFR: 8255380: (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false [v2] In-Reply-To: References: Message-ID: > Hi, > > Please review the fix for JDK-8255380 which addresses an issue when the Zip file is > 4GB. Zip FS when processing the CEN extra data does not take into account the fact that there is no specific order to how the extra data fields are written. Info-ZIP writes the fields in a different order than Zip FS which presents a problem when evaluating the Info-ZIP extended timestamp and the LOC offset is 0XFFFFFFFF therefore the LOC offset needs to be read from the EXTID_ZIP64 extra data prior to attempting to read the LOC extra data field. > > The fix will defer reading of the LOC extra data field, if needed until all of the CEN extra data has been processed. > > Using jdk.nio.zipfs.ZipInfo, you can see the ordering difference of the CEND extra data fields when using Zip FS and info-zip. > > Info-zip is included with Mac OS so the test uses ProcessBuilder to execute zip on Mac OS and Linux. > > Mach5 tests jdk-tier1, jdk-tier2, and jdk-tier3 run cleanly. > > Best, > Lance Lance Andersen has updated the pull request incrementally with one additional commit since the last revision: Addressed feedback from Claes ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/987/files - new: https://git.openjdk.java.net/jdk/pull/987/files/7bb9a906..efa7b750 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=987&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=987&range=00-01 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/987.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/987/head:pull/987 PR: https://git.openjdk.java.net/jdk/pull/987 From lancea at openjdk.java.net Sun Nov 1 22:59:09 2020 From: lancea at openjdk.java.net (Lance Andersen) Date: Sun, 1 Nov 2020 22:59:09 GMT Subject: RFR: 8255380: (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false [v2] In-Reply-To: <-nbhvVlcWVm1c8ZOXDHij7eG-LPTMLHYIzcNqT8MWpE=.f40672cc-4544-47ec-a3c6-f05f5dcda0c7@github.com> References: <-nbhvVlcWVm1c8ZOXDHij7eG-LPTMLHYIzcNqT8MWpE=.f40672cc-4544-47ec-a3c6-f05f5dcda0c7@github.com> Message-ID: On Sun, 1 Nov 2020 22:21:25 GMT, Claes Redestad wrote: > LGTM. Some nits inline. > > I guess the test can be cause for issues in some test systems since it needs to write out the 4Gb+ file. Is this why you've only enabled it on linux and mac? Perhaps someone might have ideas on how to improve this. To validate the test, requires info-zip which comes on Mac and linux. It is not included with windows. There are no issues if the Zip is created via java.util.zip or Zip FS > src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java line 2993: > >> 2991: // We need to read the LOC extra data and the LOC offset was obtained >> 2992: // from the EXTID_ZIP64 field. >> 2993: if(hasZip64LocOffset) { > > Suggestion: > > if (hasZip64LocOffset) { Resolved ------------- PR: https://git.openjdk.java.net/jdk/pull/987 From redestad at openjdk.java.net Sun Nov 1 23:07:54 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Sun, 1 Nov 2020 23:07:54 GMT Subject: RFR: 8255380: (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false [v2] In-Reply-To: References: <-nbhvVlcWVm1c8ZOXDHij7eG-LPTMLHYIzcNqT8MWpE=.f40672cc-4544-47ec-a3c6-f05f5dcda0c7@github.com> Message-ID: On Sun, 1 Nov 2020 22:55:48 GMT, Lance Andersen wrote: > To validate the test, requires info-zip which comes on Mac and linux. It is not included with windows. There are no issues if the Zip is created via java.util.zip or Zip FS Ah, gotcha. I see and understand the use of the external zip now. This seem a bit fragile - maybe needs to be more graceful in case the system zip doesn't exist or doesn't do what you expect? ------------- PR: https://git.openjdk.java.net/jdk/pull/987 From lancea at openjdk.java.net Sun Nov 1 23:44:55 2020 From: lancea at openjdk.java.net (Lance Andersen) Date: Sun, 1 Nov 2020 23:44:55 GMT Subject: RFR: 8255380: (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false [v2] In-Reply-To: References: <-nbhvVlcWVm1c8ZOXDHij7eG-LPTMLHYIzcNqT8MWpE=.f40672cc-4544-47ec-a3c6-f05f5dcda0c7@github.com> Message-ID: On Sun, 1 Nov 2020 23:04:47 GMT, Claes Redestad wrote: > > To validate the test, requires info-zip which comes on Mac and linux. It is not included with windows. There are no issues if the Zip is created via java.util.zip or Zip FS > > Ah, gotcha. I see and understand the use of the external zip now. This seem a bit fragile - maybe needs to be more graceful in case the system zip doesn't exist or doesn't do what you expect? Zip is provided on Mac and Linux so unless the system is mis-configured, zip should be available on the path. Are you suggesting to execute "zip -h" or "which zip" and skip if zip is not there? I could make it a manual test but thought given zip should be on linux and Mac that should be OK. As long as the zip file is created, we are good to go. I chose not to fall back to Zip FS (or java.util.zip) to create the zip file as I know this will always pass. Given the size of the file needed, it would be too large to check a file into the workspace. Please let me know if you had another idea (Or prefer this to be a manual test) ------------- PR: https://git.openjdk.java.net/jdk/pull/987 From redestad at openjdk.java.net Sun Nov 1 23:47:56 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Sun, 1 Nov 2020 23:47:56 GMT Subject: RFR: 8255380: (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false [v2] In-Reply-To: References: <-nbhvVlcWVm1c8ZOXDHij7eG-LPTMLHYIzcNqT8MWpE=.f40672cc-4544-47ec-a3c6-f05f5dcda0c7@github.com> Message-ID: On Sun, 1 Nov 2020 23:41:43 GMT, Lance Andersen wrote: >>> To validate the test, requires info-zip which comes on Mac and linux. It is not included with windows. There are no issues if the Zip is created via java.util.zip or Zip FS >> >> Ah, gotcha. I see and understand the use of the external zip now. This seem a bit fragile - maybe needs to be more graceful in case the system zip doesn't exist or doesn't do what you expect? > >> > To validate the test, requires info-zip which comes on Mac and linux. It is not included with windows. There are no issues if the Zip is created via java.util.zip or Zip FS >> >> Ah, gotcha. I see and understand the use of the external zip now. This seem a bit fragile - maybe needs to be more graceful in case the system zip doesn't exist or doesn't do what you expect? > > Zip is provided on Mac and Linux so unless the system is mis-configured, zip should be available on the path. Are you suggesting to execute "zip -h" or "which zip" and skip if zip is not there? I could make it a manual test but thought given zip should be on linux and Mac that should be OK. As long as the zip file is created, we are good to go. I chose not to fall back to Zip FS (or java.util.zip) to create the zip file as I know this will always pass. > > Given the size of the file needed, it would be too large to check a file into the workspace. > > Please let me know if you had another idea (Or prefer this to be a manual test) Given the need to generate a 4Gb file on the fly I do feel this maybe ought to be a manual test, or maybe there's some annotation to ensure it's not run on systems with too little disk space (and no compatible zip). ------------- PR: https://git.openjdk.java.net/jdk/pull/987 From lancea at openjdk.java.net Sun Nov 1 23:51:54 2020 From: lancea at openjdk.java.net (Lance Andersen) Date: Sun, 1 Nov 2020 23:51:54 GMT Subject: RFR: 8255380: (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false [v2] In-Reply-To: References: <-nbhvVlcWVm1c8ZOXDHij7eG-LPTMLHYIzcNqT8MWpE=.f40672cc-4544-47ec-a3c6-f05f5dcda0c7@github.com> Message-ID: On Sun, 1 Nov 2020 23:44:53 GMT, Claes Redestad wrote: > Given the need to generate a 4Gb file on the fly I do feel this maybe ought to be a manual test, or maybe there's some annotation to ensure it's not run on systems with too little disk space (and no compatible zip). OK, I can make it a manual test, thank you for the feedback Claes! ------------- PR: https://git.openjdk.java.net/jdk/pull/987 From lancea at openjdk.java.net Mon Nov 2 00:28:08 2020 From: lancea at openjdk.java.net (Lance Andersen) Date: Mon, 2 Nov 2020 00:28:08 GMT Subject: RFR: 8255380: (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false [v3] In-Reply-To: References: Message-ID: > Hi, > > Please review the fix for JDK-8255380 which addresses an issue when the Zip file is > 4GB. Zip FS when processing the CEN extra data does not take into account the fact that there is no specific order to how the extra data fields are written. Info-ZIP writes the fields in a different order than Zip FS which presents a problem when evaluating the Info-ZIP extended timestamp and the LOC offset is 0XFFFFFFFF therefore the LOC offset needs to be read from the EXTID_ZIP64 extra data prior to attempting to read the LOC extra data field. > > The fix will defer reading of the LOC extra data field, if needed until all of the CEN extra data has been processed. > > Using jdk.nio.zipfs.ZipInfo, you can see the ordering difference of the CEND extra data fields when using Zip FS and info-zip. > > Info-zip is included with Mac OS so the test uses ProcessBuilder to execute zip on Mac OS and Linux. > > Mach5 tests jdk-tier1, jdk-tier2, and jdk-tier3 run cleanly. > > Best, > Lance Lance Andersen has updated the pull request incrementally with one additional commit since the last revision: Convert test to being a manual test ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/987/files - new: https://git.openjdk.java.net/jdk/pull/987/files/efa7b750..bcf73949 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=987&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=987&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/987.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/987/head:pull/987 PR: https://git.openjdk.java.net/jdk/pull/987 From redestad at openjdk.java.net Mon Nov 2 10:50:56 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Mon, 2 Nov 2020 10:50:56 GMT Subject: RFR: 8255380: (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false [v3] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 00:28:08 GMT, Lance Andersen wrote: >> Hi, >> >> Please review the fix for JDK-8255380 which addresses an issue when the Zip file is > 4GB. Zip FS when processing the CEN extra data does not take into account the fact that there is no specific order to how the extra data fields are written. Info-ZIP writes the fields in a different order than Zip FS which presents a problem when evaluating the Info-ZIP extended timestamp and the LOC offset is 0XFFFFFFFF therefore the LOC offset needs to be read from the EXTID_ZIP64 extra data prior to attempting to read the LOC extra data field. >> >> The fix will defer reading of the LOC extra data field, if needed until all of the CEN extra data has been processed. >> >> Using jdk.nio.zipfs.ZipInfo, you can see the ordering difference of the CEND extra data fields when using Zip FS and info-zip. >> >> Info-zip is included with Mac OS so the test uses ProcessBuilder to execute zip on Mac OS and Linux. >> >> Mach5 tests jdk-tier1, jdk-tier2, and jdk-tier3 run cleanly. >> >> Best, >> Lance > > Lance Andersen has updated the pull request incrementally with one additional commit since the last revision: > > Convert test to being a manual test Marked as reviewed by redestad (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/987 From github.com+760524+mjpt777 at openjdk.java.net Mon Nov 2 15:24:55 2020 From: github.com+760524+mjpt777 at openjdk.java.net (Martin Thompson) Date: Mon, 2 Nov 2020 15:24:55 GMT Subject: RFR: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:37:53 GMT, Brian Burkhalter wrote: > Please review this proposed fix. The previous incarnation of this review was most recently in mail threads [1, 2] under the old Hg SCM. The implementation changes proposed here are derived from the final (.02) patch posted in [1]. The corresponding CSR is [3]. > > The essence of the change is to move the bulk put buffer core functionality to a new method putBuffer(), modify the existing relative bulk put buffer method to call it, and add a new absolute bulk put buffer method which also calls it. The existing test for bulk put buffer functionality is modified to verify the new API. > > This change builds on the work completed for JDK-5029431 (bulk put and get using an array) and JDK-8245121 (relative bulk put buffer with overlapping regions). > > [1] http://mail.openjdk.java.net/pipermail/nio-dev/2020-June/007340.html > [2] http://mail.openjdk.java.net/pipermail/nio-dev/2020-July/007344.html > [3] https://bugs.openjdk.java.net/browse/JDK-8248431 The other method implementations which do a absolute put or get do not modify the position. This makes them more useful in a multi-threaded context as different threads can be updating different ranges of the buffer concurrently. To replace Unsafe we need absolute index methods that do not update position. ------------- PR: https://git.openjdk.java.net/jdk/pull/969 From redestad at openjdk.java.net Mon Nov 2 16:07:00 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Mon, 2 Nov 2020 16:07:00 GMT Subject: RFR: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 14:12:36 GMT, Martin Thompson wrote: > The other method implementations which do a absolute put or get do not modify the position. This makes them more useful in a multi-threaded context as different threads can be updating different ranges of the buffer concurrently. To replace Unsafe we need absolute index methods that do not update position. Thanks for pointing out the importance of not depending on `position` for these methods to be as helpful as they can be. I checked and the new methods here (`put(int, $Type$Buffer, int, int)`) does not modify or read `position`. I think the diff here makes it look like `position` is in use for the new methods since the diff gets bungled up with the preceding, position-relative `put($Type$Buffer)` method, and that this played some tricks. ------------- PR: https://git.openjdk.java.net/jdk/pull/969 From Alan.Bateman at oracle.com Mon Nov 2 16:14:39 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 2 Nov 2020 16:14:39 +0000 Subject: RFR: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: References: Message-ID: <70ceaa8e-c358-ae27-5b51-017e72a813e2@oracle.com> On 02/11/2020 15:24, Martin Thompson wrote: > On Fri, 30 Oct 2020 20:37:53 GMT, Brian Burkhalter wrote: > >> Please review this proposed fix. The previous incarnation of this review was most recently in mail threads [1, 2] under the old Hg SCM. The implementation changes proposed here are derived from the final (.02) patch posted in [1]. The corresponding CSR is [3]. >> >> The essence of the change is to move the bulk put buffer core functionality to a new method putBuffer(), modify the existing relative bulk put buffer method to call it, and add a new absolute bulk put buffer method which also calls it. The existing test for bulk put buffer functionality is modified to verify the new API. >> >> This change builds on the work completed for JDK-5029431 (bulk put and get using an array) and JDK-8245121 (relative bulk put buffer with overlapping regions). >> >> [1] http://mail.openjdk.java.net/pipermail/nio-dev/2020-June/007340.html >> [2] http://mail.openjdk.java.net/pipermail/nio-dev/2020-July/007344.html >> [3] https://bugs.openjdk.java.net/browse/JDK-8248431 > The other method implementations which do a absolute put or get do not modify the position. This makes them more useful in a multi-threaded context as different threads can be updating different ranges of the buffer concurrently. To replace Unsafe we need absolute index methods that do not update position. > These methods are absolute bulk operations, the index of both the source and receiver are specified. The Foreign Memory Access API, in development in Project Panama, may be more appropriate for replacing code that is using Unsafe for off-heap memory. -Alan From bpb at openjdk.java.net Mon Nov 2 20:40:04 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 2 Nov 2020 20:40:04 GMT Subject: RFR: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null Message-ID: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> Please review this simple fix to have Path.of(String,String) throw a NullPointerException if the first parameter is null. ------------- Commit messages: - 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null Changes: https://git.openjdk.java.net/jdk/pull/1020/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1020&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8254876 Stats: 14 lines in 3 files changed: 12 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1020.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1020/head:pull/1020 PR: https://git.openjdk.java.net/jdk/pull/1020 From rriggs at openjdk.java.net Mon Nov 2 21:59:55 2020 From: rriggs at openjdk.java.net (Roger Riggs) Date: Mon, 2 Nov 2020 21:59:55 GMT Subject: RFR: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null In-Reply-To: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> References: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> Message-ID: On Mon, 2 Nov 2020 20:28:33 GMT, Brian Burkhalter wrote: > Please review this simple fix to have Path.of(String,String) throw a NullPointerException if the first parameter is null. src/java.base/unix/classes/sun/nio/fs/UnixFileSystem.java line 264: > 262: path = first; > 263: } else { > 264: if (first == null) { `Objects.requireNonNull(first)` does this in fewer lines and a bit more explicitly. Should it be checking `first` always, not just in the case `more.length != 0`? (It will throw NPE, but not until `UnixPath.normalizeAndCheck(path))`. ------------- PR: https://git.openjdk.java.net/jdk/pull/1020 From bpb at openjdk.java.net Mon Nov 2 21:59:56 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 2 Nov 2020 21:59:56 GMT Subject: RFR: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null In-Reply-To: References: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> Message-ID: On Mon, 2 Nov 2020 21:54:50 GMT, Roger Riggs wrote: >> Please review this simple fix to have Path.of(String,String) throw a NullPointerException if the first parameter is null. > > src/java.base/unix/classes/sun/nio/fs/UnixFileSystem.java line 264: > >> 262: path = first; >> 263: } else { >> 264: if (first == null) { > > `Objects.requireNonNull(first)` does this in fewer lines and a bit more explicitly. > > Should it be checking `first` always, not just in the case `more.length != 0`? > (It will throw NPE, but not until `UnixPath.normalizeAndCheck(path))`. Good point on the Objects call. I checked the more.length == 0 case and let if fall through to the UnixPath method. It might be more straightforward to always check however. ------------- PR: https://git.openjdk.java.net/jdk/pull/1020 From bpb at openjdk.java.net Mon Nov 2 22:09:09 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 2 Nov 2020 22:09:09 GMT Subject: RFR: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null [v2] In-Reply-To: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> References: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> Message-ID: > Please review this simple fix to have Path.of(String,String) throw a NullPointerException if the first parameter is null. Brian Burkhalter has updated the pull request incrementally with two additional commits since the last revision: - 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null - 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1020/files - new: https://git.openjdk.java.net/jdk/pull/1020/files/7230ae92..5d5adf47 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1020&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1020&range=00-01 Stats: 8 lines in 2 files changed: 2 ins; 6 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1020.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1020/head:pull/1020 PR: https://git.openjdk.java.net/jdk/pull/1020 From rriggs at openjdk.java.net Mon Nov 2 22:53:54 2020 From: rriggs at openjdk.java.net (Roger Riggs) Date: Mon, 2 Nov 2020 22:53:54 GMT Subject: RFR: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null [v2] In-Reply-To: References: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> Message-ID: On Mon, 2 Nov 2020 22:09:09 GMT, Brian Burkhalter wrote: >> Please review this simple fix to have Path.of(String,String) throw a NullPointerException if the first parameter is null. > > Brian Burkhalter has updated the pull request incrementally with two additional commits since the last revision: > > - 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null > - 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null Marked as reviewed by rriggs (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1020 From psandoz at openjdk.java.net Tue Nov 3 00:22:57 2020 From: psandoz at openjdk.java.net (Paul Sandoz) Date: Tue, 3 Nov 2020 00:22:57 GMT Subject: RFR: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:37:53 GMT, Brian Burkhalter wrote: > Please review this proposed fix. The previous incarnation of this review was most recently in mail threads [1, 2] under the old Hg SCM. The implementation changes proposed here are derived from the final (.02) patch posted in [1]. The corresponding CSR is [3]. > > The essence of the change is to move the bulk put buffer core functionality to a new method putBuffer(), modify the existing relative bulk put buffer method to call it, and add a new absolute bulk put buffer method which also calls it. The existing test for bulk put buffer functionality is modified to verify the new API. > > This change builds on the work completed for JDK-5029431 (bulk put and get using an array) and JDK-8245121 (relative bulk put buffer with overlapping regions). > > [1] http://mail.openjdk.java.net/pipermail/nio-dev/2020-June/007340.html > [2] http://mail.openjdk.java.net/pipermail/nio-dev/2020-July/007344.html > [3] https://bugs.openjdk.java.net/browse/JDK-8248431 I like the way this patch builds on prior work and reuse, including the test. It gives us confidence, since this is a tricky area to get right, esp. with edge cases like `StringCharBuffer`. ------------- Marked as reviewed by psandoz (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/969 From psandoz at openjdk.java.net Tue Nov 3 00:31:53 2020 From: psandoz at openjdk.java.net (Paul Sandoz) Date: Tue, 3 Nov 2020 00:31:53 GMT Subject: RFR: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 00:20:32 GMT, Paul Sandoz wrote: >> Please review this proposed fix. The previous incarnation of this review was most recently in mail threads [1, 2] under the old Hg SCM. The implementation changes proposed here are derived from the final (.02) patch posted in [1]. The corresponding CSR is [3]. >> >> The essence of the change is to move the bulk put buffer core functionality to a new method putBuffer(), modify the existing relative bulk put buffer method to call it, and add a new absolute bulk put buffer method which also calls it. The existing test for bulk put buffer functionality is modified to verify the new API. >> >> This change builds on the work completed for JDK-5029431 (bulk put and get using an array) and JDK-8245121 (relative bulk put buffer with overlapping regions). >> >> [1] http://mail.openjdk.java.net/pipermail/nio-dev/2020-June/007340.html >> [2] http://mail.openjdk.java.net/pipermail/nio-dev/2020-July/007344.html >> [3] https://bugs.openjdk.java.net/browse/JDK-8248431 > > I like the way this patch builds on prior work and reuse, including the test. It gives us confidence, since this is a tricky area to get right, esp. with edge cases like `StringCharBuffer`. @mjpt777 adding to Alan's comment, we are looking for feedback on the memory API, especially in real-world performance sensitive use-cases (see design [doc](https://github.com/openjdk/panama-foreign/blob/foreign-jextract/doc/panama_memaccess.md), latest proposed incubating [api](http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html)). ------------- PR: https://git.openjdk.java.net/jdk/pull/969 From bpb at openjdk.java.net Tue Nov 3 02:31:06 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 3 Nov 2020 02:31:06 GMT Subject: RFR: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null [v3] In-Reply-To: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> References: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> Message-ID: <_V7e7T5Mikb6GKSMz37x5bBBGcgo4ofI5hhgI-wLJkY=.3464aca2-5a69-4a9f-a042-234cdff32ac1@github.com> > Please review this simple fix to have Path.of(String,String) throw a NullPointerException if the first parameter is null. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1020/files - new: https://git.openjdk.java.net/jdk/pull/1020/files/5d5adf47..3bcce9aa Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1020&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1020&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1020.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1020/head:pull/1020 PR: https://git.openjdk.java.net/jdk/pull/1020 From alanb at openjdk.java.net Tue Nov 3 07:12:57 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 3 Nov 2020 07:12:57 GMT Subject: RFR: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null [v3] In-Reply-To: <_V7e7T5Mikb6GKSMz37x5bBBGcgo4ofI5hhgI-wLJkY=.3464aca2-5a69-4a9f-a042-234cdff32ac1@github.com> References: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> <_V7e7T5Mikb6GKSMz37x5bBBGcgo4ofI5hhgI-wLJkY=.3464aca2-5a69-4a9f-a042-234cdff32ac1@github.com> Message-ID: On Tue, 3 Nov 2020 02:31:06 GMT, Brian Burkhalter wrote: >> Please review this simple fix to have Path.of(String,String) throw a NullPointerException if the first parameter is null. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null Marked as reviewed by alanb (Reviewer). test/jdk/java/nio/file/Path/PathOps.java line 2050: > 2048: } catch (NullPointerException npe) { > 2049: } > 2050: Probably should test Path.of("foo", null) too, just so we have tests for null at the first and second parameter in the same test. ------------- PR: https://git.openjdk.java.net/jdk/pull/1020 From honguye at microsoft.com Mon Nov 2 20:26:50 2020 From: honguye at microsoft.com (Nhat Nguyen) Date: Mon, 2 Nov 2020 20:26:50 +0000 Subject: [EXTERNAL] Re: Bug in Files.isSameFile(Path,Path) In-Reply-To: References: <31383fa0-ba8f-1e29-8d3a-950f34c724b0@freedom.nl> <505f6548-382b-f85c-bc89-b70bfae9e294@freedom.nl> <828f16b7-c438-9827-70fa-49ac15fcb20f@oracle.com> Message-ID: Hi Alan, We got the answers from our internal team. It turns out that volSerialNumber, as per the Windows Protocol Specs, isn't required to be non-zero [1]. Fortunately, we can use the file index value to determine whether the file systems support 64 file-index [2]. File index being 0 means the file system doesn't support 64-bit file index. In the case of webdav drive, the file index is indeed 0 from my testing. We were also cautioned that file systems that use 128-bit file index such as ReFS may return -1 in case the index doesn't fit into a 64-bit value. In such cases, they recommend that we use GetFileInformationByHandleEx [3] to get FILE_ID_INFO [4] which returns the 128-bit file id. The caveat is that FILE_ID_INFO is only supported from Windows 8.0/2012, and NTFS might have only implemented it as of 8.1/2012 R2. Furthermore, file systems are not required to implement it. >From my testing, webdav drives don't support getting FILE_ID_INFO using GetFileInformationByHandleEx. Given that FILE_ID_INFO is not fully supported by all versions of Windows which LTS JDK are required to run on, and that it isn't required to be implemented by all file systems, we think the simplest and best effort is to guard against 0 and -1 for file index, and fall back to comparing paths using GetFinalPathByHandle. What is your opinion on this? Also, we would be very appreciate if you can help us create a JBS bug for this issue so we can open PR against it. Thanks, Nhat [1]: "VolumeSerialNumber: No specific format or content of this field is required for protocol interoperation. This value is not required to be unique." (https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/bf691378-c34e-4a13-976e-404ea1a87738.) [2]: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/2d3333fe-fc98-4a6f-98a2-4bb805aff407 [3]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-getfileinformationbyhandleex [4]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/ns-winbase-file_id_info -----Original Message----- From: Nikola Grcevski Sent: Thursday, October 29, 2020 8:24 AM To: Alan Bateman ; Nhat Nguyen ; WarnerJan Veldhuis ; nio-dev at openjdk.java.net Subject: RE: [EXTERNAL] Re: Bug in Files.isSameFile(Path,Path) Hi Alan, We are still trying to get to the bottom of this, it seems like Microsoft documented the API non-zero is safe to use behaviour for Python (https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F5764%2Ffiles&data=04%7C01%7Chonguye%40microsoft.com%7C05eaa87eb7264809f76b08d87c1ea288%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637395818299194688%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=pEvBImE7NdzHcncIYAFBWAHbViYXB20K7dokKTc0L6A%3D&reserved=0), but it's not there in MSDN. We are waiting to hear back on our internal developer support channel from the API owners. Cheers, Nikola -----Original Message----- From: Alan Bateman Sent: October 29, 2020 10:38 AM To: Nhat Nguyen ; Nikola Grcevski ; WarnerJan Veldhuis ; nio-dev at openjdk.java.net Subject: Re: [EXTERNAL] Re: Bug in Files.isSameFile(Path,Path) On 29/10/2020 00:13, Nhat Nguyen wrote: > Hi everyone, > > We looked into the issue and found that the values of volSerialNumber, > fileIndexHigh, and fileIndexLow in the file attributes, which are used > to determine if the two files are the same, are all zeroes when used > with webdav drives. However, this looks like a known issue as cpython seems to suffer from the same behaviour as well [1]. > > We have a suggested fix [2] that falls back to using > GetFinalPathByHandle and comparing the two paths when the > volSerialNumber is zero. We have also reached out to the Windows team to confirm if this is the preferred way to detect such cases; we will get back when we have hear back from them. > > [1]: > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith > ub.com%2Fpython%2Fcpython%2Fpull%2F5764%23discussion_r169221544&da > ta=04%7C01%7Chonguye%40microsoft.com%7C05eaa87eb7264809f76b08d87c1ea28 > 8%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637395818299194688%7CUn > known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haW > wiLCJXVCI6Mn0%3D%7C1000&sdata=Mc%2B3cceNDA%2FdnwLLHr1t9RoPWae5YkKW > pYlxiQWl%2Bx8%3D&reserved=0 > [2]: > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith > ub.com%2Fnhat-nguyen%2Fjdk%2Fcommit%2Fbd8fdc2809ed08e5564fe1eab4a7c1c2 > f32df84b&data=04%7C01%7Chonguye%40microsoft.com%7C05eaa87eb7264809 > f76b08d87c1ea288%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63739581 > 8299194688%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzI > iLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=396du2rLj4%2B7a%2FcP1 > 6P3czAAaafgjjxkMAJI1G14TPc%3D&reserved=0 > Thanks for looking into it. Do you know if the serial number = 0 is documented anywhere? I have a number of comments on how this workaround should fit in, I'll reply soon on this. -Alan. From alanb at openjdk.java.net Tue Nov 3 12:10:56 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 3 Nov 2020 12:10:56 GMT Subject: RFR: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:37:53 GMT, Brian Burkhalter wrote: > Please review this proposed fix. The previous incarnation of this review was most recently in mail threads [1, 2] under the old Hg SCM. The implementation changes proposed here are derived from the final (.02) patch posted in [1]. The corresponding CSR is [3]. > > The essence of the change is to move the bulk put buffer core functionality to a new method putBuffer(), modify the existing relative bulk put buffer method to call it, and add a new absolute bulk put buffer method which also calls it. The existing test for bulk put buffer functionality is modified to verify the new API. > > This change builds on the work completed for JDK-5029431 (bulk put and get using an array) and JDK-8245121 (relative bulk put buffer with overlapping regions). > > [1] http://mail.openjdk.java.net/pipermail/nio-dev/2020-June/007340.html > [2] http://mail.openjdk.java.net/pipermail/nio-dev/2020-July/007344.html > [3] https://bugs.openjdk.java.net/browse/JDK-8248431 Marked as reviewed by alanb (Reviewer). src/java.base/share/classes/java/nio/X-Buffer.java.template line 1013: > 1011: * > 1012: * @throws ReadOnlyBufferException > 1013: * If this buffer is read-only Can you align the parameter and throws descriptions so they are consistent with the other methods? test/jdk/java/nio/Buffer/BulkPutBuffer.java line 327: > 325: > 326: private static void expectThrows(Class exClass, > 327: Assert.ThrowingRunnable r) { Spurious line break in method declaration, also exClass should be Class ------------- PR: https://git.openjdk.java.net/jdk/pull/969 From Alan.Bateman at oracle.com Tue Nov 3 16:41:57 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 3 Nov 2020 16:41:57 +0000 Subject: [EXTERNAL] Re: Bug in Files.isSameFile(Path,Path) In-Reply-To: References: <31383fa0-ba8f-1e29-8d3a-950f34c724b0@freedom.nl> <505f6548-382b-f85c-bc89-b70bfae9e294@freedom.nl> <828f16b7-c438-9827-70fa-49ac15fcb20f@oracle.com> Message-ID: <0fb92dd2-7319-25ed-2d74-74445dc5aa8a@oracle.com> On 02/11/2020 20:26, Nhat Nguyen wrote: > Hi Alan, > > We got the answers from our internal team. > > It turns out that volSerialNumber, as per the Windows Protocol Specs, isn't required to be non-zero [1]. > Fortunately, we can use the file index value to determine whether the file systems support 64 file-index [2]. > > File index being 0 means the file system doesn't support 64-bit file index. In the case of webdav drive, the > file index is indeed 0 from my testing. We were also cautioned that file systems that use 128-bit file index > such as ReFS may return -1 in case the index doesn't fit into a 64-bit value. In such cases, they recommend > that we use GetFileInformationByHandleEx [3] to get FILE_ID_INFO [4] which returns the 128-bit file id. > > The caveat is that FILE_ID_INFO is only supported from Windows 8.0/2012, and NTFS might have only > implemented it as of 8.1/2012 R2. Furthermore, file systems are not required to implement it. > From my testing, webdav drives don't support getting FILE_ID_INFO using GetFileInformationByHandleEx. > > Given that FILE_ID_INFO is not fully supported by all versions of Windows which LTS JDK are required to run > on, and that it isn't required to be implemented by all file systems, we think the simplest and best effort > is to guard against 0 and -1 for file index, and fall back to comparing paths using GetFinalPathByHandle. > What is your opinion on this? > > Also, we would be very appreciate if you can help us create a JBS bug for this issue so we can open PR > against it. > Thanks for the update and information. I've created JDK-8255844 [1] to track this. I think I agree with the general approach. In terms of implementation then it might be simplest to add a hasFileIndex method to WindowsFileAttributes so that WindowsFileSystemProvider knows if it use the file index approach. If not, then fallback to using toRealPath as it will do all the checks needed to get the real path. -Alan [1] https://bugs.openjdk.java.net/browse/JDK-8255844 From bpb at openjdk.java.net Tue Nov 3 18:10:13 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 3 Nov 2020 18:10:13 GMT Subject: RFR: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer [v2] In-Reply-To: References: Message-ID: <1snVJzB715dy-34nOiXtQ5qlI44TMTqa4k-pXsBdAlI=.510d5dd4-dc47-4e62-82f3-da046b2e5f02@github.com> On Tue, 3 Nov 2020 12:08:23 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8219014: (bf) Add absolute bulk put methods which accept a source Buffer > > Marked as reviewed by alanb (Reviewer). Thanks for the comments. A second reviewer on the CSR would not hurt. ------------- PR: https://git.openjdk.java.net/jdk/pull/969 From bpb at openjdk.java.net Tue Nov 3 18:10:13 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 3 Nov 2020 18:10:13 GMT Subject: RFR: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer [v2] In-Reply-To: References: Message-ID: > Please review this proposed fix. The previous incarnation of this review was most recently in mail threads [1, 2] under the old Hg SCM. The implementation changes proposed here are derived from the final (.02) patch posted in [1]. The corresponding CSR is [3]. > > The essence of the change is to move the bulk put buffer core functionality to a new method putBuffer(), modify the existing relative bulk put buffer method to call it, and add a new absolute bulk put buffer method which also calls it. The existing test for bulk put buffer functionality is modified to verify the new API. > > This change builds on the work completed for JDK-5029431 (bulk put and get using an array) and JDK-8245121 (relative bulk put buffer with overlapping regions). > > [1] http://mail.openjdk.java.net/pipermail/nio-dev/2020-June/007340.html > [2] http://mail.openjdk.java.net/pipermail/nio-dev/2020-July/007344.html > [3] https://bugs.openjdk.java.net/browse/JDK-8248431 Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/969/files - new: https://git.openjdk.java.net/jdk/pull/969/files/1cfc8aba..12e4ddf5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=969&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=969&range=00-01 Stats: 13 lines in 2 files changed: 0 ins; 1 del; 12 mod Patch: https://git.openjdk.java.net/jdk/pull/969.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/969/head:pull/969 PR: https://git.openjdk.java.net/jdk/pull/969 From bpb at openjdk.java.net Tue Nov 3 18:13:09 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 3 Nov 2020 18:13:09 GMT Subject: RFR: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null [v4] In-Reply-To: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> References: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> Message-ID: > Please review this simple fix to have Path.of(String,String) throw a NullPointerException if the first parameter is null. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1020/files - new: https://git.openjdk.java.net/jdk/pull/1020/files/3bcce9aa..30a7fc6a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1020&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1020&range=02-03 Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1020.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1020/head:pull/1020 PR: https://git.openjdk.java.net/jdk/pull/1020 From rriggs at openjdk.java.net Tue Nov 3 18:46:58 2020 From: rriggs at openjdk.java.net (Roger Riggs) Date: Tue, 3 Nov 2020 18:46:58 GMT Subject: RFR: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null [v4] In-Reply-To: References: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> Message-ID: On Tue, 3 Nov 2020 18:13:09 GMT, Brian Burkhalter wrote: >> Please review this simple fix to have Path.of(String,String) throw a NullPointerException if the first parameter is null. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null Marked as reviewed by rriggs (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1020 From alanb at openjdk.java.net Tue Nov 3 19:23:00 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 3 Nov 2020 19:23:00 GMT Subject: RFR: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null [v4] In-Reply-To: References: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> Message-ID: On Tue, 3 Nov 2020 18:13:09 GMT, Brian Burkhalter wrote: >> Please review this simple fix to have Path.of(String,String) throw a NullPointerException if the first parameter is null. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1020 From lancea at openjdk.java.net Tue Nov 3 19:24:57 2020 From: lancea at openjdk.java.net (Lance Andersen) Date: Tue, 3 Nov 2020 19:24:57 GMT Subject: Integrated: 8255380: (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false In-Reply-To: References: Message-ID: On Sun, 1 Nov 2020 20:09:32 GMT, Lance Andersen wrote: > Hi, > > Please review the fix for JDK-8255380 which addresses an issue when the Zip file is > 4GB. Zip FS when processing the CEN extra data does not take into account the fact that there is no specific order to how the extra data fields are written. Info-ZIP writes the fields in a different order than Zip FS which presents a problem when evaluating the Info-ZIP extended timestamp and the LOC offset is 0XFFFFFFFF therefore the LOC offset needs to be read from the EXTID_ZIP64 extra data prior to attempting to read the LOC extra data field. > > The fix will defer reading of the LOC extra data field, if needed until all of the CEN extra data has been processed. > > Using jdk.nio.zipfs.ZipInfo, you can see the ordering difference of the CEND extra data fields when using Zip FS and info-zip. > > Info-zip is included with Mac OS so the test uses ProcessBuilder to execute zip on Mac OS and Linux. > > Mach5 tests jdk-tier1, jdk-tier2, and jdk-tier3 run cleanly. > > Best, > Lance This pull request has now been integrated. Changeset: 6606e090 Author: Lance Andersen URL: https://git.openjdk.java.net/jdk/commit/6606e090 Stats: 311 lines in 2 files changed: 272 ins; 29 del; 10 mod 8255380: (zipfs) ZipFileSystem::readExtra can fail if zipinfo-time is not set to false Reviewed-by: redestad ------------- PR: https://git.openjdk.java.net/jdk/pull/987 From bpb at openjdk.java.net Tue Nov 3 19:43:54 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 3 Nov 2020 19:43:54 GMT Subject: Integrated: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null In-Reply-To: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> References: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> Message-ID: On Mon, 2 Nov 2020 20:28:33 GMT, Brian Burkhalter wrote: > Please review this simple fix to have Path.of(String,String) throw a NullPointerException if the first parameter is null. This pull request has now been integrated. Changeset: 450452bb Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/450452bb Stats: 17 lines in 3 files changed: 14 ins; 0 del; 3 mod 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null Reviewed-by: rriggs, alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/1020 From honguye at microsoft.com Wed Nov 4 00:16:26 2020 From: honguye at microsoft.com (Nhat Nguyen) Date: Wed, 4 Nov 2020 00:16:26 +0000 Subject: [EXTERNAL] Re: Bug in Files.isSameFile(Path,Path) In-Reply-To: <0fb92dd2-7319-25ed-2d74-74445dc5aa8a@oracle.com> References: <31383fa0-ba8f-1e29-8d3a-950f34c724b0@freedom.nl> <505f6548-382b-f85c-bc89-b70bfae9e294@freedom.nl> <828f16b7-c438-9827-70fa-49ac15fcb20f@oracle.com> , <0fb92dd2-7319-25ed-2d74-74445dc5aa8a@oracle.com> Message-ID: Hi Alan, Thank you for the suggestion! I tried using toRealPath and noticed this interesting scenario. Let's assume that we have a drive "Z:" mapped to a sharepoint drive containing file "a.txt" and another soft link "C:/link" whose target is "Z:/a.txt". Further assume we want to see if "Z:/a.txt" and "C:/link" are the same. If we perform toRealPath on both inputs, "Z:/a.txt" will stay as "Z:/a.txt". However, "C:/link" will follow a different code path in WindowsLinkSupport.getRealPath where it notices that the input is a reparse point and eventually calls getFinalPath. The result for "C:/link" is then "\\sharepointHostName.com at SSL\DavWWWRoot\a.txt". So, we conclude that the files are different even though?they are the same. If we use GetFinalPath for both, we will be able to notice that they are the same. I'd love to know what you think about this scenario and if it's worth supporting it. Nhat ________________________________________ From: Alan Bateman Sent: Tuesday, November 3, 2020 8:41 AM To: Nhat Nguyen ; Nikola Grcevski ; WarnerJan Veldhuis ; nio-dev at openjdk.java.net Subject: Re: [EXTERNAL] Re: Bug in Files.isSameFile(Path,Path) ? On 02/11/2020 20:26, Nhat Nguyen wrote: > Hi Alan, > > We got the answers from our internal team. > > It turns out that volSerialNumber, as per the Windows Protocol Specs, isn't required to be non-zero [1]. > Fortunately, we can use the file index value to determine whether the file systems support 64 file-index [2]. > > File index being 0 means the file system doesn't support 64-bit file index. In the case of webdav drive, the > file index is indeed 0 from my testing. We were also cautioned that file systems that use 128-bit file index > such as ReFS may return -1 in case the index doesn't fit into a 64-bit value. In such cases, they recommend > that we use GetFileInformationByHandleEx [3] to get FILE_ID_INFO [4] which returns the 128-bit file id. > > The caveat is that FILE_ID_INFO is only supported from Windows 8.0/2012, and NTFS might have only > implemented it as of 8.1/2012 R2. Furthermore, file systems are not required to implement it. >? From my testing, webdav drives don't support getting FILE_ID_INFO using GetFileInformationByHandleEx. > > Given that FILE_ID_INFO is not fully supported by all versions of Windows which LTS JDK are required to run > on, and that it isn't required to be implemented by all file systems, we think the simplest and best effort > is to guard against 0 and -1 for file index, and fall back to comparing paths using GetFinalPathByHandle. > What is your opinion on this? > > Also, we would be very appreciate if you can help us create a JBS bug for this issue so we can open PR > against it. > Thanks for the update and information. I've created JDK-8255844 [1] to track this. I think I agree with the general approach. In terms of implementation then it might be simplest to add a hasFileIndex method to WindowsFileAttributes so that WindowsFileSystemProvider knows if it use the file index approach. If not, then fallback to using toRealPath as it will do all the checks needed to get the real path. -Alan [1] https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8255844&data=04%7C01%7Chonguye%40microsoft.com%7C942942d5be2f4e9d462d08d880176eaf%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637400185413261473%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XMTArZhuX%2B0VsfQUGqALll%2B1tLHwfcCn4yukTCZLPZI%3D&reserved=0 From alanb at openjdk.java.net Wed Nov 4 08:29:58 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 4 Nov 2020 08:29:58 GMT Subject: RFR: 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null [v4] In-Reply-To: References: <0s2YErQ2aM8bfrJ2fABvugYqT63jdERdM3IN9p-funY=.7326c384-9ec3-4591-b8d2-bc32fc5d99fe@github.com> Message-ID: On Tue, 3 Nov 2020 19:20:20 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8254876: (fs) NullPointerException not thrown when first argument to Path.of or Paths.get is null > > Marked as reviewed by alanb (Reviewer). I've added release-note=yes to the JBS issue as I think this one will need a release note. There may be an argument to submit a CSR too as this is changing long standing behavior (to align the method with its specification). ------------- PR: https://git.openjdk.java.net/jdk/pull/1020 From bpb at openjdk.java.net Thu Nov 5 00:38:00 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 5 Nov 2020 00:38:00 GMT Subject: RFR: 8255913: Decrease number of iterations in TestMaxCachedBufferSize Message-ID: Please review this test-only change to TestMaxCachedBufferSize. The test times out frequently, primarily on older hardware. The proposed change is to decrease the number of iterations in the test, which should not compromise its value. ------------- Commit messages: - 8255913: Decrease number of iterations in TestMaxCachedBufferSize Changes: https://git.openjdk.java.net/jdk/pull/1066/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1066&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255913 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1066.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1066/head:pull/1066 PR: https://git.openjdk.java.net/jdk/pull/1066 From lancea at openjdk.java.net Thu Nov 5 01:35:54 2020 From: lancea at openjdk.java.net (Lance Andersen) Date: Thu, 5 Nov 2020 01:35:54 GMT Subject: RFR: 8255913: Decrease number of iterations in TestMaxCachedBufferSize In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 23:28:12 GMT, Brian Burkhalter wrote: > Please review this test-only change to TestMaxCachedBufferSize. The test times out frequently, primarily on older hardware. The proposed change is to decrease the number of iterations in the test, which should not compromise its value. Marked as reviewed by lancea (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1066 From dfuchs at openjdk.java.net Thu Nov 5 10:23:03 2020 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Thu, 5 Nov 2020 10:23:03 GMT Subject: RFR: 8255758: JEP 380 spec clarifications In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 22:04:56 GMT, Michael McMahon wrote: > Minor spec changes from spec approved in initial CSR Marked as reviewed by dfuchs (Reviewer). src/java.base/share/classes/java/net/doc-files/net-properties.html line 250: > 248: Automatic binding of a server socket occurs when {@link > 249: java.nio.channels.ServerSocketChannel#bind(SocketAddress,int) ServerSocketChannel.bind} > 250: is called with a {@code null} address parameter. In this case, the system doesn't it also happen when it is called with a UnixDomainSocketAddress that has an empty path? src/java.base/share/classes/java/net/doc-files/net-properties.html line 246: > 244:

Unix domain sockets

> 245:

There are a number of system (and networking) properties that affect the behavior of > 246: channels to Unix domain server sockets when binding automatically. Should this also mention implicit binding? src/java.base/share/classes/java/net/doc-files/net-properties.html line 252: > 250: is requested to choose a unique path for the socket in some predefined system temporary > 251: directory. > 252:

There are a number of system (and networking) properties that affect the behavior of nit: `

` (lowercase and on a single line to match the style elsewhere) src/java.base/share/classes/java/net/doc-files/net-properties.html line 271: > 269: are set, then some systems (eg Windows) may check a commonly used environment > 270: variable as temporary directory. > 271:

  • {@systemProperty java.io.tmpdir} If the previous step fails to locate nit again (two places): `

  • ` should probably be `

  • ` especially since the `` tag is closed with `` ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From michaelm at openjdk.java.net Thu Nov 5 10:23:01 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Thu, 5 Nov 2020 10:23:01 GMT Subject: RFR: 8255758: JEP 380 spec clarifications Message-ID: Minor spec changes from spec approved in initial CSR ------------- Commit messages: - update to net-properties.html - 8255758: JEP 380 spec clarifications Changes: https://git.openjdk.java.net/jdk/pull/1021/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1021&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255758 Stats: 70 lines in 3 files changed: 66 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/1021.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1021/head:pull/1021 PR: https://git.openjdk.java.net/jdk/pull/1021 From michaelm at openjdk.java.net Thu Nov 5 10:23:05 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Thu, 5 Nov 2020 10:23:05 GMT Subject: RFR: 8255758: JEP 380 spec clarifications In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 09:25:52 GMT, Michael McMahon wrote: >> src/java.base/share/classes/java/net/doc-files/net-properties.html line 250: >> >>> 248: Automatic binding of a server socket occurs when {@link >>> 249: java.nio.channels.ServerSocketChannel#bind(SocketAddress,int) ServerSocketChannel.bind} >>> 250: is called with a {@code null} address parameter. In this case, the system >> >> doesn't it also happen when it is called with a UnixDomainSocketAddress that has an empty path? > > No, an empty path signifies an unnamed address and only client sockets (SocketChannels) are allowed to be unnamed. You get a BindException if you try to bind a ServerSocketChannel to the unnamed address. But, the exception message is "unhelpful". I would like to fix that. ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From michaelm at openjdk.java.net Thu Nov 5 10:23:04 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Thu, 5 Nov 2020 10:23:04 GMT Subject: RFR: 8255758: JEP 380 spec clarifications In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 09:09:18 GMT, Daniel Fuchs wrote: >> Minor spec changes from spec approved in initial CSR > > src/java.base/share/classes/java/net/doc-files/net-properties.html line 246: > >> 244:

    Unix domain sockets

    >> 245:

    There are a number of system (and networking) properties that affect the behavior of >> 246: channels to Unix domain server sockets when binding automatically. > > Should this also mention implicit binding? I'm not sure, as the system properties don't affect implicit binding, but I would have liked to have a place to "explain" all aspects of binding of Unix domain sockets/server-sockets. Maybe, I could add a paragraph at the end, just as an aside or note. > src/java.base/share/classes/java/net/doc-files/net-properties.html line 250: > >> 248: Automatic binding of a server socket occurs when {@link >> 249: java.nio.channels.ServerSocketChannel#bind(SocketAddress,int) ServerSocketChannel.bind} >> 250: is called with a {@code null} address parameter. In this case, the system > > doesn't it also happen when it is called with a UnixDomainSocketAddress that has an empty path? No, an empty path signifies an unnamed address and only client sockets (SocketChannels) are allowed to be unnamed. ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From dfuchs at openjdk.java.net Thu Nov 5 10:23:05 2020 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Thu, 5 Nov 2020 10:23:05 GMT Subject: RFR: 8255758: JEP 380 spec clarifications In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 09:29:54 GMT, Michael McMahon wrote: >> No, an empty path signifies an unnamed address and only client sockets (SocketChannels) are allowed to be unnamed. > > You get a BindException if you try to bind a ServerSocketChannel to the unnamed address. But, the exception message is "unhelpful". I would like to fix that. Ah good. I missed that paragraph was only talking about the _server_ sockets. ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From dfuchs at openjdk.java.net Thu Nov 5 10:23:05 2020 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Thu, 5 Nov 2020 10:23:05 GMT Subject: RFR: 8255758: JEP 380 spec clarifications In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 09:21:05 GMT, Michael McMahon wrote: >> src/java.base/share/classes/java/net/doc-files/net-properties.html line 246: >> >>> 244:

    Unix domain sockets

    >>> 245:

    There are a number of system (and networking) properties that affect the behavior of >>> 246: channels to Unix domain server sockets when binding automatically. >> >> Should this also mention implicit binding? > > I'm not sure, as the system properties don't affect implicit binding, but I would have liked to have a place to "explain" all aspects of binding of Unix domain sockets/server-sockets. Maybe, I could add a paragraph at the end, just as an aside or note. Oh - I was assuming that implicit binding & automatic binding were identical - just had a different trigger. Maybe it would be worth spelling out that implicit binding is **not** affected by the property then? For my own curiosity - is it because automatic binding is performed by our java implementation while implicit binding is performed by the system? ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From michaelm at openjdk.java.net Thu Nov 5 10:23:05 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Thu, 5 Nov 2020 10:23:05 GMT Subject: RFR: 8255758: JEP 380 spec clarifications In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 09:42:05 GMT, Daniel Fuchs wrote: >> I'm not sure, as the system properties don't affect implicit binding, but I would have liked to have a place to "explain" all aspects of binding of Unix domain sockets/server-sockets. Maybe, I could add a paragraph at the end, just as an aside or note. > > Oh - I was assuming that implicit binding & automatic binding were identical - just had a different trigger. Maybe it would be worth spelling out that implicit binding is **not** affected by the property then? For my own curiosity - is it because automatic binding is performed by our java implementation while implicit binding is performed by the system? These concepts aren't new by the way. "Implicit" binding is when you create a client socket and connect it without calling bind first. This is not possible with server sockets. "Automatic" binding is when you bind a server socket with a null address. All the above is the same for tcp and unix domain. tcp has more options relating to the interface and specifying a port = 0. These cases don't exist for unix domain. Just to add, one other point. It is also possible to automatically bind a client socket (either tcp or unix) (ie bind(null)) There is a small conceptual difference here between tcp and unix. With tcp you get a random port number, but with unix domain you get an unnamed socket address. ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From Alan.Bateman at oracle.com Thu Nov 5 14:12:04 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 5 Nov 2020 14:12:04 +0000 Subject: [EXTERNAL] Re: Bug in Files.isSameFile(Path,Path) In-Reply-To: References: <31383fa0-ba8f-1e29-8d3a-950f34c724b0@freedom.nl> <505f6548-382b-f85c-bc89-b70bfae9e294@freedom.nl> <828f16b7-c438-9827-70fa-49ac15fcb20f@oracle.com> <0fb92dd2-7319-25ed-2d74-74445dc5aa8a@oracle.com> Message-ID: On 04/11/2020 00:16, Nhat Nguyen wrote: > Hi Alan, > > Thank you for the suggestion! I tried using toRealPath and noticed this interesting scenario. > > Let's assume that we have a drive "Z:" mapped to a sharepoint drive containing file "a.txt" and another soft link "C:/link" > whose target is "Z:/a.txt". Further assume we want to see if "Z:/a.txt" and "C:/link" are the same. > > If we perform toRealPath on both inputs, "Z:/a.txt" will stay as "Z:/a.txt". However, "C:/link" will follow a different code path > in WindowsLinkSupport.getRealPath where it notices that the input is a reparse point and eventually calls getFinalPath. > The result for "C:/link" is then "\\sharepointHostName.com at SSL\DavWWWRoot\a.txt". So, we conclude that the > files are different even though?they are the same. > > If we use GetFinalPath for both, we will be able to notice that they are the same. I'd love to know what you think about this > scenario and if it's worth supporting it. > I agree we need to re-examine WindowsLinkSupport.getRealPath, at least for the followLinks=true case. If I remember correctly, we couldn't rely on GetFinalPathNameByHandle because it wasn't available on Windows XP and that is historical now. Also there was an issue with needing to open the file with backup semantics, I need to page in some of the details but I think that was needed for directories. There was another issue with sym links on NTFS that linked to files on non-NTFS volumes. If you confirm that GetFinalPathNameByHandle is essentially POSIX realpath then I would be happy to see WindowsLinkSupport.getRealPath changed to use that for the followLinks=true case. It might be that we need a fallback for cases where GetFinalPathNameByHandle fails. I'd also would be okay with change isSameFile to use toRealPath on both files so that it does not rely on the volume or file index. -Alan From dfuchs at openjdk.java.net Thu Nov 5 15:32:56 2020 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Thu, 5 Nov 2020 15:32:56 GMT Subject: RFR: 8255913: Decrease number of iterations in TestMaxCachedBufferSize In-Reply-To: References: Message-ID: <9osXlxmMlyBXtSYvOqRl6SMKwEkai0Vp_HujEuBcmDU=.34122d83-779d-4c38-9480-01d0993abf44@github.com> On Wed, 4 Nov 2020 23:28:12 GMT, Brian Burkhalter wrote: > Please review this test-only change to TestMaxCachedBufferSize. The test times out frequently, primarily on older hardware. The proposed change is to decrease the number of iterations in the test, which should not compromise its value. Marked as reviewed by dfuchs (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1066 From bpb at openjdk.java.net Thu Nov 5 16:31:00 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 5 Nov 2020 16:31:00 GMT Subject: Integrated: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:37:53 GMT, Brian Burkhalter wrote: > Please review this proposed fix. The previous incarnation of this review was most recently in mail threads [1, 2] under the old Hg SCM. The implementation changes proposed here are derived from the final (.02) patch posted in [1]. The corresponding CSR is [3]. > > The essence of the change is to move the bulk put buffer core functionality to a new method putBuffer(), modify the existing relative bulk put buffer method to call it, and add a new absolute bulk put buffer method which also calls it. The existing test for bulk put buffer functionality is modified to verify the new API. > > This change builds on the work completed for JDK-5029431 (bulk put and get using an array) and JDK-8245121 (relative bulk put buffer with overlapping regions). > > [1] http://mail.openjdk.java.net/pipermail/nio-dev/2020-June/007340.html > [2] http://mail.openjdk.java.net/pipermail/nio-dev/2020-July/007344.html > [3] https://bugs.openjdk.java.net/browse/JDK-8248431 This pull request has now been integrated. Changeset: a50fdd54 Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/a50fdd54 Stats: 204 lines in 4 files changed: 185 ins; 15 del; 4 mod 8219014: (bf) Add absolute bulk put methods which accept a source Buffer Reviewed-by: psandoz, alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/969 From bpb at openjdk.java.net Thu Nov 5 18:34:00 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 5 Nov 2020 18:34:00 GMT Subject: Integrated: 8255913: Decrease number of iterations in TestMaxCachedBufferSize In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 23:28:12 GMT, Brian Burkhalter wrote: > Please review this test-only change to TestMaxCachedBufferSize. The test times out frequently, primarily on older hardware. The proposed change is to decrease the number of iterations in the test, which should not compromise its value. This pull request has now been integrated. Changeset: d6f09404 Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/d6f09404 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8255913: Decrease number of iterations in TestMaxCachedBufferSize Reviewed-by: lancea, dfuchs ------------- PR: https://git.openjdk.java.net/jdk/pull/1066 From dfuchs at openjdk.java.net Tue Nov 10 17:11:02 2020 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Tue, 10 Nov 2020 17:11:02 GMT Subject: RFR: 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java Message-ID: Hi, Please find here a trivial cleanup change for test/jdk/java/nio/channels/DatagramChannel/Connect.java The change caters for possible evolution of the test - if we ever decided to bind the Reactor to the wildcard address; (the reactor is currently bound to the loopback). I would usually not bother but I had this experiment in my local workspace: it could as well be pushed as it cleans up the Actor logic which no longer has to depend on the fact that the Reactor is not bound to the wildcard. ------------- Commit messages: - 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java Changes: https://git.openjdk.java.net/jdk/pull/1144/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1144&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256146 Stats: 17 lines in 1 file changed: 15 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1144.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1144/head:pull/1144 PR: https://git.openjdk.java.net/jdk/pull/1144 From msheppar at openjdk.java.net Thu Nov 12 17:35:54 2020 From: msheppar at openjdk.java.net (Mark Sheppard) Date: Thu, 12 Nov 2020 17:35:54 GMT Subject: RFR: 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java In-Reply-To: References: Message-ID: On Tue, 10 Nov 2020 17:06:11 GMT, Daniel Fuchs wrote: > Hi, > > Please find here a trivial cleanup change for test/jdk/java/nio/channels/DatagramChannel/Connect.java > The change caters for possible evolution of the test - if we ever decided to bind the Reactor to the wildcard address; > (the reactor is currently bound to the loopback). > > I would usually not bother but I had this experiment in my local workspace: it could as well be pushed as it cleans up the Actor logic which no longer has to depend on the fact that the Reactor is not bound to the wildcard. Looks like a reasonable future proofing change :-) Couple of minor observations and comments: Actor constructor parameter name change: from socketAddress to either peerSocketAddress or connectSocketAddress or reactorSocketAddress to convey some semantics of its purpose. Change of Actor member variable name from socketAddress to connectSocketAddress - again more explicit semantic description When I saw the previous changes for this test and saw Reactor, the immediate thought was of Reactor pattern, but it is not. Names are somewhat idiosyncratic but Actor/Reactor don?t convey immediate semantics of the test. The more conventional Sender/Receiver, or if one seeks a bit of OSI nostalgia and more abstract terminology Initiator/Responder would convey clearer semantics of sending and receiving! ------------- PR: https://git.openjdk.java.net/jdk/pull/1144 From alanb at openjdk.java.net Thu Nov 12 19:37:57 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 12 Nov 2020 19:37:57 GMT Subject: RFR: 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java In-Reply-To: References: Message-ID: On Tue, 10 Nov 2020 17:06:11 GMT, Daniel Fuchs wrote: > Hi, > > Please find here a trivial cleanup change for test/jdk/java/nio/channels/DatagramChannel/Connect.java > The change caters for possible evolution of the test - if we ever decided to bind the Reactor to the wildcard address; > (the reactor is currently bound to the loopback). > > I would usually not bother but I had this experiment in my local workspace: it could as well be pushed as it cleans up the Actor logic which no longer has to depend on the fact that the Reactor is not bound to the wildcard. Looks okay. ------------- Marked as reviewed by alanb (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1144 From honguye at microsoft.com Fri Nov 13 00:25:48 2020 From: honguye at microsoft.com (Nhat Nguyen) Date: Fri, 13 Nov 2020 00:25:48 +0000 Subject: [EXTERNAL] Re: Bug in Files.isSameFile(Path,Path) In-Reply-To: References: <31383fa0-ba8f-1e29-8d3a-950f34c724b0@freedom.nl> <505f6548-382b-f85c-bc89-b70bfae9e294@freedom.nl> <828f16b7-c438-9827-70fa-49ac15fcb20f@oracle.com> <0fb92dd2-7319-25ed-2d74-74445dc5aa8a@oracle.com> Message-ID: Hi Alan, > If you confirm that GetFinalPathNameByHandle is essentially POSIX realpath then I would be happy > to see WindowsLinkSupport.getRealPath changed to use that for the followLinks=true case. It might > be that we need a fallback for cases where GetFinalPathNameByHandle fails I found an interesting discussion from the cpython folks [1]. They recently started using GetFinalPathNameByHandle by default for their realpath implementation [2]. If the call fails, they fall back to a combination of GetFinalPathNameByHandle and manually resolving links [3]. In this discussion, there was a particularly useful comment from Eryk Sun [4] where he mentioned that GetFinalPathNameByHandle is not entirely similar to POSIX realpath when the path contains a mount point: > Eryk Sun: > What we need is an implementation of realpath("C:/spam/scripts") that returns "C:\\spam\\scripts" > when "scripts" is a mount point and returns "C:\\spam\\etc\\scripts" when "scripts" is a symlink. > This means we need an implementation of realpath() that looks a lot like posixpath.realpath. > Generally a mount point should be walked over like a directory, just as mount points are handled in Unix. However, looking at the current implementation of OpenJDK's toRealPath, as soon as we detect that there's a subpath that contains the flag FILE_ATTRIBUTE_REPARSE_POINT, we immediately call getFinalPath on the original path. So if a path contains a symlink or a mount point, the result of toRealPath is always the result of getFinalPath (unless the call fails due to broken links, at which point we try to resolve them and recursively call toRealPath again). So my understanding is that we may still be ok to use getFinalPath for toRealPath, at least in the case where the path contains links/mount points, since the behaviour is the same as what we currently have. However, assuming we always use getFinalPath when followLinks=true, users who mount webdav on drive Z: will now get back the full link \\sharepointHostName.com at SSL\DavWWWRoot\a.txt" which used to be "Z:\a.txt". It is still unclear to me what should be the right behaviour in this particular case. > I'd also would be okay with change isSameFile to use toRealPath on both files so that it does not rely on the volume or file index. Solely for the purpose of file comparison, what is your opinion on using getFinalPath for isSameFile instead of toRealPath, which would also require some changes to its implementation? The paths are only used for comparison and aren't returned to the users, so we won't have to worry about breaking compatibility for existing code. Thanks, Nhat [1]: https://bugs.python.org/issue9949 [2]: https://github.com/python/cpython/blob/cc75ab791dd5ae2cb9f6e0c3c5f734a6ae1eb2a9/Lib/ntpath.py#L647 [3]: https://github.com/python/cpython/blob/cc75ab791dd5ae2cb9f6e0c3c5f734a6ae1eb2a9/Lib/ntpath.py#L579 [4]: https://bugs.python.org/msg350138, this is the directly link to one of the comments in [1] -----Original Message----- From: Alan Bateman Sent: Thursday, November 5, 2020 6:12 AM To: Nhat Nguyen ; Nikola Grcevski ; WarnerJan Veldhuis ; nio-dev at openjdk.java.net Subject: Re: [EXTERNAL] Re: Bug in Files.isSameFile(Path,Path) On 04/11/2020 00:16, Nhat Nguyen wrote: > Hi Alan, > > Thank you for the suggestion! I tried using toRealPath and noticed this interesting scenario. > > Let's assume that we have a drive "Z:" mapped to a sharepoint drive containing file "a.txt" and another soft link "C:/link" > whose target is "Z:/a.txt". Further assume we want to see if "Z:/a.txt" and "C:/link" are the same. > > If we perform toRealPath on both inputs, "Z:/a.txt" will stay as > "Z:/a.txt". However, "C:/link" will follow a different code path in WindowsLinkSupport.getRealPath where it notices that the input is a reparse point and eventually calls getFinalPath. > The result for "C:/link" is then > "\\sharepointHostName.com at SSL\DavWWWRoot\a.txt". So, we conclude that the files are different even though?they are the same. > > If we use GetFinalPath for both, we will be able to notice that they > are the same. I'd love to know what you think about this scenario and if it's worth supporting it. > I agree we need to re-examine WindowsLinkSupport.getRealPath, at least for the followLinks=true case. If I remember correctly, we couldn't rely on GetFinalPathNameByHandle because it wasn't available on Windows XP and that is historical now. Also there was an issue with needing to open the file with backup semantics, I need to page in some of the details but I think that was needed for directories. There was another issue with sym links on NTFS that linked to files on non-NTFS volumes. If you confirm that GetFinalPathNameByHandle is essentially POSIX realpath then I would be happy to see WindowsLinkSupport.getRealPath changed to use that for the followLinks=true case. It might be that we need a fallback for cases where GetFinalPathNameByHandle fails. I'd also would be okay with change isSameFile to use toRealPath on both files so that it does not rely on the volume or file index. -Alan From dfuchs at openjdk.java.net Fri Nov 13 17:05:19 2020 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Fri, 13 Nov 2020 17:05:19 GMT Subject: RFR: 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java [v2] In-Reply-To: References: Message-ID: <7qvr3pDjylU7nRjasS_zqlbW9gK44pkE6glEDyg9VFs=.5aae44aa-1237-40d4-a1b4-04a4ebde5aca@github.com> On Thu, 12 Nov 2020 19:35:10 GMT, Alan Bateman wrote: >> Daniel Fuchs has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: >> >> - 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java >> >> Integrated review comments. >> - Merge >> - 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java > > Looks okay. @msheppar Good suggestions! I have updated the PR. ------------- PR: https://git.openjdk.java.net/jdk/pull/1144 From dfuchs at openjdk.java.net Fri Nov 13 17:05:17 2020 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Fri, 13 Nov 2020 17:05:17 GMT Subject: RFR: 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java [v2] In-Reply-To: References: Message-ID: <5FQlbJpTjz3tJooahxezvApacIY_UJdLM0MfMVoW_bg=.80d6ab55-7b8a-451b-ba1e-c6f256fba682@github.com> > Hi, > > Please find here a trivial cleanup change for test/jdk/java/nio/channels/DatagramChannel/Connect.java > The change caters for possible evolution of the test - if we ever decided to bind the Reactor to the wildcard address; > (the reactor is currently bound to the loopback). > > I would usually not bother but I had this experiment in my local workspace: it could as well be pushed as it cleans up the Actor logic which no longer has to depend on the fact that the Reactor is not bound to the wildcard. Daniel Fuchs has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java Integrated review comments. - Merge - 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1144/files - new: https://git.openjdk.java.net/jdk/pull/1144/files/c884adb0..253d2ccc Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1144&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1144&range=00-01 Stats: 23220 lines in 438 files changed: 13605 ins; 6615 del; 3000 mod Patch: https://git.openjdk.java.net/jdk/pull/1144.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1144/head:pull/1144 PR: https://git.openjdk.java.net/jdk/pull/1144 From alanb at openjdk.java.net Fri Nov 13 17:22:04 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 13 Nov 2020 17:22:04 GMT Subject: RFR: 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java [v2] In-Reply-To: <7qvr3pDjylU7nRjasS_zqlbW9gK44pkE6glEDyg9VFs=.5aae44aa-1237-40d4-a1b4-04a4ebde5aca@github.com> References: <7qvr3pDjylU7nRjasS_zqlbW9gK44pkE6glEDyg9VFs=.5aae44aa-1237-40d4-a1b4-04a4ebde5aca@github.com> Message-ID: On Fri, 13 Nov 2020 17:02:24 GMT, Daniel Fuchs wrote: >> Looks okay. > > @msheppar Good suggestions! I have updated the PR. The previous iteration was a cleaner but okay. ------------- PR: https://git.openjdk.java.net/jdk/pull/1144 From alanb at openjdk.java.net Sun Nov 15 08:34:54 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Sun, 15 Nov 2020 08:34:54 GMT Subject: RFR: 8255758: JEP 380 spec clarifications In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 14:53:16 GMT, Daniel Fuchs wrote: >> Minor spec changes from spec approved in initial CSR > > Marked as reviewed by dfuchs (Reviewer). The clarification to the SecurityException looks good. The API docs specify that bind(null) will "bind to an automatically assigned socket address". Would it be better to lead with that phrase in the properties doc rather than switching to "automatically bound". That might also avoid it getting confused with an implicitly bound sockets. A minor comment on the test is that "peer" might be a better than "acc1". ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From michaelm at openjdk.java.net Mon Nov 16 13:29:26 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Mon, 16 Nov 2020 13:29:26 GMT Subject: RFR: 8255758: JEP 380 spec clarifications [v2] In-Reply-To: References: Message-ID: On Sun, 15 Nov 2020 08:32:28 GMT, Alan Bateman wrote: > The clarification to the SecurityException looks good. > > The API docs specify that bind(null) will "bind to an automatically assigned socket address". Would it be better to lead with that phrase in the properties doc rather than switching to "automatically bound". That might also avoid it getting confused with an implicitly bound sockets. > > A minor comment on the test is that "peer" might be a better than "acc1". I've just updated the spec per these comments and will update the CSR accordingly. Thx. ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From michaelm at openjdk.java.net Mon Nov 16 13:29:26 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Mon, 16 Nov 2020 13:29:26 GMT Subject: RFR: 8255758: JEP 380 spec clarifications [v2] In-Reply-To: References: Message-ID: > Minor spec changes from spec approved in initial CSR Michael McMahon has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: - spec update after Alan's comments - Merge branch 'master' into 8255758-spec-change - net-properties.html change only. src apidoc change to come - update to net-properties.html - 8255758: JEP 380 spec clarifications ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1021/files - new: https://git.openjdk.java.net/jdk/pull/1021/files/e17e78b4..52c12762 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1021&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1021&range=00-01 Stats: 118240 lines in 1590 files changed: 64315 ins; 42007 del; 11918 mod Patch: https://git.openjdk.java.net/jdk/pull/1021.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1021/head:pull/1021 PR: https://git.openjdk.java.net/jdk/pull/1021 From dfuchs at openjdk.java.net Mon Nov 16 14:45:08 2020 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Mon, 16 Nov 2020 14:45:08 GMT Subject: RFR: 8255758: JEP 380 spec clarifications [v2] In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 13:29:26 GMT, Michael McMahon wrote: >> Minor spec changes from spec approved in initial CSR > > Michael McMahon has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - spec update after Alan's comments > - Merge branch 'master' into 8255758-spec-change > - net-properties.html change only. src apidoc change to come > - update to net-properties.html > - 8255758: JEP 380 spec clarifications Marked as reviewed by dfuchs (Reviewer). src/java.base/share/classes/java/net/doc-files/net-properties.html line 276: > 274: {@code conf/net.properties} configuration file. > 275:

    > 276: Implicit binding of {@link java.nio.channels.SocketChannel SocketChannel}s Nit: could this be changed to: Implicit binding of a {@link java.nio.channels.SocketChannel SocketChannel} ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From michaelm at openjdk.java.net Mon Nov 16 14:58:12 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Mon, 16 Nov 2020 14:58:12 GMT Subject: RFR: 8255758: JEP 380 spec clarifications [v2] In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 14:42:02 GMT, Daniel Fuchs wrote: >> Michael McMahon has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: >> >> - spec update after Alan's comments >> - Merge branch 'master' into 8255758-spec-change >> - net-properties.html change only. src apidoc change to come >> - update to net-properties.html >> - 8255758: JEP 380 spec clarifications > > src/java.base/share/classes/java/net/doc-files/net-properties.html line 276: > >> 274: {@code conf/net.properties} configuration file. >> 275:

    >> 276: Implicit binding of {@link java.nio.channels.SocketChannel SocketChannel}s > > Nit: could this be changed to: > Implicit binding of a {@link java.nio.channels.SocketChannel SocketChannel} Done. Would you mind adding yourself as reviewer of the CSR? ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From alanb at openjdk.java.net Tue Nov 17 11:30:12 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 17 Nov 2020 11:30:12 GMT Subject: RFR: 8255758: JEP 380 spec clarifications [v2] In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 13:29:26 GMT, Michael McMahon wrote: >> Minor spec changes from spec approved in initial CSR > > Michael McMahon has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - spec update after Alan's comments > - Merge branch 'master' into 8255758-spec-change > - net-properties.html change only. src apidoc change to come > - update to net-properties.html > - 8255758: JEP 380 spec clarifications src/java.base/share/classes/java/net/doc-files/net-properties.html line 251: > 249: with a {@code null} address parameter will bind to an automatically assigned socket address. > 250: For Unix domain sockets, this means a unique path in some predefined system temporary directory. > 251:

    There are a number of system (and networking) properties that affect this behavior. "There are are a number of system properties ..." follows directly from the previous sentence so I assume you don't want a paragraph break here. src/java.base/share/classes/java/net/doc-files/net-properties.html line 253: > 251:

    There are a number of system (and networking) properties that affect this behavior. > 252:

    > 253: Bearing in mind that Unix domain socket addresses are limited in length to approximately 100 I think "Bearing in mind" should be dropped here and just say that Unix domain sockets are typically limited to a file path of 100 characters (typically it is bytes that may be too subtle for the context). ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From michaelm at openjdk.java.net Tue Nov 17 11:53:14 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Tue, 17 Nov 2020 11:53:14 GMT Subject: RFR: 8255758: JEP 380 spec clarifications [v3] In-Reply-To: References: Message-ID: > Minor spec changes from spec approved in initial CSR Michael McMahon has updated the pull request incrementally with two additional commits since the last revision: - update from Alan's comment Nov 17 - minor change suggested by Daniel ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1021/files - new: https://git.openjdk.java.net/jdk/pull/1021/files/52c12762..ce3bb6ff Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1021&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1021&range=01-02 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1021.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1021/head:pull/1021 PR: https://git.openjdk.java.net/jdk/pull/1021 From michaelm at openjdk.java.net Tue Nov 17 11:53:18 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Tue, 17 Nov 2020 11:53:18 GMT Subject: RFR: 8255758: JEP 380 spec clarifications [v2] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 11:25:22 GMT, Alan Bateman wrote: >> Michael McMahon has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: >> >> - spec update after Alan's comments >> - Merge branch 'master' into 8255758-spec-change >> - net-properties.html change only. src apidoc change to come >> - update to net-properties.html >> - 8255758: JEP 380 spec clarifications > > src/java.base/share/classes/java/net/doc-files/net-properties.html line 251: > >> 249: with a {@code null} address parameter will bind to an automatically assigned socket address. >> 250: For Unix domain sockets, this means a unique path in some predefined system temporary directory. >> 251:

    There are a number of system (and networking) properties that affect this behavior. > > "There are are a number of system properties ..." follows directly from the previous sentence so I assume you don't want a paragraph break here. Ok, that's fine. > src/java.base/share/classes/java/net/doc-files/net-properties.html line 253: > >> 251:

    There are a number of system (and networking) properties that affect this behavior. >> 252:

    >> 253: Bearing in mind that Unix domain socket addresses are limited in length to approximately 100 > > I think "Bearing in mind" should be dropped here and just say that Unix domain sockets are typically limited to a file path of 100 characters (typically it is bytes that may be too subtle for the context). Ok. Thanks ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From alanb at openjdk.java.net Tue Nov 17 12:02:06 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 17 Nov 2020 12:02:06 GMT Subject: RFR: 8255758: JEP 380 spec clarifications [v3] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 11:53:14 GMT, Michael McMahon wrote: >> Minor spec changes from spec approved in initial CSR > > Michael McMahon has updated the pull request incrementally with two additional commits since the last revision: > > - update from Alan's comment Nov 17 > - minor change suggested by Daniel Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From michaelm at openjdk.java.net Tue Nov 17 12:15:08 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Tue, 17 Nov 2020 12:15:08 GMT Subject: Integrated: 8255758: JEP 380 spec clarifications In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 22:04:56 GMT, Michael McMahon wrote: > Minor spec changes from spec approved in initial CSR This pull request has now been integrated. Changeset: 9d0ee66f Author: Michael McMahon URL: https://git.openjdk.java.net/jdk/commit/9d0ee66f Stats: 70 lines in 3 files changed: 64 ins; 0 del; 6 mod 8255758: JEP 380 spec clarifications Reviewed-by: dfuchs, alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/1021 From dfuchs at openjdk.java.net Tue Nov 17 12:24:04 2020 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Tue, 17 Nov 2020 12:24:04 GMT Subject: Integrated: 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java In-Reply-To: References: Message-ID: On Tue, 10 Nov 2020 17:06:11 GMT, Daniel Fuchs wrote: > Hi, > > Please find here a trivial cleanup change for test/jdk/java/nio/channels/DatagramChannel/Connect.java > The change caters for possible evolution of the test - if we ever decided to bind the Reactor to the wildcard address; > (the reactor is currently bound to the loopback). > > I would usually not bother but I had this experiment in my local workspace: it could as well be pushed as it cleans up the Actor logic which no longer has to depend on the fact that the Reactor is not bound to the wildcard. This pull request has now been integrated. Changeset: 9dbbe83a Author: Daniel Fuchs URL: https://git.openjdk.java.net/jdk/commit/9dbbe83a Stats: 37 lines in 1 file changed: 15 ins; 0 del; 22 mod 8256146: Cleanup test/jdk/java/nio/channels/DatagramChannel/Connect.java Reviewed-by: alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/1144 From michaelm at openjdk.java.net Tue Nov 17 13:47:11 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Tue, 17 Nov 2020 13:47:11 GMT Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' Message-ID: Could I get the following change reviewed please? The problem results from AbstractFileSystemProvider.getSunPathForSocketCall returning a path of '.' when given an empty path argument. The socket API needs an empty path byte[] in this situation. It causes a BindException which is actually correct, but the error message is confusing because "." always exists and the error message is saying 'path already exists' when the actual error is that server socket channels cannot be bound to the empty path. Thanks, Michael. ------------- Commit messages: - windows build error - remove TABs - Merge branch 'master' into JDK-8256461-AbstractFileSystemProvider - added a test - initial fix - spec update after Alan's comments - Merge branch 'master' into 8255758-spec-change - net-properties.html change only. src apidoc change to come - update to net-properties.html - 8255758: JEP 380 spec clarifications Changes: https://git.openjdk.java.net/jdk/pull/1258/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1258&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256461 Stats: 60 lines in 4 files changed: 60 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1258.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1258/head:pull/1258 PR: https://git.openjdk.java.net/jdk/pull/1258 From Alan.Bateman at oracle.com Tue Nov 17 14:00:57 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 17 Nov 2020 06:00:57 -0800 (PST) Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' In-Reply-To: References: Message-ID: <98c2648f-13de-70d7-96a0-e8e5d5b25779@oracle.com> On 17/11/2020 13:47, Michael McMahon wrote: > Could I get the following change reviewed please? > > The problem results from AbstractFileSystemProvider.getSunPathForSocketCall returning a path of '.' when given an empty path argument. The socket API needs an empty path byte[] in this situation. It causes a BindException which is actually correct, but the error message is confusing because "." always exists and the error message is saying 'path already exists' when the actual error is that server socket channels cannot be bound to the empty path. Can method description of getSunPathForSocketPath be updated to align with the new behavior? Also can you fix the exception in UnixDomainSocekts so that the casing is consistent with the other exceptions. I don't think test/sun/nio/fs is the right place for the test. Would it be better to create a test in test/jdk/java/nio/channels/unixdomain to check that bind throws an exception? -Alan. From michael.x.mcmahon at oracle.com Tue Nov 17 14:05:41 2020 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Tue, 17 Nov 2020 14:05:41 +0000 Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' In-Reply-To: <98c2648f-13de-70d7-96a0-e8e5d5b25779@oracle.com> References: <98c2648f-13de-70d7-96a0-e8e5d5b25779@oracle.com> Message-ID: On 17/11/2020 14:00, Alan Bateman wrote: > > > On 17/11/2020 13:47, Michael McMahon wrote: >> Could I get the following change reviewed please? >> >> The problem results from >> AbstractFileSystemProvider.getSunPathForSocketCall? returning a path >> of '.' when given an empty path argument. The socket API needs an >> empty path byte[] in this situation. It causes a BindException which >> is actually correct, but the error message is confusing because "." >> always exists and the error message is saying 'path already exists' >> when the actual error is that server socket channels cannot be bound >> to the empty path. > Can method description of getSunPathForSocketPath be updated to align > with the new behavior? Also can you fix the exception in > UnixDomainSocekts so that the casing is consistent with the other > exceptions. > Okay. > I don't think test/sun/nio/fs is the right place for the test. Would > it be better to create a test in test/jdk/java/nio/channels/unixdomain > to check that bind throws an exception? > The problem is that bind already throws a BindException, but for a different (confusing and incorrect) reason. I didn't really want to put a test in checking the exception text, and thought a white box test of the lower level method might be better. Thanks, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Nov 17 14:18:29 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 17 Nov 2020 14:18:29 +0000 Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' In-Reply-To: References: <98c2648f-13de-70d7-96a0-e8e5d5b25779@oracle.com> Message-ID: On 17/11/2020 14:05, Michael McMahon wrote: > > The problem is that bind already throws a BindException, but for a > different (confusing and incorrect) > reason. I didn't really want to put a test in checking the exception > text, and thought a white box test > of the lower level method might be better. > I would prefer if there were a test in the unixdomain directory for this. If there is no way to test this with the bind method then maybe you could move the test for getSunPathForSocketFile to that location. My main concern is that it complicates local testing when working on the file system API. The getSunPathForSocketFile is for Unix domain socket use only. -Alan From dfuchs at openjdk.java.net Tue Nov 17 14:47:11 2020 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Tue, 17 Nov 2020 14:47:11 GMT Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 13:42:46 GMT, Michael McMahon wrote: > Could I get the following change reviewed please? > > The problem results from AbstractFileSystemProvider.getSunPathForSocketCall returning a path of '.' when given an empty path argument. The socket API needs an empty path byte[] in this situation. It causes a BindException which is actually correct, but the error message is confusing because "." always exists and the error message is saying 'path already exists' when the actual error is that server socket channels cannot be bound to the empty path. > > Thanks, > Michael. src/java.base/windows/classes/sun/nio/fs/WindowsFileSystemProvider.java line 635: > 633: } > 634: String s = file.toString(); > 635: return s.getBytes(StandardCharsets.UTF_8); I'd suggest to refactor: WindowsPath file = WindowsPath.toWindowsPath(obj); String s = file.toString(); return s.isEmpty() ? emptyPath : s.getBytes(StandardCharsets.UTF_8); ------------- PR: https://git.openjdk.java.net/jdk/pull/1258 From michaelm at openjdk.java.net Tue Nov 17 15:06:02 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Tue, 17 Nov 2020 15:06:02 GMT Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 13:42:46 GMT, Michael McMahon wrote: > Could I get the following change reviewed please? > > The problem results from AbstractFileSystemProvider.getSunPathForSocketCall returning a path of '.' when given an empty path argument. The socket API needs an empty path byte[] in this situation. It causes a BindException which is actually correct, but the error message is confusing because "." always exists and the error message is saying 'path already exists' when the actual error is that server socket channels cannot be bound to the empty path. > > Thanks, > Michael. > _Mailing list message from [Alan Bateman](mailto:Alan.Bateman at oracle.com) on [nio-dev](mailto:nio-dev at openjdk.java.net):_ > > On 17/11/2020 14:05, Michael McMahon wrote: > > > The problem is that bind already throws a BindException, but for a > > different (confusing and incorrect) > > reason. I didn't really want to put a test in checking the exception > > text, and thought a white box test > > of the lower level method might be better. > > I would prefer if there were a test in the unixdomain directory for > this. If there is no way to test this with the bind method then maybe > you could move the test for getSunPathForSocketFile to that location. My > main concern is that it complicates local testing when working on the > file system API. The getSunPathForSocketFile is for Unix domain socket > use only. > > -Alan Okay, I'll move the test then. - Michael ------------- PR: https://git.openjdk.java.net/jdk/pull/1258 From michaelm at openjdk.java.net Tue Nov 17 15:06:06 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Tue, 17 Nov 2020 15:06:06 GMT Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 14:36:14 GMT, Daniel Fuchs wrote: >> Could I get the following change reviewed please? >> >> The problem results from AbstractFileSystemProvider.getSunPathForSocketCall returning a path of '.' when given an empty path argument. The socket API needs an empty path byte[] in this situation. It causes a BindException which is actually correct, but the error message is confusing because "." always exists and the error message is saying 'path already exists' when the actual error is that server socket channels cannot be bound to the empty path. >> >> Thanks, >> Michael. > > src/java.base/windows/classes/sun/nio/fs/WindowsFileSystemProvider.java line 635: > >> 633: } >> 634: String s = file.toString(); >> 635: return s.getBytes(StandardCharsets.UTF_8); > > I'd suggest to refactor: > > WindowsPath file = WindowsPath.toWindowsPath(obj); > String s = file.toString(); > return s.isEmpty() ? emptyPath : s.getBytes(StandardCharsets.UTF_8); Good idea. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/1258 From alanb at openjdk.java.net Tue Nov 17 15:15:09 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 17 Nov 2020 15:15:09 GMT Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' In-Reply-To: References: Message-ID: <1MIwgY4YZaqjs4cb5jVZ95v7aVoZMjO6F6bNS7ymK2Y=.3befb846-f83b-4df5-89b5-7a5e15b6e789@github.com> On Tue, 17 Nov 2020 13:42:46 GMT, Michael McMahon wrote: > Could I get the following change reviewed please? > > The problem results from AbstractFileSystemProvider.getSunPathForSocketCall returning a path of '.' when given an empty path argument. The socket API needs an empty path byte[] in this situation. It causes a BindException which is actually correct, but the error message is confusing because "." always exists and the error message is saying 'path already exists' when the actual error is that server socket channels cannot be bound to the empty path. > > Thanks, > Michael. src/java.base/unix/classes/sun/nio/fs/UnixFileSystemProvider.java line 51: > 49: { > 50: private static final String USER_DIR = "user.dir"; > 51: private static final byte[] emptyPath = new byte[0]; EMPTY_PATH might be better here. ------------- PR: https://git.openjdk.java.net/jdk/pull/1258 From michaelm at openjdk.java.net Tue Nov 17 15:52:20 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Tue, 17 Nov 2020 15:52:20 GMT Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' [v2] In-Reply-To: References: Message-ID: > Could I get the following change reviewed please? > > The problem results from AbstractFileSystemProvider.getSunPathForSocketCall returning a path of '.' when given an empty path argument. The socket API needs an empty path byte[] in this situation. It causes a BindException which is actually correct, but the error message is confusing because "." always exists and the error message is saying 'path already exists' when the actual error is that server socket channels cannot be bound to the empty path. > > Thanks, > Michael. Michael McMahon has updated the pull request incrementally with one additional commit since the last revision: update after Alan and Daniel's comments Tues 17th ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1258/files - new: https://git.openjdk.java.net/jdk/pull/1258/files/3706cc3c..30980aca Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1258&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1258&range=00-01 Stats: 8 lines in 5 files changed: 1 ins; 3 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/1258.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1258/head:pull/1258 PR: https://git.openjdk.java.net/jdk/pull/1258 From dfuchs at openjdk.java.net Tue Nov 17 17:34:03 2020 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Tue, 17 Nov 2020 17:34:03 GMT Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' [v2] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 15:52:20 GMT, Michael McMahon wrote: >> Could I get the following change reviewed please? >> >> The problem results from AbstractFileSystemProvider.getSunPathForSocketCall returning a path of '.' when given an empty path argument. The socket API needs an empty path byte[] in this situation. It causes a BindException which is actually correct, but the error message is confusing because "." always exists and the error message is saying 'path already exists' when the actual error is that server socket channels cannot be bound to the empty path. >> >> Thanks, >> Michael. > > Michael McMahon has updated the pull request incrementally with one additional commit since the last revision: > > update after Alan and Daniel's comments Tues 17th Marked as reviewed by dfuchs (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1258 From alanb at openjdk.java.net Tue Nov 17 18:02:13 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 17 Nov 2020 18:02:13 GMT Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' [v2] In-Reply-To: References: Message-ID: <5Yl7trEggzzzJUfG2j49dkHOChkMKnb3yF_6itM32oQ=.2f2e3eb6-fe35-4a70-9b9e-06556c565be7@github.com> On Tue, 17 Nov 2020 15:52:20 GMT, Michael McMahon wrote: >> Could I get the following change reviewed please? >> >> The problem results from AbstractFileSystemProvider.getSunPathForSocketCall returning a path of '.' when given an empty path argument. The socket API needs an empty path byte[] in this situation. It causes a BindException which is actually correct, but the error message is confusing because "." always exists and the error message is saying 'path already exists' when the actual error is that server socket channels cannot be bound to the empty path. >> >> Thanks, >> Michael. > > Michael McMahon has updated the pull request incrementally with one additional commit since the last revision: > > update after Alan and Daniel's comments Tues 17th src/java.base/share/classes/sun/nio/fs/AbstractFileSystemProvider.java line 161: > 159: * Returns a path name as bytes for a Unix domain socket. > 160: * Different encodings may be used for these names on some platforms. > 161: * If file is empty, then an empty byte[] is returned. This should be mis-read as "empty file" when you mean "empty path". src/java.base/windows/classes/sun/nio/fs/WindowsFileSystemProvider.java line 50: > 48: { > 49: private static final Unsafe unsafe = Unsafe.getUnsafe(); > 50: private static final byte[] emptyPath = new byte[0]; Is "unsafe" used, maybe this could be removed while you are there. Probably should be EMPTY_PATH to be consistent with the name in UnixFileSystemProvider. ------------- PR: https://git.openjdk.java.net/jdk/pull/1258 From michaelm at openjdk.java.net Tue Nov 17 20:13:19 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Tue, 17 Nov 2020 20:13:19 GMT Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' [v3] In-Reply-To: References: Message-ID: > Could I get the following change reviewed please? > > The problem results from AbstractFileSystemProvider.getSunPathForSocketCall returning a path of '.' when given an empty path argument. The socket API needs an empty path byte[] in this situation. It causes a BindException which is actually correct, but the error message is confusing because "." always exists and the error message is saying 'path already exists' when the actual error is that server socket channels cannot be bound to the empty path. > > Thanks, > Michael. Michael McMahon has updated the pull request incrementally with one additional commit since the last revision: update Tue evening ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1258/files - new: https://git.openjdk.java.net/jdk/pull/1258/files/30980aca..14169162 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1258&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1258&range=01-02 Stats: 5 lines in 2 files changed: 0 ins; 1 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/1258.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1258/head:pull/1258 PR: https://git.openjdk.java.net/jdk/pull/1258 From michaelm at openjdk.java.net Tue Nov 17 20:13:20 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Tue, 17 Nov 2020 20:13:20 GMT Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' [v2] In-Reply-To: <5Yl7trEggzzzJUfG2j49dkHOChkMKnb3yF_6itM32oQ=.2f2e3eb6-fe35-4a70-9b9e-06556c565be7@github.com> References: <5Yl7trEggzzzJUfG2j49dkHOChkMKnb3yF_6itM32oQ=.2f2e3eb6-fe35-4a70-9b9e-06556c565be7@github.com> Message-ID: <3RcFX17lutf3W8DDohTeFS_V58myre6bx3dAntXv0HQ=.1164281e-1524-4ecd-858c-3ccb4efa6f2f@github.com> On Tue, 17 Nov 2020 17:57:36 GMT, Alan Bateman wrote: >> Michael McMahon has updated the pull request incrementally with one additional commit since the last revision: >> >> update after Alan and Daniel's comments Tues 17th > > src/java.base/share/classes/sun/nio/fs/AbstractFileSystemProvider.java line 161: > >> 159: * Returns a path name as bytes for a Unix domain socket. >> 160: * Different encodings may be used for these names on some platforms. >> 161: * If file is empty, then an empty byte[] is returned. > > This should be mis-read as "empty file" when you mean "empty path". Thanks. Have updated after last two comments. ------------- PR: https://git.openjdk.java.net/jdk/pull/1258 From alanb at openjdk.java.net Wed Nov 18 07:36:03 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 18 Nov 2020 07:36:03 GMT Subject: RFR: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' [v3] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 20:13:19 GMT, Michael McMahon wrote: >> Could I get the following change reviewed please? >> >> The problem results from AbstractFileSystemProvider.getSunPathForSocketCall returning a path of '.' when given an empty path argument. The socket API needs an empty path byte[] in this situation. It causes a BindException which is actually correct, but the error message is confusing because "." always exists and the error message is saying 'path already exists' when the actual error is that server socket channels cannot be bound to the empty path. >> >> Thanks, >> Michael. > > Michael McMahon has updated the pull request incrementally with one additional commit since the last revision: > > update Tue evening Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1258 From michaelm at openjdk.java.net Wed Nov 18 08:47:04 2020 From: michaelm at openjdk.java.net (Michael McMahon) Date: Wed, 18 Nov 2020 08:47:04 GMT Subject: Integrated: 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 13:42:46 GMT, Michael McMahon wrote: > Could I get the following change reviewed please? > > The problem results from AbstractFileSystemProvider.getSunPathForSocketCall returning a path of '.' when given an empty path argument. The socket API needs an empty path byte[] in this situation. It causes a BindException which is actually correct, but the error message is confusing because "." always exists and the error message is saying 'path already exists' when the actual error is that server socket channels cannot be bound to the empty path. > > Thanks, > Michael. This pull request has now been integrated. Changeset: 6948456d Author: Michael McMahon URL: https://git.openjdk.java.net/jdk/commit/6948456d Stats: 60 lines in 5 files changed: 57 ins; 0 del; 3 mod 8256461: AbstractFileSystemProvider.getSunPathForSocketCall for empty Path returns '.' Reviewed-by: dfuchs, alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/1258 From chegar at openjdk.java.net Wed Nov 25 12:50:01 2020 From: chegar at openjdk.java.net (Chris Hegarty) Date: Wed, 25 Nov 2020 12:50:01 GMT Subject: RFR: 8257074 Update the ByteBuffers micro benchmark Message-ID: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> The ByteBuffers micro benchmark seems to be a little dated. It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations. This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case: 1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed. ------------- Commit messages: - Initial changes Changes: https://git.openjdk.java.net/jdk/pull/1430/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8257074 Stats: 69 lines in 1 file changed: 22 ins; 14 del; 33 mod Patch: https://git.openjdk.java.net/jdk/pull/1430.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1430/head:pull/1430 PR: https://git.openjdk.java.net/jdk/pull/1430 From redestad at openjdk.java.net Wed Nov 25 13:14:56 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Wed, 25 Nov 2020 13:14:56 GMT Subject: RFR: 8257074 Update the ByteBuffers micro benchmark In-Reply-To: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> References: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> Message-ID: On Wed, 25 Nov 2020 12:41:40 GMT, Chris Hegarty wrote: > The ByteBuffers micro benchmark seems to be a little dated. > > It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations. > > This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case: > > 1. Remove allocation from the individual benchmarks - it just creates noise. > 2. Consolidate per-thread shared heap and direct buffers. > 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. > 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. > 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed. Overall a reasonable clean-up. I do wonder if there's some value to at least some of these noisy, allocating variants, though. Could be interesting to zoom in on code where we allocate transient, small buffers, e.g. when encoding/decoding strings. But perhaps that needs to be designed differently and focus on a specific use case. ------------- Marked as reviewed by redestad (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1430 From dfuchs at openjdk.java.net Wed Nov 25 13:25:59 2020 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Wed, 25 Nov 2020 13:25:59 GMT Subject: RFR: 8257074 Update the ByteBuffers micro benchmark In-Reply-To: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> References: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> Message-ID: On Wed, 25 Nov 2020 12:41:40 GMT, Chris Hegarty wrote: > The ByteBuffers micro benchmark seems to be a little dated. > > It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations. > > This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case: > > 1. Remove allocation from the individual benchmarks - it just creates noise. > 2. Consolidate per-thread shared heap and direct buffers. > 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. > 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. > 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed. I am not an expert in JMH microbenchmarks but the proposed changes look reasonable to me. ------------- Marked as reviewed by dfuchs (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1430 From jvernee at openjdk.java.net Wed Nov 25 13:36:03 2020 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Wed, 25 Nov 2020 13:36:03 GMT Subject: RFR: 8257074 Update the ByteBuffers micro benchmark In-Reply-To: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> References: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> Message-ID: On Wed, 25 Nov 2020 12:41:40 GMT, Chris Hegarty wrote: > The ByteBuffers micro benchmark seems to be a little dated. > > It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations. > > This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case: > > 1. Remove allocation from the individual benchmarks - it just creates noise. > 2. Consolidate per-thread shared heap and direct buffers. > 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. > 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. > 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed. Marked as reviewed by jvernee (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1430 From chegar at openjdk.java.net Wed Nov 25 13:36:04 2020 From: chegar at openjdk.java.net (Chris Hegarty) Date: Wed, 25 Nov 2020 13:36:04 GMT Subject: RFR: 8257074 Update the ByteBuffers micro benchmark In-Reply-To: References: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> Message-ID: On Wed, 25 Nov 2020 13:12:34 GMT, Claes Redestad wrote: > I do wonder if there's some value to at least some of these noisy, allocating variants, though. Could be interesting to zoom in on code where we allocate transient, small buffers, e.g. when encoding/decoding strings. But perhaps that needs to be designed differently and focus on a specific use case. Yes, that is an interest case, but largely orthogonal to the primary use-case for this particular micro benchmark (or maybe; the primary use-case that I want to use this benchmark for ;-) ). It appears to me that the primary purpose is to evaluate the memory access operations, so I want to keep buffer allocation out of the picture. Of course, we could decide to just create a new benchmark, but then I'm not sure what we'd use the old one for. ------------- PR: https://git.openjdk.java.net/jdk/pull/1430 From mcimadamore at openjdk.java.net Wed Nov 25 13:41:00 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 25 Nov 2020 13:41:00 GMT Subject: RFR: 8257074 Update the ByteBuffers micro benchmark In-Reply-To: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> References: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> Message-ID: On Wed, 25 Nov 2020 12:41:40 GMT, Chris Hegarty wrote: > The ByteBuffers micro benchmark seems to be a little dated. > > It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations. > > This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case: > > 1. Remove allocation from the individual benchmarks - it just creates noise. > 2. Consolidate per-thread shared heap and direct buffers. > 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. > 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. > 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed. Looks like a good cleanup. I agree that mixing access and instantiation is not good from a benchmark perspective - although, given that direct buffers have a rather convoluted allocation process, I guess it would be useful, in the long term, to track performances of that too (this can be done by coupling direct buffer allocation with calls to Unsafe::invokeCleaner, so that the benchmark is not affected by the GC activity). Also, given a carrier e.g. `float` there are at least two ways to get a float from a byte array: * ByteBuffer::getFloat (both direct/heap) * FloatBuffer::get() (both direct/heap) The benchmark here focuses on the first bullet, but I think at some point we should also cover the remaining axis, so that, for each carrier, we really do at least 4 tests. Of course if you take into account swappiness for non-byte carriers, you get an extra variant for direct buffers as well, as you wanna test native endian vs. non-native endian. So many possibilities :-) ------------- Marked as reviewed by mcimadamore (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1430 From chegar at openjdk.java.net Wed Nov 25 13:49:06 2020 From: chegar at openjdk.java.net (Chris Hegarty) Date: Wed, 25 Nov 2020 13:49:06 GMT Subject: RFR: 8257074 Update the ByteBuffers micro benchmark In-Reply-To: References: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> Message-ID: On Wed, 25 Nov 2020 13:37:48 GMT, Maurizio Cimadamore wrote: > Looks like a good cleanup. I agree that mixing access and instantiation is not good from a benchmark perspective - although, given that direct buffers have a rather convoluted allocation process, I guess it would be useful, in the long term, to track performances of that too (this can be done by coupling direct buffer allocation with calls to Unsafe::invokeCleaner, so that the benchmark is not affected by the GC activity). I'll see if I can write a test for this. > Also, given a carrier e.g. `float` there are at least two ways to get a float from a byte array: > > * ByteBuffer::getFloat (both direct/heap) > * FloatBuffer::get() (both direct/heap) > > The benchmark here focuses on the first bullet, but I think at some point we should also cover the remaining axis, so that, for each carrier, we really do at least 4 tests. Of course if you take into account swappiness for non-byte carriers, you get an extra variant for direct buffers as well, as you wanna test native endian vs. non-native endian. Yes, good observation - I had similar thoughts. Let me check if we could easily integrate some of these variants. What I'm trying to end up with is a comprehensive(ish) set of test scenarios that can be used to analyse performance ( of which views and endianness are important). It may be reasonable for this particular benchmark to include views, etc, with appropriately named benchmark methods, so that one can easily run a subset, e.g. "org.openjdk.bench.java.nio.ByteBuffers.test(Direct)?(BulkPut|SinglePutByte)", when trying to "zoom in" on a specific code path. ------------- PR: https://git.openjdk.java.net/jdk/pull/1430 From janirutec at gmail.com Wed Nov 25 21:16:38 2020 From: janirutec at gmail.com (JaniruTEC) Date: Wed, 25 Nov 2020 22:16:38 +0100 Subject: 8030048: (fs) Support UserDefinedFileAttributeView/extended attributes on OS X / HFS+ Message-ID: To whom it may concern, I was tasked by my employer to fix issue 8030048 and contribute an implementation of the XAttr API for MacOS to the JDK. As outlined in the JDK contribution guidelines this is my request for the permission to contribute this change and my query for the current state of development if there is any. Please excuse any missing information as this is my first time contributing to the JDK. If there are any questions, please don't hesitate to contact me. Kind regards, JaniruTEC From Alan.Bateman at oracle.com Thu Nov 26 07:35:33 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 26 Nov 2020 07:35:33 +0000 Subject: 8030048: (fs) Support UserDefinedFileAttributeView/extended attributes on OS X / HFS+ In-Reply-To: References: Message-ID: <5946d00b-0534-6313-4a4b-05a1a3330aff@oracle.com> On 25/11/2020 21:16, JaniruTEC wrote: > To whom it may concern, > > I was tasked by my employer to fix issue 8030048 and contribute an > implementation of the XAttr API for MacOS to the JDK. > As outlined in the JDK contribution guidelines this is my request for > the permission to contribute this change and my query for the current > state of development if there is any. > > Please excuse any missing information as this is my first time > contributing to the JDK. > If there are any questions, please don't hesitate to contact me. Adding support for this attribute view on macOS has been on the list for a long time but never bubbled to the top of anyone's list. So I don't think anyone has spent any time on it. One starting point is to look at the Linux implementation and see if it can be ported over. Discussion points will be whether the support is added to the BSD provider or the macOS provider, also details on the encoding of the attribute name. -Alan. From janirutec at gmail.com Thu Nov 26 08:39:09 2020 From: janirutec at gmail.com (Janiru Tec) Date: Thu, 26 Nov 2020 09:39:09 +0100 Subject: 8030048: (fs) Support UserDefinedFileAttributeView/extended attributes on OS X / HFS+ In-Reply-To: <5946d00b-0534-6313-4a4b-05a1a3330aff@oracle.com> References: <5946d00b-0534-6313-4a4b-05a1a3330aff@oracle.com> Message-ID: >From my perspective the Linux implementation can be ported or at least be used for many parts. It heavily utilizes an "Unsafe"-class, tho. Is this a Problem in modern Implementations? As far as I can see the support itself is added using it's own class that is loaded by a provider, macOS would probably be the best pick for that. - Dominik > Adding support for this attribute view on macOS has been on the list for a long time but never bubbled to the top of anyone's list. So I don't think anyone has spent any time on it. One starting point is to look at the Linux implementation and see if it can be ported over. Discussion points will be whether the support is added to the BSD provider or the macOS provider, also details on the encoding of the attribute name. > > -Alan. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chegar at openjdk.java.net Thu Nov 26 14:34:12 2020 From: chegar at openjdk.java.net (Chris Hegarty) Date: Thu, 26 Nov 2020 14:34:12 GMT Subject: RFR: 8257074 Update the ByteBuffers micro benchmark [v2] In-Reply-To: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> References: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> Message-ID: > The ByteBuffers micro benchmark seems to be a little dated. > > It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations. > > This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case: > > 1. Remove allocation from the individual benchmarks - it just creates noise. > 2. Consolidate per-thread shared heap and direct buffers. > 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. > 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. > 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed. Chris Hegarty has updated the pull request incrementally with one additional commit since the last revision: Add additional carrier views and endianness variants. A large number of variants are now covered. The individual benchmarks conform to the following convention: test(Direct|Heap)(Bulk|Single)(Get|Put)(Byte|Char|Short|Int|Long|Float|Double)(View)?(Swap)? This allows to easily run a subset of particular interest. For example: Direct only :- "org.openjdk.bench.java.nio.ByteBuffers.testDirect.*" Char only :- "org.openjdk.bench.java.nio.ByteBuffers.test.*Char.*" Bulk only :- "org.openjdk.bench.java.nio.ByteBuffers.test.*Bulk.*" Put with Int or Long carrier :- test(Direct|Heap)(Single)(Put)(Int|Long)(View)?(Swap)?" Running all variants together is likely not all that useful, since there will be a lot of data. The param sizes are changed so as to better allow for wider carrier views. There are a lot of individual benchmark methods, but their implementation is trivial, and largely mechanical. Question: where do folk stand on having a `main` method in a benchmark - as a standalone-run sanity? I found this useful to assert the validity of the benchmark code. It can be commented out if it could somehow affect the benchmark runs. ( I omitted read-only views, since they less interesting, and we already have a lot of variants ) ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1430/files - new: https://git.openjdk.java.net/jdk/pull/1430/files/5e91e63e..84dabc30 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=00-01 Stats: 1022 lines in 1 file changed: 995 ins; 1 del; 26 mod Patch: https://git.openjdk.java.net/jdk/pull/1430.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1430/head:pull/1430 PR: https://git.openjdk.java.net/jdk/pull/1430 From chegar at openjdk.java.net Thu Nov 26 14:38:17 2020 From: chegar at openjdk.java.net (Chris Hegarty) Date: Thu, 26 Nov 2020 14:38:17 GMT Subject: RFR: 8257074 Update the ByteBuffers micro benchmark [v3] In-Reply-To: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> References: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> Message-ID: > The ByteBuffers micro benchmark seems to be a little dated. > > It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations. > > This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case: > > 1. Remove allocation from the individual benchmarks - it just creates noise. > 2. Consolidate per-thread shared heap and direct buffers. > 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. > 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. > 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed. Chris Hegarty has updated the pull request incrementally with one additional commit since the last revision: whitespace ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1430/files - new: https://git.openjdk.java.net/jdk/pull/1430/files/84dabc30..814e1819 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=01-02 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1430.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1430/head:pull/1430 PR: https://git.openjdk.java.net/jdk/pull/1430 From Alan.Bateman at oracle.com Thu Nov 26 17:01:31 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 26 Nov 2020 17:01:31 +0000 Subject: 8030048: (fs) Support UserDefinedFileAttributeView/extended attributes on OS X / HFS+ In-Reply-To: References: <5946d00b-0534-6313-4a4b-05a1a3330aff@oracle.com> Message-ID: <5c45f6bf-b27b-047b-2be7-9fa1f2a477b0@oracle.com> On 26/11/2020 08:39, Janiru Tec wrote: > From my perspective the Linux implementation can be ported or at least > be used for many parts. It heavily utilizes an "Unsafe"-class, tho. Is > this a Problem in modern Implementations? > As far as I can see the support itself is added using it's own class > that is loaded by a provider, macOS would probably be the best pick > for that. At this time, the attributes views that are supported is configured in BsdFileSystem but it's okay for MacOSXFileSystem to augment this. There isn't any issue using unsafe here but the interface to these APIs involved memory outside of the heap then we'll need to study it closely to make sure that it is robust. -Alan From mcimadamore at openjdk.java.net Fri Nov 27 18:04:00 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Fri, 27 Nov 2020 18:04:00 GMT Subject: RFR: 8257074 Update the ByteBuffers micro benchmark [v3] In-Reply-To: References: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> Message-ID: <3-P6Lc_CwEXGZIbLdnywWBTqtVbWxq08nA4HsdLp5jk=.b91cac90-ef2f-49bc-93c3-666fc8af192a@github.com> On Thu, 26 Nov 2020 14:38:17 GMT, Chris Hegarty wrote: >> The ByteBuffers micro benchmark seems to be a little dated. >> >> It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations. >> >> This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case: >> >> 1. Remove allocation from the individual benchmarks - it just creates noise. >> 2. Consolidate per-thread shared heap and direct buffers. >> 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. >> 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. >> 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed. > > Chris Hegarty has updated the pull request incrementally with one additional commit since the last revision: > > whitespace Looks a great improvements. Two comments: * the names always mention the "Single" word, when in fact all benchmark involve some kind of a loop. I'd suggest making that more explicit, both in the benchmark method and in the helpers. * I see that you added support for views - but note that, for heap buffers there are two different kind of views (!!). E.g. I believe that: ByteBuffer.allocate(size * 4).asFloatBuffer() returns a buffer of a different class than: FloatBuffer.allocate(size); I think you are only testing the former? ------------- PR: https://git.openjdk.java.net/jdk/pull/1430 From chris.hegarty at oracle.com Fri Nov 27 19:22:00 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 27 Nov 2020 19:22:00 +0000 Subject: RFR: 8257074 Update the ByteBuffers micro benchmark [v3] In-Reply-To: <3-P6Lc_CwEXGZIbLdnywWBTqtVbWxq08nA4HsdLp5jk=.b91cac90-ef2f-49bc-93c3-666fc8af192a@github.com> References: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> <3-P6Lc_CwEXGZIbLdnywWBTqtVbWxq08nA4HsdLp5jk=.b91cac90-ef2f-49bc-93c3-666fc8af192a@github.com> Message-ID: Maurizio, > On 27 Nov 2020, at 18:04, Maurizio Cimadamore wrote: > > ... > Looks a great improvements. Two comments: > > * the names always mention the "Single" word, when in fact all benchmark involve some kind of a loop. I'd suggest making that more explicit, both in the benchmark method and in the helpers. Yeah, I dislike (the preexisting) ?Single? too. How about ?Loop? - so we have [Bulk|Loop] ? > * I see that you added support for views - but note that, for heap buffers there are two different kind of views (!!). E.g. I believe that: > > ByteBuffer.allocate(size * 4).asFloatBuffer() > returns a buffer of a different class than: > > FloatBuffer.allocate(size); > I think you are only testing the former? Correct. I did consider the latter, but then thought that this micro had enough scenarios. Let me take another look and see how much more scenarios it adds. -Chris. -------------- next part -------------- An HTML attachment was scrubbed... URL: From chegar at openjdk.java.net Mon Nov 30 13:39:12 2020 From: chegar at openjdk.java.net (Chris Hegarty) Date: Mon, 30 Nov 2020 13:39:12 GMT Subject: RFR: 8257074 Update the ByteBuffers micro benchmark [v4] In-Reply-To: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> References: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> Message-ID: > The ByteBuffers micro benchmark seems to be a little dated. > > It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations. > > This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case: > > 1. Remove allocation from the individual benchmarks - it just creates noise. > 2. Consolidate per-thread shared heap and direct buffers. > 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. > 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. > 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed. Chris Hegarty has updated the pull request incrementally with two additional commits since the last revision: - Add explicitly allocated heap carrier buffer tests - Replace Single with Loop ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1430/files - new: https://git.openjdk.java.net/jdk/pull/1430/files/814e1819..5ee5d8bf Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=02-03 Stats: 407 lines in 1 file changed: 151 ins; 0 del; 256 mod Patch: https://git.openjdk.java.net/jdk/pull/1430.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1430/head:pull/1430 PR: https://git.openjdk.java.net/jdk/pull/1430 From bpb at openjdk.java.net Mon Nov 30 20:57:03 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 30 Nov 2020 20:57:03 GMT Subject: RFR: 8257074 Update the ByteBuffers micro benchmark [v4] In-Reply-To: References: <9w1Ut1qQWUq8nY6uSKt6Cp7nizh_eZOxo3FgSUTlExM=.09e6e0c6-e442-46bf-b313-8c6e6544aa8a@github.com> Message-ID: <4QlCNtToXuqjt-0lnwZgVwjkEr04wWmh_AYvQLOq2BA=.38b61f48-c633-488a-a9b0-8137346013c5@github.com> On Mon, 30 Nov 2020 13:39:12 GMT, Chris Hegarty wrote: >> The ByteBuffers micro benchmark seems to be a little dated. >> >> It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations. >> >> This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case: >> >> 1. Remove allocation from the individual benchmarks - it just creates noise. >> 2. Consolidate per-thread shared heap and direct buffers. >> 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. >> 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. >> 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed. > > Chris Hegarty has updated the pull request incrementally with two additional commits since the last revision: > > - Add explicitly allocated heap carrier buffer tests > - Replace Single with Loop Marked as reviewed by bpb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1430 From lancea at openjdk.java.net Mon Nov 30 22:09:08 2020 From: lancea at openjdk.java.net (Lance Andersen) Date: Mon, 30 Nov 2020 22:09:08 GMT Subject: RFR: 8257445: (zipfs) Add DataProvider to TestLocOffsetFromZip64EF.java Message-ID: HI all, Please review this trivial change which adds a DataProvider to the manual test TestLocOffsetFromZip64EF.java, added as part of the fix for https://bugs.openjdk.java.net/browse/JDK-8255380, to specify additional values for the Zip FS file System property zipinfo-time. Best, Lance ------------- Commit messages: - Minor update to DataProvider comment - Add DataProvider to further exercise zipinfo-time Zip FS file System property Changes: https://git.openjdk.java.net/jdk/pull/1527/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1527&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8257445 Stats: 21 lines in 1 file changed: 17 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/1527.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1527/head:pull/1527 PR: https://git.openjdk.java.net/jdk/pull/1527 From bpb at openjdk.java.net Mon Nov 30 22:27:54 2020 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 30 Nov 2020 22:27:54 GMT Subject: RFR: 8257445: (zipfs) Add DataProvider to TestLocOffsetFromZip64EF.java In-Reply-To: References: Message-ID: <3K1y6pSOPU0tDz1HIN_9vNkzHLh7BqLBeG6JqO5BL5o=.3358a02f-b2d0-44cd-af91-96e1c1b753c9@github.com> On Mon, 30 Nov 2020 21:59:23 GMT, Lance Andersen wrote: > HI all, > > Please review this trivial change which adds a DataProvider to the manual test TestLocOffsetFromZip64EF.java, added as part of the fix for https://bugs.openjdk.java.net/browse/JDK-8255380, to specify additional values for the Zip FS file System property zipinfo-time. > > Best, > Lance Marked as reviewed by bpb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1527