From duke at openjdk.java.net Wed Dec 1 08:23:32 2021 From: duke at openjdk.java.net (Markus KARG) Date: Wed, 1 Dec 2021 08:23:32 GMT Subject: RFR: 8276994: java/nio/channels/Channels/TransferTo.java leaves multi-GB files in /tmp [v3] In-Reply-To: References: <9ymWYkP8CxDCuRV7-U-KnlnOU3Up4_4kzILhBrYJpb0=.e5ce5b6b-db9f-4c24-9022-55009c943ad3@github.com> <1HeCPKTcBeozYrgRzJag_Vz9PB5MLZLItpxKgTNWwtc=.eba7421c-2a28-4a4a-86c2-556028803e3e@github.com> Message-ID: <-PGVg_FNbsjaPfqrvP-u63bpTTFmtVXIzZISn71lXjs=.4e2be409-db90-40d9-8deb-2a4716b16bf3@github.com> On Mon, 15 Nov 2021 17:58:21 GMT, Daniel Fuchs wrote: >> As I explained, this is not written due to a *need* but due to *deliberate separation of concerns*. So is separation of concerns a design strategy that is forbidden in OpenJDK? > > It's better to avoid using raw-types unless there's a specific reason to do so (I guess this is just an oversight). Yes this was just an oversight, and this thread got outdated due to Alan`s resolution. ------------- PR: https://git.openjdk.java.net/jdk/pull/6379 From duke at openjdk.java.net Wed Dec 1 08:24:31 2021 From: duke at openjdk.java.net (Markus KARG) Date: Wed, 1 Dec 2021 08:24:31 GMT Subject: RFR: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels [v18] In-Reply-To: References: Message-ID: On Mon, 29 Nov 2021 19:51:51 GMT, Lance Andersen wrote: > We are getting closer, the methods fileChannelInput/Output should also create the temp files in the working directory and include a prefix for name construction. This is exactly what https://github.com/openjdk/jdk/pull/6379 is good for. Please review *that* PR first. Once it got merged, I will rebase *this* PR ontop. ------------- PR: https://git.openjdk.java.net/jdk/pull/5179 From lancea at openjdk.java.net Wed Dec 1 11:59:31 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Wed, 1 Dec 2021 11:59:31 GMT Subject: RFR: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels [v18] In-Reply-To: References: Message-ID: On Sat, 27 Nov 2021 16:57:36 GMT, Markus KARG wrote: >> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. This PR is a spin-off from https://github.com/openjdk/jdk/pull/4263 and deliberately concentrates **only** on `FileChannel`s. Other types of channels will be discussed in future PRs. >> >> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer. >> >> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. A rather similar approach in different use casse was recently implemented by https://github.com/openjdk/jdk/pull/5097 which was reported by even provide 5x performance (https://github.com/openjdk/jdk/pull/5097#issuecomment-897271997). >> >> **Update:** This sub-issue *only* defines the work to be done to implement JDK-8265891 *solely* for the particular case of FileChannel.transferTo(WriteableByteChannel), including special treatment of SelectableByteChannel, as the `master` branch already contains code to handle the specific case of FileChannel.transferTo(FileChannel). > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > Creating temp file in current working directory > > Signed-off-by: Markus Karg > JDK-8276994 was resolved previously so I am not sure why you still have a PR open for that issue as you would not be able to integrate without a new bug. Please integrate any remaining changes into the test and we can finalize the review for you ------------- PR: https://git.openjdk.java.net/jdk/pull/5179 From bchristi at openjdk.java.net Wed Dec 1 18:21:04 2021 From: bchristi at openjdk.java.net (Brent Christian) Date: Wed, 1 Dec 2021 18:21:04 GMT Subject: RFR: JDK-8276447 Deprecate finalization-related methods for removal [v2] In-Reply-To: References: Message-ID: > Here are the code changes for the "Deprecate finalizers in the standard Java API" portion of JEP 421 ("Deprecate Finalization for Removal") for code review. > > This change makes the indicated deprecations, and updates the API spec for JEP 421. It also updates the relevant @SuppressWarning annotations. > > The CSR has been approved. > An automated test build+test run passes cleanly (FWIW :D ). Brent Christian has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 32 commits: - Merge branch 'master' into 8276447 - merge - @SuppressWarnings(removal) in Windows CKey.java - Add jls tag to runFinalization methods - Update wording of Object.finalize, new paragraph is now bold - update Object.finalize javadoc - update Object.finalize JavaDoc and @deprecated tag - Update Object.finalize deprecation message - Indicate that runFinalizers does nothing if finalization is disabled or removed - Fix @since on @Deprecated for java.lang.Enum - ... and 22 more: https://git.openjdk.java.net/jdk/compare/a363b7b9...e4986a48 ------------- Changes: https://git.openjdk.java.net/jdk/pull/6465/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6465&range=01 Stats: 194 lines in 62 files changed: 54 ins; 38 del; 102 mod Patch: https://git.openjdk.java.net/jdk/pull/6465.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6465/head:pull/6465 PR: https://git.openjdk.java.net/jdk/pull/6465 From alanb at openjdk.java.net Wed Dec 1 18:33:26 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 1 Dec 2021 18:33:26 GMT Subject: RFR: 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) [v4] In-Reply-To: References: Message-ID: <1h1e45LCDCzmVX56dkxs1uHm-vs2xs5Le2BlkBOsi3k=.5cac53e7-ded2-45dd-a939-1208e291bee8@github.com> On Fri, 19 Nov 2021 17:32:34 GMT, Brian Burkhalter wrote: >> Please consider this change which uses a new method `WindowsPath::getPathWithPrefixForWin32Calls`. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8276661: addPrefixIfNeeded calls addPrefix; remove UNC test src/java.base/windows/classes/sun/nio/fs/WindowsPath.java line 187: > 185: // short absolute paths can be used directly > 186: if (isAbsolute() && path.length() <= MAX_PATH) > 187: return forceLongPrefix ? addPrefix(path) : path; Brian and I discussed this issue today and the reason this version didn't work is that it prepends the long path prefix without first using GetFullPathName to resolve the "." and ".." elements in the name. Brian is working on a new version now that will fix this issue and we should end up with a cleaner solution. ------------- PR: https://git.openjdk.java.net/jdk/pull/6435 From bchristi at openjdk.java.net Wed Dec 1 19:23:59 2021 From: bchristi at openjdk.java.net (Brent Christian) Date: Wed, 1 Dec 2021 19:23:59 GMT Subject: RFR: JDK-8276447 Deprecate finalization-related methods for removal [v3] In-Reply-To: References: Message-ID: > Here are the code changes for the "Deprecate finalizers in the standard Java API" portion of JEP 421 ("Deprecate Finalization for Removal") for code review. > > This change makes the indicated deprecations, and updates the API spec for JEP 421. It also updates the relevant @SuppressWarning annotations. > > The CSR has been approved. > An automated test build+test run passes cleanly (FWIW :D ). Brent Christian has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 33 commits: - Merge branch 'master' into 8276447 - Merge branch 'master' into 8276447 - merge - @SuppressWarnings(removal) in Windows CKey.java - Add jls tag to runFinalization methods - Update wording of Object.finalize, new paragraph is now bold - update Object.finalize javadoc - update Object.finalize JavaDoc and @deprecated tag - Update Object.finalize deprecation message - Indicate that runFinalizers does nothing if finalization is disabled or removed - ... and 23 more: https://git.openjdk.java.net/jdk/compare/0dfb3a70...8cde0674 ------------- Changes: https://git.openjdk.java.net/jdk/pull/6465/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6465&range=02 Stats: 194 lines in 62 files changed: 54 ins; 38 del; 102 mod Patch: https://git.openjdk.java.net/jdk/pull/6465.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6465/head:pull/6465 PR: https://git.openjdk.java.net/jdk/pull/6465 From bpb at openjdk.java.net Wed Dec 1 20:55:51 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 1 Dec 2021 20:55:51 GMT Subject: RFR: 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) [v6] In-Reply-To: References: Message-ID: <-HMdXCzb6ExaOdMfD5Lofo8FYYMw5Yl8raItFzViW48=.3b812e68-3bbf-41fe-8145-0249361cb33a@github.com> > Please consider this change which uses a new method `WindowsPath::getPathWithPrefixForWin32Calls`. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8276661: Ensure GetFullPathName() is used ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6435/files - new: https://git.openjdk.java.net/jdk/pull/6435/files/5703b012..3567e5ac Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6435&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6435&range=04-05 Stats: 13 lines in 2 files changed: 0 ins; 4 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/6435.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6435/head:pull/6435 PR: https://git.openjdk.java.net/jdk/pull/6435 From bpb at openjdk.java.net Wed Dec 1 23:48:59 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 1 Dec 2021 23:48:59 GMT Subject: RFR: 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) [v7] In-Reply-To: References: Message-ID: <3iAwD3rg_oFZIbBOPMgcuepbZr8s7dustYMk0Fhw7to=.7cbc428a-d6a2-4139-97d9-20cbb554d7a7@github.com> > Please consider this change which uses a new method `WindowsPath::getPathWithPrefixForWin32Calls`. Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: - Merge - 8276661: Ensure GetFullPathName() is used - 8276661: Change getPathWithPrefixForWin32Calls to getPathWithPadForWin32Calls - 8276661: addPrefixIfNeeded calls addPrefix; remove UNC test - 8276661: Replace addPrefixIfAbsent() with addPrefix() - 8276661: Support UNC paths - 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6435/files - new: https://git.openjdk.java.net/jdk/pull/6435/files/3567e5ac..9f51bd7a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6435&range=06 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6435&range=05-06 Stats: 75632 lines in 1379 files changed: 50644 ins; 14508 del; 10480 mod Patch: https://git.openjdk.java.net/jdk/pull/6435.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6435/head:pull/6435 PR: https://git.openjdk.java.net/jdk/pull/6435 From bpb at openjdk.java.net Wed Dec 1 23:49:01 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 1 Dec 2021 23:49:01 GMT Subject: RFR: 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) [v6] In-Reply-To: <-HMdXCzb6ExaOdMfD5Lofo8FYYMw5Yl8raItFzViW48=.3b812e68-3bbf-41fe-8145-0249361cb33a@github.com> References: <-HMdXCzb6ExaOdMfD5Lofo8FYYMw5Yl8raItFzViW48=.3b812e68-3bbf-41fe-8145-0249361cb33a@github.com> Message-ID: On Wed, 1 Dec 2021 20:55:51 GMT, Brian Burkhalter wrote: >> Please consider this change which uses a new method `WindowsPath::getPathWithPrefixForWin32Calls`. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8276661: Ensure GetFullPathName() is used Merged code passes tiers 1-3 tests in the CI. ------------- PR: https://git.openjdk.java.net/jdk/pull/6435 From bpb at openjdk.java.net Wed Dec 1 23:49:07 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 1 Dec 2021 23:49:07 GMT Subject: RFR: 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) [v4] In-Reply-To: <1h1e45LCDCzmVX56dkxs1uHm-vs2xs5Le2BlkBOsi3k=.5cac53e7-ded2-45dd-a939-1208e291bee8@github.com> References: <1h1e45LCDCzmVX56dkxs1uHm-vs2xs5Le2BlkBOsi3k=.5cac53e7-ded2-45dd-a939-1208e291bee8@github.com> Message-ID: <0KuiYQMRwRj1BkVMUtZU0hVGhCI22_hcKgw1OE5MgNg=.c613abe7-895d-4f21-85d1-7c21110b7734@github.com> On Wed, 1 Dec 2021 18:30:01 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8276661: addPrefixIfNeeded calls addPrefix; remove UNC test > > src/java.base/windows/classes/sun/nio/fs/WindowsPath.java line 187: > >> 185: // short absolute paths can be used directly >> 186: if (isAbsolute() && path.length() <= MAX_PATH) >> 187: return forceLongPrefix ? addPrefix(path) : path; > > Brian and I discussed this issue today and the reason this version didn't work is that it prepends the long path prefix without first using GetFullPathName to resolve the "." and ".." elements in the name. Brian is working on a new version now that will fix this issue and we should end up with a cleaner solution. Committed. ------------- PR: https://git.openjdk.java.net/jdk/pull/6435 From smarks at openjdk.java.net Thu Dec 2 01:26:30 2021 From: smarks at openjdk.java.net (Stuart Marks) Date: Thu, 2 Dec 2021 01:26:30 GMT Subject: RFR: JDK-8276447 Deprecate finalization-related methods for removal [v3] In-Reply-To: References: Message-ID: On Wed, 1 Dec 2021 19:23:59 GMT, Brent Christian wrote: >> Here are the code changes for the "Deprecate finalizers in the standard Java API" portion of JEP 421 ("Deprecate Finalization for Removal") for code review. >> >> This change makes the indicated deprecations, and updates the API spec for JEP 421. It also updates the relevant @SuppressWarning annotations. >> >> The CSR has been approved. >> An automated test build+test run passes cleanly (FWIW :D ). > > Brent Christian has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 33 commits: > > - Merge branch 'master' into 8276447 > - Merge branch 'master' into 8276447 > - merge > - @SuppressWarnings(removal) in Windows CKey.java > - Add jls tag to runFinalization methods > - Update wording of Object.finalize, new paragraph is now bold > - update Object.finalize javadoc > - update Object.finalize JavaDoc and @deprecated tag > - Update Object.finalize deprecation message > - Indicate that runFinalizers does nothing if finalization is disabled or removed > - ... and 23 more: https://git.openjdk.java.net/jdk/compare/0dfb3a70...8cde0674 Marked as reviewed by smarks (Reviewer). src/jdk.jconsole/share/classes/sun/tools/jconsole/SummaryTab.java line 116: > 114: StringBuilder buf; > 115: > 116: @SuppressWarnings("deprecation") Item for future cleanup: refactor the call to `getObjectPendingFinalizationCount()` at line 219 (!) into a local variable declaration, and then move the warnings suppression to that declaration. This reduces the scope of warnings suppression. ------------- PR: https://git.openjdk.java.net/jdk/pull/6465 From alanb at openjdk.java.net Thu Dec 2 12:15:29 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 2 Dec 2021 12:15:29 GMT Subject: RFR: 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) [v7] In-Reply-To: <3iAwD3rg_oFZIbBOPMgcuepbZr8s7dustYMk0Fhw7to=.7cbc428a-d6a2-4139-97d9-20cbb554d7a7@github.com> References: <3iAwD3rg_oFZIbBOPMgcuepbZr8s7dustYMk0Fhw7to=.7cbc428a-d6a2-4139-97d9-20cbb554d7a7@github.com> Message-ID: On Wed, 1 Dec 2021 23:48:59 GMT, Brian Burkhalter wrote: >> Please consider this change which uses a new method `WindowsPath::getPathWithPrefixForWin32Calls`. > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: > > - Merge > - 8276661: Ensure GetFullPathName() is used > - 8276661: Change getPathWithPrefixForWin32Calls to getPathWithPadForWin32Calls > - 8276661: addPrefixIfNeeded calls addPrefix; remove UNC test > - 8276661: Replace addPrefixIfAbsent() with addPrefix() > - 8276661: Support UNC paths > - 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) The latest revision looks good and I think we were lucky that the CheckPermissions test caught the issue. One suggestion is to invert forceLongPrefix to something like allowShortPath. I think this would make getPathForWin32Calls a bit easier to read because because it avoid two uses of !forceLongPrefix. ------------- PR: https://git.openjdk.java.net/jdk/pull/6435 From duke at openjdk.java.net Thu Dec 2 16:57:59 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 2 Dec 2021 16:57:59 GMT Subject: RFR: 8276994: java/nio/channels/Channels/TransferTo.java leaves multi-GB files in /tmp In-Reply-To: References: Message-ID: On Mon, 15 Nov 2021 07:39:09 GMT, Alan Bateman wrote: >> I don't think we need this. Instead, it would be better if the tests uses the test work directory so the files are easily found for post failure analysis. This isn't practical for the tests that create >2GB files so I think leave the code that Lance put in there to delete them so they aren't archived when the work directory is zipped up. > >> @AlanBateman @LanceAndersen You are confusing me. Lance requests style changes, while Alan does not want the changes. These answers are incompatible. _Either_ I change the code according Lance's wishes, _or_ we simple drop this PR. Please find a concensus. > > I think it's better to pause a moment and understand the environment when jtreg runs tests. The important thing is that the current directory (as in user.dir) will be a scratch directory that jtreg has created. My comments are just pointing out that the bookkeeping proposed in this PR to delete files is not needed that you can just change the test to create the temp file in the current directory. We can leave the try-finally that Lance added in the >2GB tests because they are too big to keep for any post failure analysis. @AlanBateman @LanceAndersen I kindly request your review on this PR. ------------- PR: https://git.openjdk.java.net/jdk/pull/6379 From duke at openjdk.java.net Thu Dec 2 16:58:03 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 2 Dec 2021 16:58:03 GMT Subject: RFR: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels [v18] In-Reply-To: References: Message-ID: On Wed, 1 Dec 2021 11:56:19 GMT, Lance Andersen wrote: > JDK-8276994 was resolved previously so I am not sure why you still have a PR open for that issue as you would not be able to integrate without a new bug. Understood. I did not find a public rule that says that there cannot be multiple PRs referencing the same JIRA issue. > Please integrate any remaining changes into the test and we can finalize the review for you Ok so I will merge both PRs into one. ------------- PR: https://git.openjdk.java.net/jdk/pull/5179 From bpb at openjdk.java.net Thu Dec 2 17:51:59 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 2 Dec 2021 17:51:59 GMT Subject: RFR: 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) [v8] In-Reply-To: References: Message-ID: > Please consider this change which uses a new method `WindowsPath::getPathWithPrefixForWin32Calls`. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8276661: s/\!forceLongPrefix/allowShortPath/ ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6435/files - new: https://git.openjdk.java.net/jdk/pull/6435/files/9f51bd7a..6dd9f3ae Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6435&range=07 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6435&range=06-07 Stats: 6 lines in 1 file changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/6435.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6435/head:pull/6435 PR: https://git.openjdk.java.net/jdk/pull/6435 From alanb at openjdk.java.net Thu Dec 2 18:20:18 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 2 Dec 2021 18:20:18 GMT Subject: RFR: 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) [v8] In-Reply-To: References: Message-ID: On Thu, 2 Dec 2021 17:51:59 GMT, Brian Burkhalter wrote: >> Please consider this change which uses a new method `WindowsPath::getPathWithPrefixForWin32Calls`. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8276661: s/\!forceLongPrefix/allowShortPath/ Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6435 From prappo at openjdk.java.net Thu Dec 2 20:54:28 2021 From: prappo at openjdk.java.net (Pavel Rappo) Date: Thu, 2 Dec 2021 20:54:28 GMT Subject: RFR: 8278168: Add a few missing words to the specification of Files.mismatch Message-ID: Please review this trivial fix. ------------- Commit messages: - Initial commit Changes: https://git.openjdk.java.net/jdk/pull/6679/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6679&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8278168 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/6679.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6679/head:pull/6679 PR: https://git.openjdk.java.net/jdk/pull/6679 From bpb at openjdk.java.net Thu Dec 2 20:56:18 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 2 Dec 2021 20:56:18 GMT Subject: Integrated: 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) In-Reply-To: References: Message-ID: On Wed, 17 Nov 2021 19:28:19 GMT, Brian Burkhalter wrote: > Please consider this change which uses a new method `WindowsPath::getPathWithPrefixForWin32Calls`. This pull request has now been integrated. Changeset: 15345e3e Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/15345e3edcd8beee0d2c3192cfaaf76c7accee1f Stats: 48 lines in 3 files changed: 24 ins; 6 del; 18 mod 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) Reviewed-by: alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/6435 From rriggs at openjdk.java.net Thu Dec 2 20:59:17 2021 From: rriggs at openjdk.java.net (Roger Riggs) Date: Thu, 2 Dec 2021 20:59:17 GMT Subject: RFR: 8278168: Add a few missing words to the specification of Files.mismatch In-Reply-To: References: Message-ID: On Thu, 2 Dec 2021 20:46:10 GMT, Pavel Rappo wrote: > Please review this trivial fix. Marked as reviewed by rriggs (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6679 From duke at openjdk.java.net Thu Dec 2 21:38:23 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 2 Dec 2021 21:38:23 GMT Subject: Withdrawn: 8276994: java/nio/channels/Channels/TransferTo.java leaves multi-GB files in /tmp In-Reply-To: References: Message-ID: <-xSKls2lLrVkBLlTvvOeqThFpbbxw_b5G_2NBKpMrpU=.52ae01db-ec15-48f7-ae42-0133c805a1d0@github.com> On Sat, 13 Nov 2021 21:56:36 GMT, Markus KARG wrote: > This PR proposes to cleanup *all* temp files of the `TransferTo` test, not just the 2GB files. Also it separates the actual test logic from the cleanup logic, so the test cases are simple to understand again as they are not interwoven with cleanup code anymore. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.java.net/jdk/pull/6379 From duke at openjdk.java.net Thu Dec 2 21:38:50 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 2 Dec 2021 21:38:50 GMT Subject: RFR: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels [v18] In-Reply-To: References: Message-ID: On Wed, 1 Dec 2021 11:56:19 GMT, Lance Andersen wrote: >> Markus KARG has updated the pull request incrementally with one additional commit since the last revision: >> >> Creating temp file in current working directory >> >> Signed-off-by: Markus Karg > >> > > JDK-8276994 was resolved previously so I am not sure why you still have a PR open for that issue as you would not be able to integrate without a new bug. > > Please integrate any remaining changes into the test and we can finalize the review for you @LanceAndersen This PR now contains the remaining changes and I have closed https://github.com/openjdk/jdk/pull/6379 in turn. ------------- PR: https://git.openjdk.java.net/jdk/pull/5179 From duke at openjdk.java.net Thu Dec 2 21:38:48 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 2 Dec 2021 21:38:48 GMT Subject: RFR: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels [v19] In-Reply-To: References: Message-ID: > As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. This PR is a spin-off from https://github.com/openjdk/jdk/pull/4263 and deliberately concentrates **only** on `FileChannel`s. Other types of channels will be discussed in future PRs. > > * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer. > > Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. A rather similar approach in different use casse was recently implemented by https://github.com/openjdk/jdk/pull/5097 which was reported by even provide 5x performance (https://github.com/openjdk/jdk/pull/5097#issuecomment-897271997). > > **Update:** This sub-issue *only* defines the work to be done to implement JDK-8265891 *solely* for the particular case of FileChannel.transferTo(WriteableByteChannel), including special treatment of SelectableByteChannel, as the `master` branch already contains code to handle the specific case of FileChannel.transferTo(FileChannel). Markus KARG has updated the pull request incrementally with one additional commit since the last revision: Merged PR 6379 as requested by Lance Andersen Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5179/files - new: https://git.openjdk.java.net/jdk/pull/5179/files/807139e0..50044c82 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5179&range=18 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5179&range=17-18 Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/5179.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5179/head:pull/5179 PR: https://git.openjdk.java.net/jdk/pull/5179 From duke at openjdk.java.net Thu Dec 2 21:38:22 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 2 Dec 2021 21:38:22 GMT Subject: RFR: 8276994: java/nio/channels/Channels/TransferTo.java leaves multi-GB files in /tmp [v3] In-Reply-To: References: Message-ID: <08a-w4jzlBeAuhbmgmvrbVPvS0DM6u39_NYWCCinADg=.1d953c01-e493-4ad8-809c-c84ba6aed2a1@github.com> On Sat, 27 Nov 2021 10:07:36 GMT, Markus KARG wrote: >> This PR proposes to cleanup *all* temp files of the `TransferTo` test, not just the 2GB files. Also it separates the actual test logic from the cleanup logic, so the test cases are simple to understand again as they are not interwoven with cleanup code anymore. > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > Creating temp files in current working directory > > Signed-off-by: Markus Karg I have integrated the changes we agreed upon from this PR into PR https://github.com/openjdk/jdk/pull/5179, hence I hereby close *this* PR. ------------- PR: https://git.openjdk.java.net/jdk/pull/6379 From bpb at openjdk.java.net Thu Dec 2 21:39:16 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 2 Dec 2021 21:39:16 GMT Subject: RFR: 8278168: Add a few missing words to the specification of Files.mismatch In-Reply-To: References: Message-ID: On Thu, 2 Dec 2021 20:46:10 GMT, Pavel Rappo wrote: > Please review this trivial fix. Marked as reviewed by bpb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6679 From bpb at openjdk.java.net Thu Dec 2 22:38:36 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 2 Dec 2021 22:38:36 GMT Subject: RFR: 8278172: java/nio/channels/FileChannel/BlockDeviceSize.java should only run on Linux Message-ID: Please review this simple change which merely adds `@requires (os.family == "linux")` to the test. ------------- Commit messages: - 8278172: java/nio/channels/FileChannel/BlockDeviceSize.java should only run on Linux Changes: https://git.openjdk.java.net/jdk/pull/6681/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6681&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8278172 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6681.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6681/head:pull/6681 PR: https://git.openjdk.java.net/jdk/pull/6681 From lancea at openjdk.java.net Thu Dec 2 22:51:17 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Thu, 2 Dec 2021 22:51:17 GMT Subject: RFR: 8278172: java/nio/channels/FileChannel/BlockDeviceSize.java should only run on Linux In-Reply-To: References: Message-ID: On Thu, 2 Dec 2021 22:29:30 GMT, Brian Burkhalter wrote: > Please review this simple change which merely adds `@requires (os.family == "linux")` to the test. Marked as reviewed by lancea (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6681 From iris at openjdk.java.net Fri Dec 3 01:43:17 2021 From: iris at openjdk.java.net (Iris Clark) Date: Fri, 3 Dec 2021 01:43:17 GMT Subject: RFR: 8278168: Add a few missing words to the specification of Files.mismatch In-Reply-To: References: Message-ID: On Thu, 2 Dec 2021 20:46:10 GMT, Pavel Rappo wrote: > Please review this trivial fix. Marked as reviewed by iris (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6679 From alanb at openjdk.java.net Fri Dec 3 07:39:15 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 3 Dec 2021 07:39:15 GMT Subject: RFR: 8278172: java/nio/channels/FileChannel/BlockDeviceSize.java should only run on Linux In-Reply-To: References: Message-ID: On Thu, 2 Dec 2021 22:29:30 GMT, Brian Burkhalter wrote: > Please review this simple change which merely adds `@requires (os.family == "linux")` to the test. Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6681 From alanb at openjdk.java.net Fri Dec 3 08:16:15 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 3 Dec 2021 08:16:15 GMT Subject: RFR: 8278168: Add a few missing words to the specification of Files.mismatch In-Reply-To: References: Message-ID: On Thu, 2 Dec 2021 20:46:10 GMT, Pavel Rappo wrote: > Please review this trivial fix. Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6679 From alanb at openjdk.java.net Fri Dec 3 09:11:14 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 3 Dec 2021 09:11:14 GMT Subject: RFR: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels [v19] In-Reply-To: References: Message-ID: <4-2VPz9tmL6hRjNQqtPCRLpokorKMhS7ZsWoED18kvY=.c7f05a57-9e1d-4254-8a91-4ad604c00343@github.com> On Thu, 2 Dec 2021 21:38:48 GMT, Markus KARG wrote: >> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. This PR is a spin-off from https://github.com/openjdk/jdk/pull/4263 and deliberately concentrates **only** on `FileChannel`s. Other types of channels will be discussed in future PRs. >> >> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer. >> >> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. A rather similar approach in different use casse was recently implemented by https://github.com/openjdk/jdk/pull/5097 which was reported by even provide 5x performance (https://github.com/openjdk/jdk/pull/5097#issuecomment-897271997). >> >> **Update:** This sub-issue *only* defines the work to be done to implement JDK-8265891 *solely* for the particular case of FileChannel.transferTo(WriteableByteChannel), including special treatment of SelectableByteChannel, as the `master` branch already contains code to handle the specific case of FileChannel.transferTo(FileChannel). > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > Merged PR 6379 as requested by Lance Andersen > > Signed-off-by: Markus Karg Marked as reviewed by alanb (Reviewer). test/jdk/java/nio/channels/Channels/TransferTo.java line 217: > 215: pipe.sink().close(); > 216: } > 217: } Minor nit, indentation at L217 is lost. ------------- PR: https://git.openjdk.java.net/jdk/pull/5179 From duke at openjdk.java.net Fri Dec 3 09:39:18 2021 From: duke at openjdk.java.net (Andrey Turbanov) Date: Fri, 3 Dec 2021 09:39:18 GMT Subject: RFR: 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win) [v8] In-Reply-To: References: Message-ID: On Thu, 2 Dec 2021 17:51:59 GMT, Brian Burkhalter wrote: >> Please consider this change which uses a new method `WindowsPath::getPathWithPrefixForWin32Calls`. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8276661: s/\!forceLongPrefix/allowShortPath/ src/java.base/windows/classes/sun/nio/fs/WindowsPath.java line 219: > 217: // of the VM) > 218: if (allowShortPath && type != WindowsPathType.DRIVE_RELATIVE) { > 219: synchronized (this) { Hm. Why do we need this `synchronized` here? Seems redundant for me. ------------- PR: https://git.openjdk.java.net/jdk/pull/6435 From prappo at openjdk.java.net Fri Dec 3 09:57:19 2021 From: prappo at openjdk.java.net (Pavel Rappo) Date: Fri, 3 Dec 2021 09:57:19 GMT Subject: Integrated: 8278168: Add a few missing words to the specification of Files.mismatch In-Reply-To: References: Message-ID: <-W-lNMH2wMLh8oa-MaPNP4cp6Ls-b3dU9QEMNItW3Wo=.b2c4c0b9-dcd9-4540-a56f-822e1d4e93cf@github.com> On Thu, 2 Dec 2021 20:46:10 GMT, Pavel Rappo wrote: > Please review this trivial fix. This pull request has now been integrated. Changeset: d1cad246 Author: Pavel Rappo URL: https://git.openjdk.java.net/jdk/commit/d1cad2463a3f70d4ff9246d5bbae268bfa6717d7 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod 8278168: Add a few missing words to the specification of Files.mismatch Reviewed-by: rriggs, bpb, iris, alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/6679 From lancea at openjdk.java.net Fri Dec 3 12:04:24 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Fri, 3 Dec 2021 12:04:24 GMT Subject: RFR: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels [v19] In-Reply-To: References: Message-ID: On Thu, 2 Dec 2021 21:38:48 GMT, Markus KARG wrote: >> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. This PR is a spin-off from https://github.com/openjdk/jdk/pull/4263 and deliberately concentrates **only** on `FileChannel`s. Other types of channels will be discussed in future PRs. >> >> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer. >> >> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. A rather similar approach in different use casse was recently implemented by https://github.com/openjdk/jdk/pull/5097 which was reported by even provide 5x performance (https://github.com/openjdk/jdk/pull/5097#issuecomment-897271997). >> >> **Update:** This sub-issue *only* defines the work to be done to implement JDK-8265891 *solely* for the particular case of FileChannel.transferTo(WriteableByteChannel), including special treatment of SelectableByteChannel, as the `master` branch already contains code to handle the specific case of FileChannel.transferTo(FileChannel). > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > Merged PR 6379 as requested by Lance Andersen > > Signed-off-by: Markus Karg The changes look OK. I will run the PR through Mach5 tier1 - tier3 prior to approving as a sanity check. ------------- PR: https://git.openjdk.java.net/jdk/pull/5179 From duke at openjdk.java.net Fri Dec 3 12:36:55 2021 From: duke at openjdk.java.net (Markus KARG) Date: Fri, 3 Dec 2021 12:36:55 GMT Subject: RFR: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels [v20] In-Reply-To: References: Message-ID: > As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. This PR is a spin-off from https://github.com/openjdk/jdk/pull/4263 and deliberately concentrates **only** on `FileChannel`s. Other types of channels will be discussed in future PRs. > > * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer. > > Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. A rather similar approach in different use casse was recently implemented by https://github.com/openjdk/jdk/pull/5097 which was reported by even provide 5x performance (https://github.com/openjdk/jdk/pull/5097#issuecomment-897271997). > > **Update:** This sub-issue *only* defines the work to be done to implement JDK-8265891 *solely* for the particular case of FileChannel.transferTo(WriteableByteChannel), including special treatment of SelectableByteChannel, as the `master` branch already contains code to handle the specific case of FileChannel.transferTo(FileChannel). Markus KARG has updated the pull request incrementally with one additional commit since the last revision: Fixed indentation Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5179/files - new: https://git.openjdk.java.net/jdk/pull/5179/files/50044c82..39c36cbb Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5179&range=19 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5179&range=18-19 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/5179.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5179/head:pull/5179 PR: https://git.openjdk.java.net/jdk/pull/5179 From duke at openjdk.java.net Fri Dec 3 12:36:56 2021 From: duke at openjdk.java.net (Markus KARG) Date: Fri, 3 Dec 2021 12:36:56 GMT Subject: RFR: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels [v19] In-Reply-To: References: Message-ID: On Fri, 3 Dec 2021 12:01:02 GMT, Lance Andersen wrote: > The changes look OK. I will run the PR through Mach5 tier1 - tier3 prior to approving as a sanity check. Ok thank you, so I wait with the integration for your test result. ------------- PR: https://git.openjdk.java.net/jdk/pull/5179 From duke at openjdk.java.net Fri Dec 3 12:36:59 2021 From: duke at openjdk.java.net (Markus KARG) Date: Fri, 3 Dec 2021 12:36:59 GMT Subject: RFR: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels [v19] In-Reply-To: <4-2VPz9tmL6hRjNQqtPCRLpokorKMhS7ZsWoED18kvY=.c7f05a57-9e1d-4254-8a91-4ad604c00343@github.com> References: <4-2VPz9tmL6hRjNQqtPCRLpokorKMhS7ZsWoED18kvY=.c7f05a57-9e1d-4254-8a91-4ad604c00343@github.com> Message-ID: On Fri, 3 Dec 2021 09:08:30 GMT, Alan Bateman wrote: >> Markus KARG has updated the pull request incrementally with one additional commit since the last revision: >> >> Merged PR 6379 as requested by Lance Andersen >> >> Signed-off-by: Markus Karg > > test/jdk/java/nio/channels/Channels/TransferTo.java line 217: > >> 215: pipe.sink().close(); >> 216: } >> 217: } > > Minor nit, indentation at L217 is lost. Good catch! Indentation is fixed in https://github.com/openjdk/jdk/pull/5179/commits/39c36cbb1d408c5f7a7afd74b282f65ca06be547. ------------- PR: https://git.openjdk.java.net/jdk/pull/5179 From bpb at openjdk.java.net Fri Dec 3 16:04:16 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 3 Dec 2021 16:04:16 GMT Subject: Integrated: 8278172: java/nio/channels/FileChannel/BlockDeviceSize.java should only run on Linux In-Reply-To: References: Message-ID: On Thu, 2 Dec 2021 22:29:30 GMT, Brian Burkhalter wrote: > Please review this simple change which merely adds `@requires (os.family == "linux")` to the test. This pull request has now been integrated. Changeset: 0d938ced Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/0d938cedcaf55886058b94dc280a8f7181c79fdf Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod 8278172: java/nio/channels/FileChannel/BlockDeviceSize.java should only run on Linux Reviewed-by: lancea, alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/6681 From bpb at openjdk.java.net Fri Dec 3 20:15:29 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 3 Dec 2021 20:15:29 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out Message-ID: In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. ------------- Commit messages: - 8278166: java/nio/channels/Channels/TransferTo.java timed out Changes: https://git.openjdk.java.net/jdk/pull/6707/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6707&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8278166 Stats: 20 lines in 1 file changed: 16 ins; 1 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/6707.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6707/head:pull/6707 PR: https://git.openjdk.java.net/jdk/pull/6707 From bpb at openjdk.java.net Fri Dec 3 20:15:30 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 3 Dec 2021 20:15:30 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out In-Reply-To: References: Message-ID: <4NqlcouwZRznSpD7w-NU4WK_mIzMQVLkxPIWFwmsq3M=.0d1a3701-936c-45f6-8fff-c29dc6cd1a3d@github.com> On Fri, 3 Dec 2021 20:07:14 GMT, Brian Burkhalter wrote: > In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. Total time to run this test locally decreased by about 25% on my development laptop. ------------- PR: https://git.openjdk.java.net/jdk/pull/6707 From lancea at openjdk.java.net Fri Dec 3 22:49:21 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Fri, 3 Dec 2021 22:49:21 GMT Subject: RFR: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels [v20] In-Reply-To: References: Message-ID: On Fri, 3 Dec 2021 12:36:55 GMT, Markus KARG wrote: >> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. This PR is a spin-off from https://github.com/openjdk/jdk/pull/4263 and deliberately concentrates **only** on `FileChannel`s. Other types of channels will be discussed in future PRs. >> >> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer. >> >> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. A rather similar approach in different use casse was recently implemented by https://github.com/openjdk/jdk/pull/5097 which was reported by even provide 5x performance (https://github.com/openjdk/jdk/pull/5097#issuecomment-897271997). >> >> **Update:** This sub-issue *only* defines the work to be done to implement JDK-8265891 *solely* for the particular case of FileChannel.transferTo(WriteableByteChannel), including special treatment of SelectableByteChannel, as the `master` branch already contains code to handle the specific case of FileChannel.transferTo(FileChannel). > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > Fixed indentation > > Signed-off-by: Markus Karg Marked as reviewed by lancea (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/5179 From lancea at openjdk.java.net Fri Dec 3 22:49:21 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Fri, 3 Dec 2021 22:49:21 GMT Subject: RFR: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels [v19] In-Reply-To: References: Message-ID: On Fri, 3 Dec 2021 12:32:53 GMT, Markus KARG wrote: > > The changes look OK. I will run the PR through Mach5 tier1 - tier3 prior to approving as a sanity check. > > Ok thank you, so I wait with the integration for your test result. Mach5 tiers1 - 3 ran fine ------------- PR: https://git.openjdk.java.net/jdk/pull/5179 From lancea at openjdk.java.net Fri Dec 3 23:02:15 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Fri, 3 Dec 2021 23:02:15 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out In-Reply-To: References: Message-ID: On Fri, 3 Dec 2021 20:07:14 GMT, Brian Burkhalter wrote: > In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. Looks OK overall. Minor comment below test/jdk/java/nio/channels/Channels/TransferTo.java line 150: > 148: try { > 149: // create (hopefully sparse) file less than 2GB in size > 150: final long initLen = (2*1024 - 1)*BYTES_PER_WRITE; Perhaps consider clarifying the comment for the initLen calculation I do suspect the use of RandomAccessFile improves the throughput here ------------- Marked as reviewed by lancea (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6707 From bpb at openjdk.java.net Fri Dec 3 23:09:12 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 3 Dec 2021 23:09:12 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out In-Reply-To: References: Message-ID: On Fri, 3 Dec 2021 22:57:53 GMT, Lance Andersen wrote: >> In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. > > test/jdk/java/nio/channels/Channels/TransferTo.java line 150: > >> 148: try { >> 149: // create (hopefully sparse) file less than 2GB in size >> 150: final long initLen = (2*1024 - 1)*BYTES_PER_WRITE; > > Perhaps consider clarifying the comment for the initLen calculation > > I do suspect the use of RandomAccessFile improves the throughput here I shall clean up the constants in the next revision. A crude measurement showed that the execution time of `testMoreThanTwoGB()` was cut in half by this change. This sub-test is the slowest `@Test` in the test. ------------- PR: https://git.openjdk.java.net/jdk/pull/6707 From alanb at openjdk.java.net Sat Dec 4 06:53:15 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Sat, 4 Dec 2021 06:53:15 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out In-Reply-To: References: Message-ID: <0bPxRAsnD0NLvVJmxd-2z3niT7QuOIuoR-YWe1qxMkM=.6120c9e6-25f7-4acf-8b84-b3ba9b057db0@github.com> On Fri, 3 Dec 2021 23:05:42 GMT, Brian Burkhalter wrote: >> test/jdk/java/nio/channels/Channels/TransferTo.java line 150: >> >>> 148: try { >>> 149: // create (hopefully sparse) file less than 2GB in size >>> 150: final long initLen = (2*1024 - 1)*BYTES_PER_WRITE; >> >> Perhaps consider clarifying the comment for the initLen calculation >> >> I do suspect the use of RandomAccessFile improves the throughput here > > I shall clean up the constants in the next revision. > > A crude measurement showed that the execution time of `testMoreThanTwoGB()` was cut in half by this change. This sub-test is the slowest `@Test` in the test. The general approach looks okay but the RAF to open/setLength/close is not needed. Instead change the FC.open to use WRITE and use fc.position(initLen) to set the initial position before writing. ------------- PR: https://git.openjdk.java.net/jdk/pull/6707 From duke at openjdk.java.net Sat Dec 4 09:32:14 2021 From: duke at openjdk.java.net (Markus KARG) Date: Sat, 4 Dec 2021 09:32:14 GMT Subject: Integrated: 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels In-Reply-To: References: Message-ID: On Thu, 19 Aug 2021 06:16:42 GMT, Markus KARG wrote: > As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. This PR is a spin-off from https://github.com/openjdk/jdk/pull/4263 and deliberately concentrates **only** on `FileChannel`s. Other types of channels will be discussed in future PRs. > > * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer. > > Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. A rather similar approach in different use casse was recently implemented by https://github.com/openjdk/jdk/pull/5097 which was reported by even provide 5x performance (https://github.com/openjdk/jdk/pull/5097#issuecomment-897271997). > > **Update:** This sub-issue *only* defines the work to be done to implement JDK-8265891 *solely* for the particular case of FileChannel.transferTo(WriteableByteChannel), including special treatment of SelectableByteChannel, as the `master` branch already contains code to handle the specific case of FileChannel.transferTo(FileChannel). This pull request has now been integrated. Changeset: 9642629d Author: Markus Karg Committer: Alan Bateman URL: https://git.openjdk.java.net/jdk/commit/9642629d15d9631d8da9a3abdabc40323f3c774b Stats: 110 lines in 3 files changed: 99 ins; 0 del; 11 mod 8276779: (ch) InputStream returned by Channels.newInputStream should have fast path for SelectableChannels Reviewed-by: lancea, alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/5179 From duke at openjdk.java.net Sun Dec 5 13:12:30 2021 From: duke at openjdk.java.net (Markus KARG) Date: Sun, 5 Dec 2021 13:12:30 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets Message-ID: This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. ------------- Commit messages: - Implementing 8278268 Changes: https://git.openjdk.java.net/jdk/pull/6711/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8278268 Stats: 72 lines in 2 files changed: 59 ins; 2 del; 11 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Sun Dec 5 15:16:36 2021 From: duke at openjdk.java.net (Markus KARG) Date: Sun, 5 Dec 2021 15:16:36 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v2] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: Implementing 8278268 Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/ee5ab91d..e78676a7 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=00-01 Stats: 9 lines in 1 file changed: 9 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Sun Dec 5 15:59:49 2021 From: duke at openjdk.java.net (Markus KARG) Date: Sun, 5 Dec 2021 15:59:49 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v3] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with one additional commit since the last revision: testing 2GB+ Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/e78676a7..717440b1 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=01-02 Stats: 49 lines in 1 file changed: 43 ins; 0 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Mon Dec 6 07:10:21 2021 From: duke at openjdk.java.net (Markus KARG) Date: Mon, 6 Dec 2021 07:10:21 GMT Subject: RFR: 8265891: (ch) InputStream returned by Channels.newInputStream should override transferTo [v13] In-Reply-To: References: Message-ID: On Sun, 1 Aug 2021 22:01:33 GMT, Markus KARG wrote: >> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review. >> >> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are: >> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer. >> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides. >> >> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. >> >> I encourage everybody to discuss this draft: >> * Are there valid arguments for *not* doing this change? >> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`? >> * How to go on from here: What is missing to get this ready for an actual review? > > Markus KARG has updated the pull request incrementally with two additional commits since the last revision: > > - Draft: Eliminated duplicate code using lambda expressions > - Draft: Use blocking mode also for target channel Please keep this PR open. More commits will follow on the next weeks. ------------- PR: https://git.openjdk.java.net/jdk/pull/4263 From lbourges at openjdk.java.net Mon Dec 6 08:45:22 2021 From: lbourges at openjdk.java.net (Laurent =?UTF-8?B?Qm91cmfDqHM=?=) Date: Mon, 6 Dec 2021 08:45:22 GMT Subject: RFR: 8265891: (ch) InputStream returned by Channels.newInputStream should override transferTo [v13] In-Reply-To: References: Message-ID: On Mon, 6 Dec 2021 07:07:03 GMT, Markus KARG wrote: >> Markus KARG has updated the pull request incrementally with two additional commits since the last revision: >> >> - Draft: Eliminated duplicate code using lambda expressions >> - Draft: Use blocking mode also for target channel > > Please keep this PR open. More commits will follow on the next weeks. Looks promising, keep moving, @mkarg ! ------------- PR: https://git.openjdk.java.net/jdk/pull/4263 From duke at openjdk.java.net Mon Dec 6 13:26:29 2021 From: duke at openjdk.java.net (Maxim Kartashev) Date: Mon, 6 Dec 2021 13:26:29 GMT Subject: RFR: 8274883: (se) Selector.open throws IAE when the default file system provider is changed to a custom provider Message-ID: The gist of the problem: when a file system is specified via `-Djava.nio.file.spi.DefaultFileSystemProvider`, a call to `SelectorProvider.provider().openSelector()` ends with a throw on Windows. There are two distinct components to the problem: 1. `ExceptionInInitializerError` is thrown during the static initialization of the `UnixDomainSockets.UNNAMED` field even though it isn't used on this code path (see `UnixDomainSocketAddress.of()` that throws `IllegalArgumentException` if invoked on a path from a non-default file system). This is fixed by lazy-initializing the static member `UNNAMED` of `UnixDomainSockets` so that this initialization doesn't throw unless actually used. 2. `IllegalArgumentException` is thrown by `UnixDomainSocketAddress.of()` later on when `ServerSocketChannel` tries to use Windows version of `PipeImpl` and its method `createListener()` specifically. That `PipeImpl` probes for the availability of Unix Domain Sockets by trying to bind to a unique temporary name. That call throws `IAE` when a non-default Java file system is installed while the probing code (`PipeImpl.createListener()`) only expects `UnsupportedOperationException` or `IOException`. The fix is to re-throw `UOE` instead of `IAE` in `UnixDomainSockets.genrateTempName()`. This is more consistent with the definition of the exception purpose ("requested operation is not supported"). So with this change, a loopback network socket will be used to implement a pipe on a non-default Java file system. Also, pipes do not rely on the default Java file system on other platforms (Linux, MacOS) as well. Tested by running `jtreg:test/jdk/java/nio` on Window, MacOS, and Linux. ------------- Commit messages: - 8274883: (se) Selector.open throws IAE when the default file system provider is changed to a custom provider Changes: https://git.openjdk.java.net/jdk/pull/6722/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6722&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8274883 Stats: 50 lines in 3 files changed: 46 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/6722.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6722/head:pull/6722 PR: https://git.openjdk.java.net/jdk/pull/6722 From alanb at openjdk.java.net Mon Dec 6 13:44:22 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Mon, 6 Dec 2021 13:44:22 GMT Subject: RFR: 8274883: (se) Selector.open throws IAE when the default file system provider is changed to a custom provider In-Reply-To: References: Message-ID: On Mon, 6 Dec 2021 13:19:54 GMT, Maxim Kartashev wrote: > The gist of the problem: when a file system is specified via `-Djava.nio.file.spi.DefaultFileSystemProvider`, a call to `SelectorProvider.provider().openSelector()` ends with a throw on Windows. > > There are two distinct components to the problem: > > 1. `ExceptionInInitializerError` is thrown during the static initialization of the `UnixDomainSockets.UNNAMED` field even though it isn't used on this code path (see `UnixDomainSocketAddress.of()` that throws `IllegalArgumentException` if invoked on a path from a non-default file system). > This is fixed by lazy-initializing the static member `UNNAMED` of `UnixDomainSockets` so that this initialization doesn't throw unless actually used. > > 2. `IllegalArgumentException` is thrown by `UnixDomainSocketAddress.of()` later on when `ServerSocketChannel` tries to use Windows version of `PipeImpl` and its method `createListener()` specifically. That `PipeImpl` probes for the availability of Unix Domain Sockets by trying to bind to a unique temporary name. That call throws `IAE` when a non-default Java file system is installed while the probing code (`PipeImpl.createListener()`) only expects `UnsupportedOperationException` or `IOException`. > The fix is to re-throw `UOE` instead of `IAE` in `UnixDomainSockets.genrateTempName()`. This is more consistent with the definition of the exception purpose ("requested operation is not supported"). So with this change, a loopback network socket will be used to implement a pipe on a non-default Java file system. Also, pipes do not rely on the default Java file system on other platforms (Linux, MacOS) as well. > > Tested by running `jtreg:test/jdk/java/nio` on Window, MacOS, and Linux. The JBS issue is assigned to @Michael-Mc-Mahon, might need to check if he already has a patch this for too. src/java.base/share/classes/sun/nio/ch/UnixDomainSockets.java line 47: > 45: private UnixDomainSockets() { } > 46: > 47: private static class UNNAMEDHolder { Let's rename this to UnamedHolder to avoid this strange class name. src/java.base/share/classes/sun/nio/ch/UnixDomainSockets.java line 143: > 141: throw new BindException("Invalid temporary directory"); > 142: } catch (IllegalArgumentException e) { > 143: throw new UnsupportedOperationException("Unix Domain Sockets not supported on non-default file system"); There should be no need to catch IAE here, instead you can check the provider. src/java.base/share/classes/sun/nio/ch/UnixDomainSockets.java line 186: > 184: } > 185: > 186: static UnixDomainSocketAddress getUNNAMED() { Can you rename this to unnamed() and move it up with the other package default static methods? ------------- Changes requested by alanb (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6722 From duke at openjdk.java.net Mon Dec 6 15:09:52 2021 From: duke at openjdk.java.net (Maxim Kartashev) Date: Mon, 6 Dec 2021 15:09:52 GMT Subject: RFR: 8274883: (se) Selector.open throws IAE when the default file system provider is changed to a custom provider [v2] In-Reply-To: References: Message-ID: > The gist of the problem: when a file system is specified via `-Djava.nio.file.spi.DefaultFileSystemProvider`, a call to `SelectorProvider.provider().openSelector()` ends with a throw on Windows. > > There are two distinct components to the problem: > > 1. `ExceptionInInitializerError` is thrown during the static initialization of the `UnixDomainSockets.UNNAMED` field even though it isn't used on this code path (see `UnixDomainSocketAddress.of()` that throws `IllegalArgumentException` if invoked on a path from a non-default file system). > This is fixed by lazy-initializing the static member `UNNAMED` of `UnixDomainSockets` so that this initialization doesn't throw unless actually used. > > 2. `IllegalArgumentException` is thrown by `UnixDomainSocketAddress.of()` later on when `ServerSocketChannel` tries to use Windows version of `PipeImpl` and its method `createListener()` specifically. That `PipeImpl` probes for the availability of Unix Domain Sockets by trying to bind to a unique temporary name. That call throws `IAE` when a non-default Java file system is installed while the probing code (`PipeImpl.createListener()`) only expects `UnsupportedOperationException` or `IOException`. > The fix is to re-throw `UOE` instead of `IAE` in `UnixDomainSockets.genrateTempName()`. This is more consistent with the definition of the exception purpose ("requested operation is not supported"). So with this change, a loopback network socket will be used to implement a pipe on a non-default Java file system. Also, pipes do not rely on the default Java file system on other platforms (Linux, MacOS) as well. > > Tested by running `jtreg:test/jdk/java/nio` on Window, MacOS, and Linux. Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: Addressed PR comments: - renamed UNNAMEDHolder to UnnamedHolder, - renamed getUNNAMED() to unnamed(), - replaced catch IllegalArgumentException with a check for the provider. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6722/files - new: https://git.openjdk.java.net/jdk/pull/6722/files/1b7d0b9f..8dcc88e3 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6722&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6722&range=00-01 Stats: 19 lines in 2 files changed: 8 ins; 6 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/6722.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6722/head:pull/6722 PR: https://git.openjdk.java.net/jdk/pull/6722 From duke at openjdk.java.net Mon Dec 6 15:09:58 2021 From: duke at openjdk.java.net (Maxim Kartashev) Date: Mon, 6 Dec 2021 15:09:58 GMT Subject: RFR: 8274883: (se) Selector.open throws IAE when the default file system provider is changed to a custom provider [v2] In-Reply-To: References: Message-ID: On Mon, 6 Dec 2021 13:32:56 GMT, Alan Bateman wrote: >> Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: >> >> Addressed PR comments: >> >> - renamed UNNAMEDHolder to UnnamedHolder, >> - renamed getUNNAMED() to unnamed(), >> - replaced catch IllegalArgumentException with a check for the provider. > > src/java.base/share/classes/sun/nio/ch/UnixDomainSockets.java line 47: > >> 45: private UnixDomainSockets() { } >> 46: >> 47: private static class UNNAMEDHolder { > > Let's rename this to UnamedHolder to avoid this strange class name. Also renamed. > src/java.base/share/classes/sun/nio/ch/UnixDomainSockets.java line 143: > >> 141: throw new BindException("Invalid temporary directory"); >> 142: } catch (IllegalArgumentException e) { >> 143: throw new UnsupportedOperationException("Unix Domain Sockets not supported on non-default file system"); > > There should be no need to catch IAE here, instead you can check the provider. Changed to `if (path.getFileSystem().provider() != sun.nio.fs.DefaultFileSystemProvider.instance()) ...` I hope that was what you meant. > src/java.base/share/classes/sun/nio/ch/UnixDomainSockets.java line 186: > >> 184: } >> 185: >> 186: static UnixDomainSocketAddress getUNNAMED() { > > Can you rename this to unnamed() and move it up with the other package default static methods? Thanks for reviewing! Renamed to `unnamed()`. ------------- PR: https://git.openjdk.java.net/jdk/pull/6722 From bpb at openjdk.java.net Mon Dec 6 16:32:17 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 6 Dec 2021 16:32:17 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out In-Reply-To: <0bPxRAsnD0NLvVJmxd-2z3niT7QuOIuoR-YWe1qxMkM=.6120c9e6-25f7-4acf-8b84-b3ba9b057db0@github.com> References: <0bPxRAsnD0NLvVJmxd-2z3niT7QuOIuoR-YWe1qxMkM=.6120c9e6-25f7-4acf-8b84-b3ba9b057db0@github.com> Message-ID: On Sat, 4 Dec 2021 06:50:12 GMT, Alan Bateman wrote: >> I shall clean up the constants in the next revision. >> >> A crude measurement showed that the execution time of `testMoreThanTwoGB()` was cut in half by this change. This sub-test is the slowest `@Test` in the test. > > The general approach looks okay but the RAF to open/setLength/close is not needed. Instead change the FC.open to use WRITE and use fc.position(initLen) to set the initial position before writing. Of course; will do. ------------- PR: https://git.openjdk.java.net/jdk/pull/6707 From duke at openjdk.java.net Mon Dec 6 17:36:41 2021 From: duke at openjdk.java.net (Markus KARG) Date: Mon, 6 Dec 2021 17:36:41 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v4] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with one additional commit since the last revision: removed trailing whitespace Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/717440b1..6aef12ae Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From bpb at openjdk.java.net Mon Dec 6 17:48:42 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 6 Dec 2021 17:48:42 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out [v2] In-Reply-To: References: Message-ID: > In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - 8278166: Replace RandomAccessFile with FileChannel.open + FileChannel.position(long) - Merge - 8278166: java/nio/channels/Channels/TransferTo.java timed out ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6707/files - new: https://git.openjdk.java.net/jdk/pull/6707/files/29c4e9f4..88a0e72d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6707&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6707&range=00-01 Stats: 1914 lines in 119 files changed: 1364 ins; 234 del; 316 mod Patch: https://git.openjdk.java.net/jdk/pull/6707.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6707/head:pull/6707 PR: https://git.openjdk.java.net/jdk/pull/6707 From prr at openjdk.java.net Mon Dec 6 17:57:14 2021 From: prr at openjdk.java.net (Phil Race) Date: Mon, 6 Dec 2021 17:57:14 GMT Subject: RFR: JDK-8276447 Deprecate finalization-related methods for removal [v3] In-Reply-To: References: Message-ID: On Wed, 1 Dec 2021 19:23:59 GMT, Brent Christian wrote: >> Here are the code changes for the "Deprecate finalizers in the standard Java API" portion of JEP 421 ("Deprecate Finalization for Removal") for code review. >> >> This change makes the indicated deprecations, and updates the API spec for JEP 421. It also updates the relevant @SuppressWarning annotations. >> >> The CSR has been approved. >> An automated test build+test run passes cleanly (FWIW :D ). > > Brent Christian has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 33 commits: > > - Merge branch 'master' into 8276447 > - Merge branch 'master' into 8276447 > - merge > - @SuppressWarnings(removal) in Windows CKey.java > - Add jls tag to runFinalization methods > - Update wording of Object.finalize, new paragraph is now bold > - update Object.finalize javadoc > - update Object.finalize JavaDoc and @deprecated tag > - Update Object.finalize deprecation message > - Indicate that runFinalizers does nothing if finalization is disabled or removed > - ... and 23 more: https://git.openjdk.java.net/jdk/compare/0dfb3a70...8cde0674 Marked as reviewed by prr (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6465 From alanb at openjdk.java.net Mon Dec 6 18:03:13 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Mon, 6 Dec 2021 18:03:13 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out [v2] In-Reply-To: References: Message-ID: On Mon, 6 Dec 2021 17:48:42 GMT, Brian Burkhalter wrote: >> In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - 8278166: Replace RandomAccessFile with FileChannel.open + FileChannel.position(long) > - Merge > - 8278166: java/nio/channels/Channels/TransferTo.java timed out This is much better, just some minor comments and I think we are good to go. There are probably other tests with large files that could benefit from the same thing. test/jdk/java/nio/channels/Channels/TransferTo.java line 160: > 158: public void testMoreThanTwoGB() throws IOException { > 159: Path sourceFile = CWD.resolve(Path.of(String.format("test3GBSource%s.tmp", > 160: String.valueOf(RND.nextInt(Integer.MAX_VALUE))))); The Path.of isn't needed here as there is an overload of resolve that takes a String. For readability then it might be simpler to create the file name (as a String) on its own line. test/jdk/java/nio/channels/Channels/TransferTo.java line 170: > 168: // create the file with a hint to be sparse > 169: try (FileChannel fc = FileChannel.open(sourceFile, CREATE_NEW, > 170: SPARSE, WRITE, APPEND);) { might be a bit more readable to not split this line. test/jdk/java/nio/channels/Channels/TransferTo.java line 178: > 176: for (int i = 0; i < nw; i++) { > 177: ByteBuffer src = > 178: ByteBuffer.wrap(createRandomBytes(BYTES_PER_WRITE, 0)); Probably not worth splitting this line either. ------------- PR: https://git.openjdk.java.net/jdk/pull/6707 From bpb at openjdk.java.net Mon Dec 6 18:12:42 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 6 Dec 2021 18:12:42 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out [v3] In-Reply-To: References: Message-ID: > In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8278166: Minor cleanup ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6707/files - new: https://git.openjdk.java.net/jdk/pull/6707/files/88a0e72d..a384a4f3 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6707&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6707&range=01-02 Stats: 5 lines in 1 file changed: 1 ins; 1 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/6707.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6707/head:pull/6707 PR: https://git.openjdk.java.net/jdk/pull/6707 From alanb at openjdk.java.net Mon Dec 6 18:12:45 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Mon, 6 Dec 2021 18:12:45 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out [v3] In-Reply-To: References: Message-ID: On Mon, 6 Dec 2021 18:08:42 GMT, Brian Burkhalter wrote: >> In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8278166: Minor cleanup Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6707 From duke at openjdk.java.net Mon Dec 6 18:24:42 2021 From: duke at openjdk.java.net (Markus KARG) Date: Mon, 6 Dec 2021 18:24:42 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v5] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - removed trailing whitespace Signed-off-by: Markus Karg - testing 2GB+ Signed-off-by: Markus Karg - Implementing 8278268 Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/6aef12ae..3d658231 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=03-04 Stats: 1078 lines in 76 files changed: 639 ins; 194 del; 245 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From bpb at openjdk.java.net Mon Dec 6 18:25:55 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 6 Dec 2021 18:25:55 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out [v4] In-Reply-To: References: Message-ID: > In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8278166: Un-split line overlooked in previous commit ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6707/files - new: https://git.openjdk.java.net/jdk/pull/6707/files/a384a4f3..a4da4f3a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6707&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6707&range=02-03 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/6707.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6707/head:pull/6707 PR: https://git.openjdk.java.net/jdk/pull/6707 From alanb at openjdk.java.net Mon Dec 6 18:25:57 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Mon, 6 Dec 2021 18:25:57 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out [v4] In-Reply-To: References: Message-ID: On Mon, 6 Dec 2021 18:22:12 GMT, Brian Burkhalter wrote: >> In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8278166: Un-split line overlooked in previous commit Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6707 From bpb at openjdk.java.net Mon Dec 6 18:26:00 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 6 Dec 2021 18:26:00 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out [v2] In-Reply-To: References: Message-ID: On Mon, 6 Dec 2021 17:58:53 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: >> >> - 8278166: Replace RandomAccessFile with FileChannel.open + FileChannel.position(long) >> - Merge >> - 8278166: java/nio/channels/Channels/TransferTo.java timed out > > test/jdk/java/nio/channels/Channels/TransferTo.java line 178: > >> 176: for (int i = 0; i < nw; i++) { >> 177: ByteBuffer src = >> 178: ByteBuffer.wrap(createRandomBytes(BYTES_PER_WRITE, 0)); > > Probably not worth splitting this line either. I missed this line split; fixed. ------------- PR: https://git.openjdk.java.net/jdk/pull/6707 From lancea at openjdk.java.net Mon Dec 6 20:25:16 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Mon, 6 Dec 2021 20:25:16 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out [v4] In-Reply-To: References: Message-ID: <3JrnhY0KkbachUPq7G6txjsr1_V3UAse8kSwg_XfxtU=.2a8dfc27-41e6-4f56-968b-9b732bb5fa7c@github.com> On Mon, 6 Dec 2021 18:25:55 GMT, Brian Burkhalter wrote: >> In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8278166: Un-split line overlooked in previous commit Looks Good Brian. For continuity, did you give any thoughts of making the same change to targetFile as you did for sourceFile to avoid using Files.createTempfile? ------------- Marked as reviewed by lancea (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6707 From bchristi at openjdk.java.net Mon Dec 6 20:41:43 2021 From: bchristi at openjdk.java.net (Brent Christian) Date: Mon, 6 Dec 2021 20:41:43 GMT Subject: RFR: JDK-8276447 Deprecate finalization-related methods for removal [v4] In-Reply-To: References: Message-ID: > Here are the code changes for the "Deprecate finalizers in the standard Java API" portion of JEP 421 ("Deprecate Finalization for Removal") for code review. > > This change makes the indicated deprecations, and updates the API spec for JEP 421. It also updates the relevant @SuppressWarning annotations. > > The CSR has been approved. > An automated test build+test run passes cleanly (FWIW :D ). Brent Christian has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 34 commits: - Merge branch 'master' into 8276447 - Merge branch 'master' into 8276447 - Merge branch 'master' into 8276447 - merge - @SuppressWarnings(removal) in Windows CKey.java - Add jls tag to runFinalization methods - Update wording of Object.finalize, new paragraph is now bold - update Object.finalize javadoc - update Object.finalize JavaDoc and @deprecated tag - Update Object.finalize deprecation message - ... and 24 more: https://git.openjdk.java.net/jdk/compare/ea8d3c92...0a983d51 ------------- Changes: https://git.openjdk.java.net/jdk/pull/6465/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6465&range=03 Stats: 194 lines in 62 files changed: 54 ins; 38 del; 102 mod Patch: https://git.openjdk.java.net/jdk/pull/6465.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6465/head:pull/6465 PR: https://git.openjdk.java.net/jdk/pull/6465 From bpb at openjdk.java.net Mon Dec 6 23:08:41 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 6 Dec 2021 23:08:41 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out [v5] In-Reply-To: References: Message-ID: > In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8278166: Change targetFile creation in testMoreThanTwoGB(); miscellaneous formatting cleanup ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6707/files - new: https://git.openjdk.java.net/jdk/pull/6707/files/a4da4f3a..96c87746 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6707&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6707&range=03-04 Stats: 80 lines in 1 file changed: 28 ins; 4 del; 48 mod Patch: https://git.openjdk.java.net/jdk/pull/6707.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6707/head:pull/6707 PR: https://git.openjdk.java.net/jdk/pull/6707 From bpb at openjdk.java.net Mon Dec 6 23:08:45 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 6 Dec 2021 23:08:45 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out [v4] In-Reply-To: <3JrnhY0KkbachUPq7G6txjsr1_V3UAse8kSwg_XfxtU=.2a8dfc27-41e6-4f56-968b-9b732bb5fa7c@github.com> References: <3JrnhY0KkbachUPq7G6txjsr1_V3UAse8kSwg_XfxtU=.2a8dfc27-41e6-4f56-968b-9b732bb5fa7c@github.com> Message-ID: On Mon, 6 Dec 2021 20:21:57 GMT, Lance Andersen wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8278166: Un-split line overlooked in previous commit > > Looks Good Brian. > > For continuity, did you give any thoughts of making the same change to targetFile as you did for sourceFile to avoid using Files.createTempfile? @LanceAndersen Changed targetFile init to similar to sourceFile init. ------------- PR: https://git.openjdk.java.net/jdk/pull/6707 From lancea at openjdk.java.net Mon Dec 6 23:21:10 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Mon, 6 Dec 2021 23:21:10 GMT Subject: RFR: 8278166: java/nio/channels/Channels/TransferTo.java timed out [v5] In-Reply-To: References: Message-ID: On Mon, 6 Dec 2021 23:08:41 GMT, Brian Burkhalter wrote: >> In the `testMoreThanTwoGB()` sub-test, only fill the last section of the file with random data in order to reduce test execution time. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8278166: Change targetFile creation in testMoreThanTwoGB(); miscellaneous formatting cleanup Marked as reviewed by lancea (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6707 From bpb at openjdk.java.net Tue Dec 7 01:55:27 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 7 Dec 2021 01:55:27 GMT Subject: RFR: 8277361: java/nio/channels/Channels/ReadXBytes.java fails with OOM error Message-ID: This request would increase the amount of heap memory that the test can use, and decrease execution time by not writing the entirety of test files. ------------- Commit messages: - 8277361: java/nio/channels/Channels/ReadXBytes.java fails with OOM error Changes: https://git.openjdk.java.net/jdk/pull/6732/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6732&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277361 Stats: 39 lines in 1 file changed: 23 ins; 2 del; 14 mod Patch: https://git.openjdk.java.net/jdk/pull/6732.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6732/head:pull/6732 PR: https://git.openjdk.java.net/jdk/pull/6732 From bpb at openjdk.java.net Tue Dec 7 01:55:27 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 7 Dec 2021 01:55:27 GMT Subject: RFR: 8277361: java/nio/channels/Channels/ReadXBytes.java fails with OOM error In-Reply-To: References: Message-ID: On Tue, 7 Dec 2021 01:47:47 GMT, Brian Burkhalter wrote: > This request would increase the amount of heap memory that the test can use, and decrease execution time by not writing the entirety of test files. The execution time of the test appeared to be decreased around 25% by writing fewer bytes to test files. ------------- PR: https://git.openjdk.java.net/jdk/pull/6732 From alanb at openjdk.java.net Tue Dec 7 06:32:11 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 7 Dec 2021 06:32:11 GMT Subject: RFR: 8277361: java/nio/channels/Channels/ReadXBytes.java fails with OOM error In-Reply-To: References: Message-ID: On Tue, 7 Dec 2021 01:47:47 GMT, Brian Burkhalter wrote: > This request would increase the amount of heap memory that the test can use, and decrease execution time by not writing the entirety of test files. Marked as reviewed by alanb (Reviewer). test/jdk/java/nio/channels/Channels/ReadXBytes.java line 91: > 89: Path path = createFilePath(); > 90: path.toFile().deleteOnExit(); > 91: try (FileChannel fc = FileChannel.open(path, CREATE_NEW, SPARSE, WRITE);) { Spurious semicolon at the end. ------------- PR: https://git.openjdk.java.net/jdk/pull/6732 From alanb at openjdk.java.net Tue Dec 7 06:42:11 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 7 Dec 2021 06:42:11 GMT Subject: RFR: 8274883: (se) Selector.open throws IAE when the default file system provider is changed to a custom provider [v2] In-Reply-To: References: Message-ID: On Mon, 6 Dec 2021 15:09:52 GMT, Maxim Kartashev wrote: >> The gist of the problem: when a file system is specified via `-Djava.nio.file.spi.DefaultFileSystemProvider`, a call to `SelectorProvider.provider().openSelector()` ends with a throw on Windows. >> >> There are two distinct components to the problem: >> >> 1. `ExceptionInInitializerError` is thrown during the static initialization of the `UnixDomainSockets.UNNAMED` field even though it isn't used on this code path (see `UnixDomainSocketAddress.of()` that throws `IllegalArgumentException` if invoked on a path from a non-default file system). >> This is fixed by lazy-initializing the static member `UNNAMED` of `UnixDomainSockets` so that this initialization doesn't throw unless actually used. >> >> 2. `IllegalArgumentException` is thrown by `UnixDomainSocketAddress.of()` later on when `ServerSocketChannel` tries to use Windows version of `PipeImpl` and its method `createListener()` specifically. That `PipeImpl` probes for the availability of Unix Domain Sockets by trying to bind to a unique temporary name. That call throws `IAE` when a non-default Java file system is installed while the probing code (`PipeImpl.createListener()`) only expects `UnsupportedOperationException` or `IOException`. >> The fix is to re-throw `UOE` instead of `IAE` in `UnixDomainSockets.genrateTempName()`. This is more consistent with the definition of the exception purpose ("requested operation is not supported"). So with this change, a loopback network socket will be used to implement a pipe on a non-default Java file system. Also, pipes do not rely on the default Java file system on other platforms (Linux, MacOS) as well. >> >> Tested by running `jtreg:test/jdk/java/nio` on Window, MacOS, and Linux. > > Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: > > Addressed PR comments: > > - renamed UNNAMEDHolder to UnnamedHolder, > - renamed getUNNAMED() to unnamed(), > - replaced catch IllegalArgumentException with a check for the provider. @Michael-Mc-Mahon The issue is assigned to you? Are you okay if @mkartashev takes it? The updated src changes looks okay. test/jdk/java/nio/channels/Selector is probably the right location for the test as this issue is about the wakeup mechanism. I'd prefer to use a test name that is a bit more consistent with the naming in the area. Something like CustomFileSystem.java might be okay. We can also change the test to use Selector.open() as that is the user-facing API. I assume @library /test/lib can be dropped from the test description as the test doesn't use common test infrastructure. ------------- PR: https://git.openjdk.java.net/jdk/pull/6722 From duke at openjdk.java.net Tue Dec 7 09:10:07 2021 From: duke at openjdk.java.net (Maxim Kartashev) Date: Tue, 7 Dec 2021 09:10:07 GMT Subject: RFR: 8274883: (se) Selector.open throws IAE when the default file system provider is changed to a custom provider [v3] In-Reply-To: References: Message-ID: <699XqslfVE-ziZTZ0srkCsEtjJ3YgS6nwV0He9meGS4=.d6627ae1-a459-4b4f-a531-031897cc53c9@github.com> > The gist of the problem: when a file system is specified via `-Djava.nio.file.spi.DefaultFileSystemProvider`, a call to `SelectorProvider.provider().openSelector()` ends with a throw on Windows. > > There are two distinct components to the problem: > > 1. `ExceptionInInitializerError` is thrown during the static initialization of the `UnixDomainSockets.UNNAMED` field even though it isn't used on this code path (see `UnixDomainSocketAddress.of()` that throws `IllegalArgumentException` if invoked on a path from a non-default file system). > This is fixed by lazy-initializing the static member `UNNAMED` of `UnixDomainSockets` so that this initialization doesn't throw unless actually used. > > 2. `IllegalArgumentException` is thrown by `UnixDomainSocketAddress.of()` later on when `ServerSocketChannel` tries to use Windows version of `PipeImpl` and its method `createListener()` specifically. That `PipeImpl` probes for the availability of Unix Domain Sockets by trying to bind to a unique temporary name. That call throws `IAE` when a non-default Java file system is installed while the probing code (`PipeImpl.createListener()`) only expects `UnsupportedOperationException` or `IOException`. > The fix is to re-throw `UOE` instead of `IAE` in `UnixDomainSockets.genrateTempName()`. This is more consistent with the definition of the exception purpose ("requested operation is not supported"). So with this change, a loopback network socket will be used to implement a pipe on a non-default Java file system. Also, pipes do not rely on the default Java file system on other platforms (Linux, MacOS) as well. > > Tested by running `jtreg:test/jdk/java/nio` on Window, MacOS, and Linux. Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: Moved the test to java/nio/channels/Selector and corrected according to comments ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6722/files - new: https://git.openjdk.java.net/jdk/pull/6722/files/8dcc88e3..123cd5d1 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6722&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6722&range=01-02 Stats: 279 lines in 3 files changed: 241 ins; 38 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/6722.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6722/head:pull/6722 PR: https://git.openjdk.java.net/jdk/pull/6722 From michaelm at openjdk.java.net Tue Dec 7 10:49:16 2021 From: michaelm at openjdk.java.net (Michael McMahon) Date: Tue, 7 Dec 2021 10:49:16 GMT Subject: RFR: 8274883: (se) Selector.open throws IAE when the default file system provider is changed to a custom provider [v3] In-Reply-To: <699XqslfVE-ziZTZ0srkCsEtjJ3YgS6nwV0He9meGS4=.d6627ae1-a459-4b4f-a531-031897cc53c9@github.com> References: <699XqslfVE-ziZTZ0srkCsEtjJ3YgS6nwV0He9meGS4=.d6627ae1-a459-4b4f-a531-031897cc53c9@github.com> Message-ID: On Tue, 7 Dec 2021 09:10:07 GMT, Maxim Kartashev wrote: >> The gist of the problem: when a file system is specified via `-Djava.nio.file.spi.DefaultFileSystemProvider`, a call to `SelectorProvider.provider().openSelector()` ends with a throw on Windows. >> >> There are two distinct components to the problem: >> >> 1. `ExceptionInInitializerError` is thrown during the static initialization of the `UnixDomainSockets.UNNAMED` field even though it isn't used on this code path (see `UnixDomainSocketAddress.of()` that throws `IllegalArgumentException` if invoked on a path from a non-default file system). >> This is fixed by lazy-initializing the static member `UNNAMED` of `UnixDomainSockets` so that this initialization doesn't throw unless actually used. >> >> 2. `IllegalArgumentException` is thrown by `UnixDomainSocketAddress.of()` later on when `ServerSocketChannel` tries to use Windows version of `PipeImpl` and its method `createListener()` specifically. That `PipeImpl` probes for the availability of Unix Domain Sockets by trying to bind to a unique temporary name. That call throws `IAE` when a non-default Java file system is installed while the probing code (`PipeImpl.createListener()`) only expects `UnsupportedOperationException` or `IOException`. >> The fix is to re-throw `UOE` instead of `IAE` in `UnixDomainSockets.genrateTempName()`. This is more consistent with the definition of the exception purpose ("requested operation is not supported"). So with this change, a loopback network socket will be used to implement a pipe on a non-default Java file system. Also, pipes do not rely on the default Java file system on other platforms (Linux, MacOS) as well. >> >> Tested by running `jtreg:test/jdk/java/nio` on Window, MacOS, and Linux. > > Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: > > Moved the test to java/nio/channels/Selector and corrected according to comments Thanks for taking this on. I can sponsor it when we are finished with the review. Would it be possible to refer to the TestProvider in the SelectorProvider/spi directory in the @build directive, instead of copying it over to the Selector directory? ------------- Changes requested by michaelm (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6722 From duke at openjdk.java.net Tue Dec 7 11:19:10 2021 From: duke at openjdk.java.net (Maxim Kartashev) Date: Tue, 7 Dec 2021 11:19:10 GMT Subject: RFR: 8274883: (se) Selector.open throws IAE when the default file system provider is changed to a custom provider [v3] In-Reply-To: References: <699XqslfVE-ziZTZ0srkCsEtjJ3YgS6nwV0He9meGS4=.d6627ae1-a459-4b4f-a531-031897cc53c9@github.com> Message-ID: On Tue, 7 Dec 2021 10:45:50 GMT, Michael McMahon wrote: > I can sponsor it when we are finished with the review. Thank you, much appreciated! > Would it be possible to refer to the TestProvider in the SelectorProvider/spi directory in the @build directive, instead of copying it over to the Selector directory? I looked for a way to do that, but couldn't find any. Besides, the new variant of the custom provider is much smaller and simpler than the "original" `TestProvider` and slightly better suits the purpose. So it's not a completely wasted effort (i.e. if things break in `TestProvider` for reasons unrelated to Unix domain sockets, the new test is less likely to break). ------------- PR: https://git.openjdk.java.net/jdk/pull/6722 From michaelm at openjdk.java.net Tue Dec 7 11:54:13 2021 From: michaelm at openjdk.java.net (Michael McMahon) Date: Tue, 7 Dec 2021 11:54:13 GMT Subject: RFR: 8274883: (se) Selector.open throws IAE when the default file system provider is changed to a custom provider [v3] In-Reply-To: References: <699XqslfVE-ziZTZ0srkCsEtjJ3YgS6nwV0He9meGS4=.d6627ae1-a459-4b4f-a531-031897cc53c9@github.com> Message-ID: On Tue, 7 Dec 2021 11:16:31 GMT, Maxim Kartashev wrote: > > I can sponsor it when we are finished with the review. > > Thank you, much appreciated! > > > Would it be possible to refer to the TestProvider in the SelectorProvider/spi directory in the @build directive, instead of copying it over to the Selector directory? > > I looked for a way to do that, but couldn't find any. Besides, the new variant of the custom provider is much smaller and simpler than the "original" `TestProvider` and slightly better suits the purpose. So it's not a completely wasted effort (i.e. if things break in `TestProvider` for reasons unrelated to Unix domain sockets, the new test is less likely to break). That's okay with me. It's useful to have it in the same directory anyway. I'm just running the patch through the test system here now. ------------- PR: https://git.openjdk.java.net/jdk/pull/6722 From lancea at openjdk.java.net Tue Dec 7 12:04:12 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Tue, 7 Dec 2021 12:04:12 GMT Subject: RFR: 8277361: java/nio/channels/Channels/ReadXBytes.java fails with OOM error In-Reply-To: References: Message-ID: On Tue, 7 Dec 2021 01:47:47 GMT, Brian Burkhalter wrote: > This request would increase the amount of heap memory that the test can use, and decrease execution time by not writing the entirety of test files. Looks good Brian Minor suggestion below test/jdk/java/nio/channels/Channels/ReadXBytes.java line 105: > 103: try (FileChannel fc = FileChannel.open(file, WRITE);) { > 104: long pos = 0L; > 105: if (length >= 2L*1024*1024*1024) { Maybe consider a constant or a comment for extra clarity ------------- Marked as reviewed by lancea (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6732 From alanb at openjdk.java.net Tue Dec 7 12:25:15 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 7 Dec 2021 12:25:15 GMT Subject: RFR: 8274883: (se) Selector.open throws IAE when the default file system provider is changed to a custom provider [v3] In-Reply-To: <699XqslfVE-ziZTZ0srkCsEtjJ3YgS6nwV0He9meGS4=.d6627ae1-a459-4b4f-a531-031897cc53c9@github.com> References: <699XqslfVE-ziZTZ0srkCsEtjJ3YgS6nwV0He9meGS4=.d6627ae1-a459-4b4f-a531-031897cc53c9@github.com> Message-ID: On Tue, 7 Dec 2021 09:10:07 GMT, Maxim Kartashev wrote: >> The gist of the problem: when a file system is specified via `-Djava.nio.file.spi.DefaultFileSystemProvider`, a call to `SelectorProvider.provider().openSelector()` ends with a throw on Windows. >> >> There are two distinct components to the problem: >> >> 1. `ExceptionInInitializerError` is thrown during the static initialization of the `UnixDomainSockets.UNNAMED` field even though it isn't used on this code path (see `UnixDomainSocketAddress.of()` that throws `IllegalArgumentException` if invoked on a path from a non-default file system). >> This is fixed by lazy-initializing the static member `UNNAMED` of `UnixDomainSockets` so that this initialization doesn't throw unless actually used. >> >> 2. `IllegalArgumentException` is thrown by `UnixDomainSocketAddress.of()` later on when `ServerSocketChannel` tries to use Windows version of `PipeImpl` and its method `createListener()` specifically. That `PipeImpl` probes for the availability of Unix Domain Sockets by trying to bind to a unique temporary name. That call throws `IAE` when a non-default Java file system is installed while the probing code (`PipeImpl.createListener()`) only expects `UnsupportedOperationException` or `IOException`. >> The fix is to re-throw `UOE` instead of `IAE` in `UnixDomainSockets.genrateTempName()`. This is more consistent with the definition of the exception purpose ("requested operation is not supported"). So with this change, a loopback network socket will be used to implement a pipe on a non-default Java file system. Also, pipes do not rely on the default Java file system on other platforms (Linux, MacOS) as well. >> >> Tested by running `jtreg:test/jdk/java/nio` on Window, MacOS, and Linux. > > Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: > > Moved the test to java/nio/channels/Selector and corrected according to comments Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6722 From alanb at openjdk.java.net Thu Dec 9 18:40:20 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 9 Dec 2021 18:40:20 GMT Subject: RFR: 8271079: JavaFileObject#toUri and multi-release jars [v2] In-Reply-To: References: Message-ID: On Thu, 9 Dec 2021 07:58:33 GMT, Christian Stein wrote: >> Prior to this PR, `toUri()` of class `ZipPath` in module `jdk.zipfs` and class `PathFileObject` in module `jdk.compiler` were always composed by base path names. Even for versioned entries of a multi-release JAR file. >> >> Now, a `URI` for an entry is composed of its real path names using an existing lookup function in the associated zip file system object. >> >> This PR also removes a superseded work around for [JDK-8134451](https://bugs.openjdk.java.net/browse/JDK-8134451). >> >> Fixes https://bugs.openjdk.java.net/browse/JDK-8271079 > > Christian Stein has updated the pull request incrementally with two additional commits since the last revision: > > - Clean up reproducer test case > - Keep helper in ZipFileSystem simple src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipPath.java line 211: > 209: > 210: private String getRealPath(byte[] resolvedPath) { > 211: byte[] path = zfs.lookupPath(resolvedPath); Can getRealPath be changed to be no-arg method that calls getResolvedPath and then does the lookup. I think that would be a clearer than assuming the argument is a resolved path. ------------- PR: https://git.openjdk.java.net/jdk/pull/6768 From cstein at openjdk.java.net Fri Dec 10 08:16:46 2021 From: cstein at openjdk.java.net (Christian Stein) Date: Fri, 10 Dec 2021 08:16:46 GMT Subject: RFR: 8271079: JavaFileObject#toUri and multi-release jars [v3] In-Reply-To: References: Message-ID: > Prior to this PR, `toUri()` of class `ZipPath` in module `jdk.zipfs` and class `PathFileObject` in module `jdk.compiler` were always composed by base path names. Even for versioned entries of a multi-release JAR file. > > Now, a `URI` for an entry is composed of its real path names using an existing lookup function in the associated zip file system object. > > This PR also removes a superseded work around for [JDK-8134451](https://bugs.openjdk.java.net/browse/JDK-8134451). > > Fixes https://bugs.openjdk.java.net/browse/JDK-8271079 Christian Stein has updated the pull request incrementally with one additional commit since the last revision: Turn `getRealPath()` into a no-arg helper method ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6768/files - new: https://git.openjdk.java.net/jdk/pull/6768/files/e62721a5..12e106c6 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6768&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6768&range=01-02 Stats: 5 lines in 1 file changed: 1 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/6768.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6768/head:pull/6768 PR: https://git.openjdk.java.net/jdk/pull/6768 From cstein at openjdk.java.net Fri Dec 10 08:16:51 2021 From: cstein at openjdk.java.net (Christian Stein) Date: Fri, 10 Dec 2021 08:16:51 GMT Subject: RFR: 8271079: JavaFileObject#toUri and multi-release jars [v2] In-Reply-To: References: Message-ID: On Thu, 9 Dec 2021 18:37:11 GMT, Alan Bateman wrote: >> Christian Stein has updated the pull request incrementally with two additional commits since the last revision: >> >> - Clean up reproducer test case >> - Keep helper in ZipFileSystem simple > > src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipPath.java line 211: > >> 209: >> 210: private String getRealPath(byte[] resolvedPath) { >> 211: byte[] path = zfs.lookupPath(resolvedPath); > > Can getRealPath be changed to be no-arg method that calls getResolvedPath and then does the lookup. I think that would be a clearer than assuming the argument is a resolved path. Done via https://github.com/openjdk/jdk/pull/6768/commits/12e106c6e69449f1fbde9e5b6a5ff3305c5de547 ------------- PR: https://git.openjdk.java.net/jdk/pull/6768 From alanb at openjdk.java.net Fri Dec 10 09:08:14 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 10 Dec 2021 09:08:14 GMT Subject: RFR: 8271079: JavaFileObject#toUri and multi-release jars [v3] In-Reply-To: References: Message-ID: <4GBNHwF6NlDHsQlgt4HM7Gatr4SStsQdn833oZFBRRo=.30c50bd4-e4ff-4eed-a1c1-5dad3b896e3f@github.com> On Fri, 10 Dec 2021 08:16:46 GMT, Christian Stein wrote: >> Prior to this PR, `toUri()` of class `ZipPath` in module `jdk.zipfs` and class `PathFileObject` in module `jdk.compiler` were always composed by base path names. Even for versioned entries of a multi-release JAR file. >> >> Now, a `URI` for an entry is composed of its real path names using an existing lookup function in the associated zip file system object. >> >> This PR also removes a superseded work around for [JDK-8134451](https://bugs.openjdk.java.net/browse/JDK-8134451). >> >> Fixes https://bugs.openjdk.java.net/browse/JDK-8271079 > > Christian Stein has updated the pull request incrementally with one additional commit since the last revision: > > Turn `getRealPath()` into a no-arg helper method The changes to zipfs in the latest revision look good ------------- PR: https://git.openjdk.java.net/jdk/pull/6768 From jjg at openjdk.java.net Fri Dec 10 16:40:19 2021 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Fri, 10 Dec 2021 16:40:19 GMT Subject: RFR: 8271079: JavaFileObject#toUri and multi-release jars [v3] In-Reply-To: References: Message-ID: <_0AxepENVOSu6YhuLVZQsyA3fJ3aTId_NtIpbrCGk0s=.e4b3fe0a-3cac-4b8a-b577-7516fc48091e@github.com> On Fri, 10 Dec 2021 08:16:46 GMT, Christian Stein wrote: >> Prior to this PR, `toUri()` of class `ZipPath` in module `jdk.zipfs` and class `PathFileObject` in module `jdk.compiler` were always composed by base path names. Even for versioned entries of a multi-release JAR file. >> >> Now, a `URI` for an entry is composed of its real path names using an existing lookup function in the associated zip file system object. >> >> This PR also removes a superseded work around for [JDK-8134451](https://bugs.openjdk.java.net/browse/JDK-8134451). >> >> Fixes https://bugs.openjdk.java.net/browse/JDK-8271079 > > Christian Stein has updated the pull request incrementally with one additional commit since the last revision: > > Turn `getRealPath()` into a no-arg helper method javac changes look good. I suggest adding this bug number to the test to check that URIs are not double-encoded. src/jdk.compiler/share/classes/com/sun/tools/javac/file/PathFileObject.java line 183: > 181: @Override @DefinedBy(Api.COMPILER) > 182: public URI toUri() { > 183: // Work around bug JDK-8134451: Nice to see the workaround go. Find the test that was added at the time of the workaround, and add this bug number (8271079) to the `@bug` line in that test, to mark that test as relevant to this issue. test/langtools/tools/javac/T8271079.java line 39: > 37: import java.util.*; > 38: import java.util.jar.JarEntry; > 39: import javax.tools.*; You could add an explicit `import java.util.spi.ToolProvider;` to remove the need for qualified names in the code test/langtools/tools/javac/T8271079.java line 58: > 56: testT8271079(mr); > 57: } finally { > 58: Files.deleteIfExists(mr); If you generate files in the test's current directory, it's not wrong to delete them but there's no need either, and you might be deleting useful debug evidence if the test fails. ------------- Marked as reviewed by jjg (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6768 From lancea at openjdk.java.net Fri Dec 10 19:43:10 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Fri, 10 Dec 2021 19:43:10 GMT Subject: RFR: 8271079: JavaFileObject#toUri and multi-release jars [v3] In-Reply-To: References: Message-ID: On Fri, 10 Dec 2021 08:16:46 GMT, Christian Stein wrote: >> Prior to this PR, `toUri()` of class `ZipPath` in module `jdk.zipfs` and class `PathFileObject` in module `jdk.compiler` were always composed by base path names. Even for versioned entries of a multi-release JAR file. >> >> Now, a `URI` for an entry is composed of its real path names using an existing lookup function in the associated zip file system object. >> >> This PR also removes a superseded work around for [JDK-8134451](https://bugs.openjdk.java.net/browse/JDK-8134451). >> >> Fixes https://bugs.openjdk.java.net/browse/JDK-8271079 > > Christian Stein has updated the pull request incrementally with one additional commit since the last revision: > > Turn `getRealPath()` into a no-arg helper method The changes look good. ------------- Marked as reviewed by lancea (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6768 From cstein at openjdk.java.net Sat Dec 11 11:03:10 2021 From: cstein at openjdk.java.net (Christian Stein) Date: Sat, 11 Dec 2021 11:03:10 GMT Subject: RFR: 8271079: JavaFileObject#toUri and multi-release jars [v3] In-Reply-To: <_0AxepENVOSu6YhuLVZQsyA3fJ3aTId_NtIpbrCGk0s=.e4b3fe0a-3cac-4b8a-b577-7516fc48091e@github.com> References: <_0AxepENVOSu6YhuLVZQsyA3fJ3aTId_NtIpbrCGk0s=.e4b3fe0a-3cac-4b8a-b577-7516fc48091e@github.com> Message-ID: On Fri, 10 Dec 2021 16:30:37 GMT, Jonathan Gibbons wrote: >> Christian Stein has updated the pull request incrementally with one additional commit since the last revision: >> >> Turn `getRealPath()` into a no-arg helper method > > test/langtools/tools/javac/T8271079.java line 39: > >> 37: import java.util.*; >> 38: import java.util.jar.JarEntry; >> 39: import javax.tools.*; > > You could add an explicit `import java.util.spi.ToolProvider;` to remove the need for qualified names in the code Such an import would break line 80: `ToolProvider.getSystemJavaCompiler()` as this one refers to `ToolProvider` in package `javax.tools`. ------------- PR: https://git.openjdk.java.net/jdk/pull/6768 From cstein at openjdk.java.net Sat Dec 11 11:22:10 2021 From: cstein at openjdk.java.net (Christian Stein) Date: Sat, 11 Dec 2021 11:22:10 GMT Subject: RFR: 8271079: JavaFileObject#toUri and multi-release jars [v3] In-Reply-To: <_0AxepENVOSu6YhuLVZQsyA3fJ3aTId_NtIpbrCGk0s=.e4b3fe0a-3cac-4b8a-b577-7516fc48091e@github.com> References: <_0AxepENVOSu6YhuLVZQsyA3fJ3aTId_NtIpbrCGk0s=.e4b3fe0a-3cac-4b8a-b577-7516fc48091e@github.com> Message-ID: On Fri, 10 Dec 2021 16:37:35 GMT, Jonathan Gibbons wrote: > javac changes look good. I suggest adding this bug number to the test to check that URIs are not double-encoded. I can't find [`8134451`](https://bugs.openjdk.java.net/browse/JDK-8134451) as noted in the now removed comment directly in an existing test, but [`8131067`](https://bugs.openjdk.java.net/browse/JDK-8131067) can be found in `ZipFSTester.java`: https://github.com/openjdk/jdk/blob/6eb6ec05fd4f80e11d0b052b58190bc8b53f4b11/test/jdk/jdk/nio/zipfs/ZipFSTester.java#L666 Thus, I'll add [`8271079`](https://bugs.openjdk.java.net/browse/JDK-8271079) to the already long list in `ZipFSTester.java`. https://github.com/openjdk/jdk/blob/6eb6ec05fd4f80e11d0b052b58190bc8b53f4b11/test/jdk/jdk/nio/zipfs/ZipFSTester.java#L74-L76 ------------- PR: https://git.openjdk.java.net/jdk/pull/6768 From cstein at openjdk.java.net Sat Dec 11 11:29:50 2021 From: cstein at openjdk.java.net (Christian Stein) Date: Sat, 11 Dec 2021 11:29:50 GMT Subject: RFR: 8271079: JavaFileObject#toUri and multi-release jars [v4] In-Reply-To: References: Message-ID: > Prior to this PR, `toUri()` of class `ZipPath` in module `jdk.zipfs` and class `PathFileObject` in module `jdk.compiler` were always composed by base path names. Even for versioned entries of a multi-release JAR file. > > Now, a `URI` for an entry is composed of its real path names using an existing lookup function in the associated zip file system object. > > This PR also removes a superseded work around for [JDK-8134451](https://bugs.openjdk.java.net/browse/JDK-8134451). > > Fixes https://bugs.openjdk.java.net/browse/JDK-8271079 Christian Stein has updated the pull request incrementally with one additional commit since the last revision: Add bug number `8271079` to ZipFS tester ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6768/files - new: https://git.openjdk.java.net/jdk/pull/6768/files/12e106c6..353cbd5d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6768&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6768&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6768.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6768/head:pull/6768 PR: https://git.openjdk.java.net/jdk/pull/6768 From cstein at openjdk.java.net Sat Dec 11 11:29:53 2021 From: cstein at openjdk.java.net (Christian Stein) Date: Sat, 11 Dec 2021 11:29:53 GMT Subject: RFR: 8271079: JavaFileObject#toUri and multi-release jars [v3] In-Reply-To: <_0AxepENVOSu6YhuLVZQsyA3fJ3aTId_NtIpbrCGk0s=.e4b3fe0a-3cac-4b8a-b577-7516fc48091e@github.com> References: <_0AxepENVOSu6YhuLVZQsyA3fJ3aTId_NtIpbrCGk0s=.e4b3fe0a-3cac-4b8a-b577-7516fc48091e@github.com> Message-ID: <-b8VPS5N4zJA2_uSs6f27yStd73jWoz3GE7aEOc3tVg=.c2a7273c-421e-44be-ac59-211039d9d9db@github.com> On Fri, 10 Dec 2021 16:33:33 GMT, Jonathan Gibbons wrote: >> Christian Stein has updated the pull request incrementally with one additional commit since the last revision: >> >> Turn `getRealPath()` into a no-arg helper method > > test/langtools/tools/javac/T8271079.java line 58: > >> 56: testT8271079(mr); >> 57: } finally { >> 58: Files.deleteIfExists(mr); > > If you generate files in the test's current directory, it's not wrong to delete them but there's no need either, and you might be deleting useful debug evidence if the test fails. Understood. ------------- PR: https://git.openjdk.java.net/jdk/pull/6768 From jjg at openjdk.java.net Sat Dec 11 16:36:14 2021 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Sat, 11 Dec 2021 16:36:14 GMT Subject: RFR: 8271079: JavaFileObject#toUri and multi-release jars [v4] In-Reply-To: References: Message-ID: On Sat, 11 Dec 2021 11:29:50 GMT, Christian Stein wrote: >> Prior to this PR, `toUri()` of class `ZipPath` in module `jdk.zipfs` and class `PathFileObject` in module `jdk.compiler` were always composed by base path names. Even for versioned entries of a multi-release JAR file. >> >> Now, a `URI` for an entry is composed of its real path names using an existing lookup function in the associated zip file system object. >> >> This PR also removes a superseded work around for [JDK-8134451](https://bugs.openjdk.java.net/browse/JDK-8134451). >> >> Fixes https://bugs.openjdk.java.net/browse/JDK-8271079 > > Christian Stein has updated the pull request incrementally with one additional commit since the last revision: > > Add bug number `8271079` to ZipFS tester Regarding the javac workaround, as best I can tell, the `createJarUri` code goes all the way back to the original file manager code in JDK 6. The workaround comment appeared in JDK 9, as part of the general conversion from the old File-based file manager to the new Path-based file manager. So yes, there does not appear to be a specific test for the workaround, and it's not clear it's worth bisecting the changes in the transition from JDK 8 to JDK 9 to investigate further. ------------- PR: https://git.openjdk.java.net/jdk/pull/6768 From duke at openjdk.java.net Sat Dec 11 16:56:29 2021 From: duke at openjdk.java.net (Markus KARG) Date: Sat, 11 Dec 2021 16:56:29 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v6] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains one commit: Implementing 8278268 Signed-off-by: Markus Karg ------------- Changes: https://git.openjdk.java.net/jdk/pull/6711/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=05 Stats: 155 lines in 2 files changed: 134 ins; 2 del; 19 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Sun Dec 12 14:37:35 2021 From: duke at openjdk.java.net (Markus KARG) Date: Sun, 12 Dec 2021 14:37:35 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v7] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with two additional commits since the last revision: - Simplified using checked function Signed-off-by: Markus Karg - Common implementation for all 2 GB tests Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/2579cef9..003c64c2 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=06 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=05-06 Stats: 81 lines in 1 file changed: 13 ins; 56 del; 12 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Sun Dec 12 14:46:19 2021 From: duke at openjdk.java.net (Markus KARG) Date: Sun, 12 Dec 2021 14:46:19 GMT Subject: RFR: 8265891: (ch) InputStream returned by Channels.newInputStream should override transferTo [v13] In-Reply-To: References: Message-ID: On Mon, 6 Dec 2021 07:07:03 GMT, Markus KARG wrote: >> Markus KARG has updated the pull request incrementally with two additional commits since the last revision: >> >> - Draft: Eliminated duplicate code using lambda expressions >> - Draft: Use blocking mode also for target channel > > Please keep this PR open. More commits will follow on the next weeks. > Looks promising, keep moving, @mkarg ! Thanks a lot, Laurent! I really appreciate! In fact, most of the work is going on in spin-off PRs, and some of it already is merged, as I explained on [Twitter](https://twitter.com/mkarg/status/1467064781978931206?s=20) and [Youtube](https://youtu.be/9Rjz3zRXoD4). :-) ------------- PR: https://git.openjdk.java.net/jdk/pull/4263 From alanb at openjdk.java.net Mon Dec 13 07:27:16 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Mon, 13 Dec 2021 07:27:16 GMT Subject: RFR: 8271079: JavaFileObject#toUri and multi-release jars [v4] In-Reply-To: References: Message-ID: On Sat, 11 Dec 2021 11:29:50 GMT, Christian Stein wrote: >> Prior to this PR, `toUri()` of class `ZipPath` in module `jdk.zipfs` and class `PathFileObject` in module `jdk.compiler` were always composed by base path names. Even for versioned entries of a multi-release JAR file. >> >> Now, a `URI` for an entry is composed of its real path names using an existing lookup function in the associated zip file system object. >> >> This PR also removes a superseded work around for [JDK-8134451](https://bugs.openjdk.java.net/browse/JDK-8134451). >> >> Fixes https://bugs.openjdk.java.net/browse/JDK-8271079 > > Christian Stein has updated the pull request incrementally with one additional commit since the last revision: > > Add bug number `8271079` to ZipFS tester I'll leave Jon or Lance to sponsor this. ------------- Marked as reviewed by alanb (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6768 From cstein at openjdk.java.net Mon Dec 13 12:03:21 2021 From: cstein at openjdk.java.net (Christian Stein) Date: Mon, 13 Dec 2021 12:03:21 GMT Subject: Integrated: 8271079: JavaFileObject#toUri and multi-release jars In-Reply-To: References: Message-ID: On Wed, 8 Dec 2021 15:26:23 GMT, Christian Stein wrote: > Prior to this PR, `toUri()` of class `ZipPath` in module `jdk.zipfs` and class `PathFileObject` in module `jdk.compiler` were always composed by base path names. Even for versioned entries of a multi-release JAR file. > > Now, a `URI` for an entry is composed of its real path names using an existing lookup function in the associated zip file system object. > > This PR also removes a superseded work around for [JDK-8134451](https://bugs.openjdk.java.net/browse/JDK-8134451). > > Fixes https://bugs.openjdk.java.net/browse/JDK-8271079 This pull request has now been integrated. Changeset: 23fd9f15 Author: Christian Stein Committer: Lance Andersen URL: https://git.openjdk.java.net/jdk/commit/23fd9f15da40cef00231380766158bc0fa537c38 Stats: 198 lines in 6 files changed: 177 ins; 18 del; 3 mod 8271079: JavaFileObject#toUri and multi-release jars Reviewed-by: jjg, lancea, alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/6768 From alanb at openjdk.java.net Mon Dec 13 16:53:09 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Mon, 13 Dec 2021 16:53:09 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v7] In-Reply-To: References: Message-ID: On Sun, 12 Dec 2021 14:37:35 GMT, Markus KARG wrote: >> This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. > > Markus KARG has updated the pull request incrementally with two additional commits since the last revision: > > - Simplified using checked function > > Signed-off-by: Markus Karg > - Common implementation for all 2 GB tests > > Signed-off-by: Markus Karg src/java.base/share/classes/sun/nio/ch/ChannelInputStream.java line 301: > 299: } > 300: bb.clear(); > 301: pos += bytesRead; I've been busy so only getting to this PR now. Since we are guaranteed that src is configured blocking then the only remaining reason for transferFrom to return 0 is when EOF is reached. Can we drop the temporary buffer and just replace the buffer in the block with: long n; while ((n = dst.transferFrom(src, pos, Long.MAX_VALUE)) > 0) { pos += n; } ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Mon Dec 13 17:26:39 2021 From: duke at openjdk.java.net (Markus KARG) Date: Mon, 13 Dec 2021 17:26:39 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with one additional commit since the last revision: Ignoring the case of intermittend zero reads as proposed by Alan Bateman Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/003c64c2..3c1d8af5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=07 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=06-07 Stats: 14 lines in 1 file changed: 0 ins; 10 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Mon Dec 13 17:26:44 2021 From: duke at openjdk.java.net (Markus KARG) Date: Mon, 13 Dec 2021 17:26:44 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v7] In-Reply-To: References: Message-ID: On Mon, 13 Dec 2021 16:50:21 GMT, Alan Bateman wrote: >> Markus KARG has updated the pull request incrementally with two additional commits since the last revision: >> >> - Simplified using checked function >> >> Signed-off-by: Markus Karg >> - Common implementation for all 2 GB tests >> >> Signed-off-by: Markus Karg > > src/java.base/share/classes/sun/nio/ch/ChannelInputStream.java line 301: > >> 299: } >> 300: bb.clear(); >> 301: pos += bytesRead; > > I've been busy so only getting to this PR now. > > Since we are guaranteed that src is configured blocking then the only remaining reason for transferFrom to return 0 is when EOF is reached. Can we drop the temporary buffer and just replace the buffer in the block with: > > long n; > while ((n = dst.transferFrom(src, pos, Long.MAX_VALUE)) > 0) { > pos += n; > } I would say "yes, for the channels being part of the JRE", and have adopted your proposal in https://github.com/openjdk/jdk/pull/6711/commits/3c1d8af58e65b6aa5d373c69f7e29e1f10010dc1. The only risk I see (and hence I wrote that originally complex code) is that *in theory* anybody is technically able to implement a **custom** `ReadableByteChannel` (as it is a public interface) which does return zero *intermittently* while the end of stream is not reached. `transferFrom` in that case will also return zero. Hence the loop would terminate *(possibly long) before* EOF is reached. So do we ignore this hypothetical case? ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From alanb at openjdk.java.net Tue Dec 14 09:45:14 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 14 Dec 2021 09:45:14 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v7] In-Reply-To: References: Message-ID: On Mon, 13 Dec 2021 17:22:51 GMT, Markus KARG wrote: >> src/java.base/share/classes/sun/nio/ch/ChannelInputStream.java line 301: >> >>> 299: } >>> 300: bb.clear(); >>> 301: pos += bytesRead; >> >> I've been busy so only getting to this PR now. >> >> Since we are guaranteed that src is configured blocking then the only remaining reason for transferFrom to return 0 is when EOF is reached. Can we drop the temporary buffer and just replace the buffer in the block with: >> >> long n; >> while ((n = dst.transferFrom(src, pos, Long.MAX_VALUE)) > 0) { >> pos += n; >> } > > I would say "yes, for the channels being part of the JRE", and have adopted your proposal in https://github.com/openjdk/jdk/pull/6711/commits/3c1d8af58e65b6aa5d373c69f7e29e1f10010dc1. The only risk I see (and hence I wrote that originally complex code) is that *in theory* anybody is technically able to implement a **custom** `ReadableByteChannel` (as it is a public interface) which does return zero *intermittently* while the end of stream is not reached. `transferFrom` in that case will also return zero. Hence the loop would terminate *(possibly long) before* EOF is reached. > > So do we ignore this hypothetical case? The read method is specified so that "It is guaranteed, however, that if a channel is in blocking mode and there is at least one byte remaining in the buffer then this method will block until at least one byte is read". So in blocking mode it means that read returns >=1 or -1/EOF. You are correct that a blocking of implementation of RBC that returns 0 would be problematic. In the case of transferFrom then it would assume the channel it at EOF. In other usages it may cause a loop as the caller might not expect 0. A completely broken implementation might return random values (-2 !!) so I don't think we can reliably defend against all cases. So I think the updated implementation in 3c1d8af5 is okay. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Tue Dec 14 12:25:09 2021 From: duke at openjdk.java.net (Markus KARG) Date: Tue, 14 Dec 2021 12:25:09 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v7] In-Reply-To: References: Message-ID: On Tue, 14 Dec 2021 09:42:10 GMT, Alan Bateman wrote: >> I would say "yes, for the channels being part of the JRE", and have adopted your proposal in https://github.com/openjdk/jdk/pull/6711/commits/3c1d8af58e65b6aa5d373c69f7e29e1f10010dc1. The only risk I see (and hence I wrote that originally complex code) is that *in theory* anybody is technically able to implement a **custom** `ReadableByteChannel` (as it is a public interface) which does return zero *intermittently* while the end of stream is not reached. `transferFrom` in that case will also return zero. Hence the loop would terminate *(possibly long) before* EOF is reached. >> >> So do we ignore this hypothetical case? > > The read method is specified so that "It is guaranteed, however, that if a channel is in blocking mode and there is at least one byte remaining in the buffer then this method will block until at least one byte is read". So in blocking mode it means that read returns >=1 or -1/EOF. You are correct that a blocking of implementation of RBC that returns 0 would be problematic. In the case of transferFrom then it would assume the channel it at EOF. In other usages it may cause a loop as the caller might not expect 0. A completely broken implementation might return random values (-2 !!) so I don't think we can reliably defend against all cases. So I think the updated implementation in 3c1d8af5 is okay. Ah, right, seems I missed that section about the one-byte-guarantee. Great, so I kindly request approval of this PR! ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From phedlin at openjdk.java.net Tue Dec 14 14:56:32 2021 From: phedlin at openjdk.java.net (Patric Hedlin) Date: Tue, 14 Dec 2021 14:56:32 GMT Subject: [jdk18] RFR: 8274243: Implement fast-path for ASCII-compatible CharsetEncoders on aarch64 Message-ID: Implementation of ISO/ASCII char set encoding, extending current implementation with ASCII encoding support. Implementation with slight focus on balance between footprint and efficiency, trying to utilise a dual SIMD path (e.g. Neoverse N1) for the additional Ascii-check and avoid performance loss in the ISO-only case. - Interleaved ISO and ASCII check code. - Avoid 'umaxv' in the ISO main flow. - Using post inc in main loop. - Retain 8-char loop. - Removing conditional prefetch (no upside). - Adding ISO-8859-1 to encode-decode benchmark. Testing: tier1-3 The revised version compares like this (master vs. update). Benchmark (size) (type) Mode Cnt Score Error Units CharsetEncodeDecode.encode 16384 UTF-8 avgt 30 17.920 ? 0.229 us/op CharsetEncodeDecode.encode 16384 BIG5 avgt 30 18.867 ? 0.356 us/op CharsetEncodeDecode.encode 16384 ISO-8859-15 avgt 30 17.419 ? 0.220 us/op CharsetEncodeDecode.encode 16384 ISO-8859-1 avgt 30 6.200 ? 0.134 us/op CharsetEncodeDecode.encode 16384 ASCII avgt 30 17.149 ? 0.219 us/op CharsetEncodeDecode.encode 16384 UTF-16 avgt 30 135.115 ? 1.440 us/op Benchmark (size) (type) Mode Cnt Score Error Units CharsetEncodeDecode.encode 16384 UTF-8 avgt 30 9.018 ? 0.179 us/op CharsetEncodeDecode.encode 16384 BIG5 avgt 30 10.550 ? 0.470 us/op CharsetEncodeDecode.encode 16384 ISO-8859-15 avgt 30 8.843 ? 0.187 us/op CharsetEncodeDecode.encode 16384 ISO-8859-1 avgt 30 6.406 ? 0.155 us/op CharsetEncodeDecode.encode 16384 ASCII avgt 30 8.822 ? 0.173 us/op CharsetEncodeDecode.encode 16384 UTF-16 avgt 30 135.195 ? 1.432 us/op ------------- Commit messages: - 8274243: Implement fast-path for ASCII-compatible CharsetEncoders on aarch64 Changes: https://git.openjdk.java.net/jdk18/pull/20/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk18&pr=20&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8274243 Stats: 256 lines in 6 files changed: 126 ins; 90 del; 40 mod Patch: https://git.openjdk.java.net/jdk18/pull/20.diff Fetch: git fetch https://git.openjdk.java.net/jdk18 pull/20/head:pull/20 PR: https://git.openjdk.java.net/jdk18/pull/20 From phedlin at openjdk.java.net Tue Dec 14 14:56:32 2021 From: phedlin at openjdk.java.net (Patric Hedlin) Date: Tue, 14 Dec 2021 14:56:32 GMT Subject: [jdk18] RFR: 8274243: Implement fast-path for ASCII-compatible CharsetEncoders on aarch64 In-Reply-To: References: Message-ID: On Tue, 14 Dec 2021 10:45:28 GMT, Patric Hedlin wrote: > Implementation of ISO/ASCII char set encoding, extending current implementation with ASCII encoding support. > > Implementation with slight focus on balance between footprint and efficiency, trying to utilise a dual SIMD path (e.g. Neoverse N1) for the additional Ascii-check and avoid performance loss in the ISO-only case. > > - Interleaved ISO and ASCII check code. > - Avoid 'umaxv' in the ISO main flow. > - Using post inc in main loop. > - Retain 8-char loop. > - Removing conditional prefetch (no upside). > - Adding ISO-8859-1 to encode-decode benchmark. > > Testing: tier1-3 > > The revised version compares like this (master vs. update). > > Benchmark (size) (type) Mode Cnt Score Error Units > CharsetEncodeDecode.encode 16384 UTF-8 avgt 30 17.920 ? 0.229 us/op > CharsetEncodeDecode.encode 16384 BIG5 avgt 30 18.867 ? 0.356 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-15 avgt 30 17.419 ? 0.220 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-1 avgt 30 6.200 ? 0.134 us/op > CharsetEncodeDecode.encode 16384 ASCII avgt 30 17.149 ? 0.219 us/op > CharsetEncodeDecode.encode 16384 UTF-16 avgt 30 135.115 ? 1.440 us/op > > > Benchmark (size) (type) Mode Cnt Score Error Units > CharsetEncodeDecode.encode 16384 UTF-8 avgt 30 9.018 ? 0.179 us/op > CharsetEncodeDecode.encode 16384 BIG5 avgt 30 10.550 ? 0.470 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-15 avgt 30 8.843 ? 0.187 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-1 avgt 30 6.406 ? 0.155 us/op > CharsetEncodeDecode.encode 16384 ASCII avgt 30 8.822 ? 0.173 us/op > CharsetEncodeDecode.encode 16384 UTF-16 avgt 30 135.195 ? 1.432 us/op Benchmarks, master vs. update (ran on Aurora/Ampere Altra): openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:16384-type:ASCII ........77.55% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:16384-type:BIG5 .........76.71% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:16384-type:ISO_8859_1 ...-2.31% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:16384-type:ISO_8859_15 ..75.58% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:16384-type:UTF_16 ....... 1.04% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:16384-type:UTF_8 ........76.90% Note that ISO-8859-1 compares with the old intrinsic implementation (essentially the same) and that UTF-16 does not utilise the intrinsic. Runs that show the more pessimistic speed-up, when processing 2^n - 1 chars. openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:2047-type:ASCII .........72.97% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:2047-type:BIG5 ..........64.46% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:2047-type:ISO_8859_1 ....-1.67% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:2047-type:ISO_8859_15 ...70.85% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:2047-type:UTF_16 ........-4.60% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:2047-type:UTF_8 .........70.44% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:511-type:ASCII ..........60.35% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:511-type:BIG5 ...........52.61% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:511-type:ISO_8859_1 ..... 1.75% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:511-type:ISO_8859_15 ....61.45% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:511-type:UTF_16 .........-1.01% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:511-type:UTF_8 ..........59.46% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:255-type:ASCII ..........54.26% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:255-type:BIG5 ...........42.82% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:255-type:ISO_8859_1 .....-0.54% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:255-type:ISO_8859_15 ....64.86% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:255-type:UTF_16 .........-0.09% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:255-type:UTF_8 ..........60.44% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:127-type:ASCII ..........51.51% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:127-type:BIG5 ...........46.54% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:127-type:ISO_8859_1 .....-0.32% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:127-type:ISO_8859_15 ....56.48% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:127-type:UTF_16 ......... 0.44% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:127-type:UTF_8 ..........54.84% Runs to illustrate the threshold effect between the loops in the implementation. openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:32-type:ASCII ...........32.30% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:32-type:BIG5 ............31.93% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:32-type:ISO_8859_1 ......-0.02% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:32-type:ISO_8859_15 .....37.92% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:32-type:UTF_16 .......... 4.45% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:32-type:UTF_8 ...........40.35% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:31-type:ASCII ...........20.06% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:31-type:BIG5 ............21.64% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:31-type:ISO_8859_1 ......-1.13% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:31-type:ISO_8859_15 .....27.04% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:31-type:UTF_16 .......... 1.20% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:31-type:UTF_8 ...........24.72% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:16-type:ASCII ...........19.37% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:16-type:BIG5 ............20.20% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:16-type:ISO_8859_1 ......-1.01% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:16-type:ISO_8859_15 .....29.16% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:16-type:UTF_16 .......... 0.34% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:16-type:UTF_8 ...........25.35% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:15-type:ASCII ...........13.03% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:15-type:BIG5 ............13.74% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:15-type:ISO_8859_1 ......-0.13% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:15-type:ISO_8859_15 .....19.26% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:15-type:UTF_16 .......... 0.78% openjdk.bench.java.nio.CharsetEncodeDecode.encode-size:15-type:UTF_8 ...........17.70% Using the microbenchmarks provided by @carterkozak here: https://github.com/carterkozak/stringbuilder-encoding-performance, comparing master vs. update as follows: Benchmark (charsetName) (message) (timesToAppend) Mode Cnt Score Error Units EncoderBenchmarks.charsetEncoder UTF-8 This is a simple ASCII message 3 avgt 4 151.025 ? 28.111 ns/op EncoderBenchmarks.charsetEncoder UTF-8 This is a message with unicode ?? 3 avgt 4 323.254 ? 5.648 ns/op EncoderBenchmarks.charsetEncoderWithAllocation UTF-8 This is a simple ASCII message 3 avgt 4 244.375 ? 98.844 ns/op EncoderBenchmarks.charsetEncoderWithAllocation UTF-8 This is a message with unicode ?? 3 avgt 4 405.415 ? 5.947 ns/op EncoderBenchmarks.charsetEncoderWithAllocationWrappingBuilder UTF-8 This is a simple ASCII message 3 avgt 4 728.172 ? 22.419 ns/op EncoderBenchmarks.charsetEncoderWithAllocationWrappingBuilder UTF-8 This is a message with unicode ?? 3 avgt 4 859.015 ? 90.541 ns/op EncoderBenchmarks.toStringGetBytes UTF-8 This is a simple ASCII message 3 avgt 4 117.044 ? 11.484 ns/op EncoderBenchmarks.toStringGetBytes UTF-8 This is a message with unicode ?? 3 avgt 4 483.399 ? 38.614 ns/op Benchmark (charsetName) (message) (timesToAppend) Mode Cnt Score Error Units EncoderBenchmarks.charsetEncoder UTF-8 This is a simple ASCII message 3 avgt 4 113.954 ? 7.657 ns/op EncoderBenchmarks.charsetEncoder UTF-8 This is a message with unicode ?? 3 avgt 4 353.266 ? 10.124 ns/op EncoderBenchmarks.charsetEncoderWithAllocation UTF-8 This is a simple ASCII message 3 avgt 4 196.643 ? 52.954 ns/op EncoderBenchmarks.charsetEncoderWithAllocation UTF-8 This is a message with unicode ?? 3 avgt 4 429.157 ? 11.506 ns/op EncoderBenchmarks.charsetEncoderWithAllocationWrappingBuilder UTF-8 This is a simple ASCII message 3 avgt 4 728.138 ? 34.898 ns/op EncoderBenchmarks.charsetEncoderWithAllocationWrappingBuilder UTF-8 This is a message with unicode ?? 3 avgt 4 859.697 ? 61.397 ns/op EncoderBenchmarks.toStringGetBytes UTF-8 This is a simple ASCII message 3 avgt 4 117.269 ? 6.623 ns/op EncoderBenchmarks.toStringGetBytes UTF-8 This is a message with unicode ?? 3 avgt 4 491.559 ? 68.169 ns/op Note: The above was ran on a local dev-machine typically producing less than _perfectly_ consistent results. ------------- PR: https://git.openjdk.java.net/jdk18/pull/20 From aph at openjdk.java.net Wed Dec 15 10:40:00 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Wed, 15 Dec 2021 10:40:00 GMT Subject: [jdk18] RFR: 8274243: Implement fast-path for ASCII-compatible CharsetEncoders on aarch64 In-Reply-To: References: Message-ID: On Tue, 14 Dec 2021 10:45:28 GMT, Patric Hedlin wrote: > Implementation of ISO/ASCII char set encoding, extending current implementation with ASCII encoding support. > > Implementation with slight focus on balance between footprint and efficiency, trying to utilise a dual SIMD path (e.g. Neoverse N1) for the additional Ascii-check and avoid performance loss in the ISO-only case. > > - Interleaved ISO and ASCII check code. > - Avoid 'umaxv' in the ISO main flow. > - Using post inc in main loop. > - Retain 8-char loop. > - Removing conditional prefetch (no upside). > - Adding ISO-8859-1 to encode-decode benchmark. > > Testing: tier1-6 > > The revised version compares like this (master vs. update). > > Benchmark (size) (type) Mode Cnt Score Error Units > CharsetEncodeDecode.encode 16384 UTF-8 avgt 30 17.920 ? 0.229 us/op > CharsetEncodeDecode.encode 16384 BIG5 avgt 30 18.867 ? 0.356 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-15 avgt 30 17.419 ? 0.220 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-1 avgt 30 6.200 ? 0.134 us/op > CharsetEncodeDecode.encode 16384 ASCII avgt 30 17.149 ? 0.219 us/op > CharsetEncodeDecode.encode 16384 UTF-16 avgt 30 135.115 ? 1.440 us/op > > > Benchmark (size) (type) Mode Cnt Score Error Units > CharsetEncodeDecode.encode 16384 UTF-8 avgt 30 9.018 ? 0.179 us/op > CharsetEncodeDecode.encode 16384 BIG5 avgt 30 10.550 ? 0.470 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-15 avgt 30 8.843 ? 0.187 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-1 avgt 30 6.406 ? 0.155 us/op > CharsetEncodeDecode.encode 16384 ASCII avgt 30 8.822 ? 0.173 us/op > CharsetEncodeDecode.encode 16384 UTF-16 avgt 30 135.195 ? 1.432 us/op I don't think this should go straight into the 18 release branch. It looks OK for mainline. ------------- PR: https://git.openjdk.java.net/jdk18/pull/20 From phedlin at openjdk.java.net Wed Dec 15 14:12:59 2021 From: phedlin at openjdk.java.net (Patric Hedlin) Date: Wed, 15 Dec 2021 14:12:59 GMT Subject: [jdk18] RFR: 8274243: Implement fast-path for ASCII-compatible CharsetEncoders on aarch64 In-Reply-To: References: Message-ID: <5EmZGX3oIMMll_eKwWVNV1xkPCQRwXjnb_QMRb2tcjA=.c00aa130-d0ef-49bb-b513-32e4709b28cb@github.com> On Wed, 15 Dec 2021 10:37:04 GMT, Andrew Haley wrote: > I don't think this should go straight into the 18 release branch. It looks OK for mainline. Any particular reason it should not be included in JDK-18? ------------- PR: https://git.openjdk.java.net/jdk18/pull/20 From alanb at openjdk.java.net Wed Dec 15 14:28:04 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 15 Dec 2021 14:28:04 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v7] In-Reply-To: References: Message-ID: On Tue, 14 Dec 2021 12:22:26 GMT, Markus KARG wrote: > Ah, right, seems I missed that section about the one-byte-guarantee. Great, so I kindly request approval of this PR! I think Lance is going to help on this. We need to study the test update closely as this test has been causing a lot of issues in our CI (timeouts, hangs, ...). I think the main issue with the test right now is JDK-8278369 and it may only be on Windows. I haven't had time myself to study it closely. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Wed Dec 15 14:48:59 2021 From: duke at openjdk.java.net (Markus KARG) Date: Wed, 15 Dec 2021 14:48:59 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v7] In-Reply-To: References: Message-ID: <2oiV8FHSRm3dC_QOJhFWqKvz8HqiMzdo3tYqzESZdS8=.a50a13f6-a469-4dbb-a57c-11d5508d07c2@github.com> On Wed, 15 Dec 2021 14:25:10 GMT, Alan Bateman wrote: >> Ah, right, seems I missed that section about the one-byte-guarantee. Great, so I kindly request approval of this PR! > >> Ah, right, seems I missed that section about the one-byte-guarantee. Great, so I kindly request approval of this PR! > > I think Lance is going to help on this. We need to study the test update closely as this test has been causing a lot of issues in our CI (timeouts, hangs, ...). I think the main issue with the test right now is JDK-8278369 and it may only be on Windows. I haven't had time myself to study it closely. The test update only allows to wrap the input file stream with a buffer input stream to skip the special file stream case and walk directly into the general stream case. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From aph at openjdk.java.net Wed Dec 15 15:03:08 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Wed, 15 Dec 2021 15:03:08 GMT Subject: [jdk18] RFR: 8274243: Implement fast-path for ASCII-compatible CharsetEncoders on aarch64 In-Reply-To: <5EmZGX3oIMMll_eKwWVNV1xkPCQRwXjnb_QMRb2tcjA=.c00aa130-d0ef-49bb-b513-32e4709b28cb@github.com> References: <5EmZGX3oIMMll_eKwWVNV1xkPCQRwXjnb_QMRb2tcjA=.c00aa130-d0ef-49bb-b513-32e4709b28cb@github.com> Message-ID: <2uKBlDzz5yUtlKD3lUXJj7nX1IW620fZomAKHzHEXvg=.da646d0d-a28f-4605-ab24-93a245a85e9b@github.com> On Wed, 15 Dec 2021 14:09:25 GMT, Patric Hedlin wrote: > > I don't think this should go straight into the 18 release branch. It looks OK for mainline. > > Any particular reason it should not be included in JDK-18? We're in RDP1 since December 9. This means that any enhancement is covered by the Late-Enhancement Request Process. https://openjdk.java.net/jeps/3#Late-Enhancement-Request-Process . I don't think this patch is urgent enough for that. ------------- PR: https://git.openjdk.java.net/jdk18/pull/20 From phedlin at openjdk.java.net Wed Dec 15 15:11:08 2021 From: phedlin at openjdk.java.net (Patric Hedlin) Date: Wed, 15 Dec 2021 15:11:08 GMT Subject: [jdk18] RFR: 8274243: Implement fast-path for ASCII-compatible CharsetEncoders on aarch64 In-Reply-To: <2uKBlDzz5yUtlKD3lUXJj7nX1IW620fZomAKHzHEXvg=.da646d0d-a28f-4605-ab24-93a245a85e9b@github.com> References: <5EmZGX3oIMMll_eKwWVNV1xkPCQRwXjnb_QMRb2tcjA=.c00aa130-d0ef-49bb-b513-32e4709b28cb@github.com> <2uKBlDzz5yUtlKD3lUXJj7nX1IW620fZomAKHzHEXvg=.da646d0d-a28f-4605-ab24-93a245a85e9b@github.com> Message-ID: On Wed, 15 Dec 2021 14:59:47 GMT, Andrew Haley wrote: > We're in RDP1 since December 9. This means that any enhancement is covered by the Late-Enhancement Request Process. https://openjdk.java.net/jeps/3#Late-Enhancement-Request-Process . I don't think this patch is urgent enough for that. It has been classified as a performance regression (bug) in line with the x86 issue (JDK-8274242). Do you mean we should change this _now_? Aarch64 would be the only platform not to address the issue in JDK-18. ------------- PR: https://git.openjdk.java.net/jdk18/pull/20 From aph at openjdk.java.net Wed Dec 15 15:27:06 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Wed, 15 Dec 2021 15:27:06 GMT Subject: [jdk18] RFR: 8274243: Implement fast-path for ASCII-compatible CharsetEncoders on aarch64 In-Reply-To: References: <5EmZGX3oIMMll_eKwWVNV1xkPCQRwXjnb_QMRb2tcjA=.c00aa130-d0ef-49bb-b513-32e4709b28cb@github.com> <2uKBlDzz5yUtlKD3lUXJj7nX1IW620fZomAKHzHEXvg=.da646d0d-a28f-4605-ab24-93a245a85e9b@github.com> Message-ID: On Wed, 15 Dec 2021 15:07:41 GMT, Patric Hedlin wrote: > > We're in RDP1 since December 9. This means that any enhancement is covered by the Late-Enhancement Request Process. https://openjdk.java.net/jeps/3#Late-Enhancement-Request-Process . I don't think this patch is urgent enough for that. > > It has been classified as a performance regression (bug) in line with the x86 issue (JDK-8274242). Do you mean we should change this _now_? Aarch64 would be the only platform not to address the issue in JDK-18. I see your point, and that is a significant difference. It's unfortunate that AArch64 got this patch late in the process, but as it's a bug, and Java performance isn't supposed to regress in a release, it does make sense to fix it. Having said that, I've had a lot of bad experiences with late patches, and it'd be a dreadful shame if we broke AArch64 string handling. I suppose the best way to proceed with this is to have two AArch64 reviewers go through the patch instruction by instruction, just to make sure the test covers all corner cases. Thanks. ------------- PR: https://git.openjdk.java.net/jdk18/pull/20 From rriggs at openjdk.java.net Wed Dec 15 16:12:56 2021 From: rriggs at openjdk.java.net (Roger Riggs) Date: Wed, 15 Dec 2021 16:12:56 GMT Subject: [jdk18] RFR: 8274243: Implement fast-path for ASCII-compatible CharsetEncoders on aarch64 In-Reply-To: References: Message-ID: On Tue, 14 Dec 2021 10:45:28 GMT, Patric Hedlin wrote: > Implementation of ISO/ASCII char set encoding, extending current implementation with ASCII encoding support. > > The motivation is found in the original x86 issue ([JDK-8274242](https://bugs.openjdk.java.net/browse/JDK-8274242)). > > Implementation with slight focus on balance between footprint and efficiency, trying to utilise a dual SIMD path (e.g. Neoverse N1) for the additional Ascii-check and avoid performance loss in the ISO-only case. > > - Interleaved ISO and ASCII check code. > - Avoid 'umaxv' in the ISO main flow. > - Using post inc in main loop. > - Retain 8-char loop. > - Removing conditional prefetch (no upside). > - Adding ISO-8859-1 to encode-decode benchmark. > > Testing: tier1-6 > > The revised version compares like this (master vs. update). > > Benchmark (size) (type) Mode Cnt Score Error Units > CharsetEncodeDecode.encode 16384 UTF-8 avgt 30 17.920 ? 0.229 us/op > CharsetEncodeDecode.encode 16384 BIG5 avgt 30 18.867 ? 0.356 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-15 avgt 30 17.419 ? 0.220 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-1 avgt 30 6.200 ? 0.134 us/op > CharsetEncodeDecode.encode 16384 ASCII avgt 30 17.149 ? 0.219 us/op > CharsetEncodeDecode.encode 16384 UTF-16 avgt 30 135.115 ? 1.440 us/op > > > Benchmark (size) (type) Mode Cnt Score Error Units > CharsetEncodeDecode.encode 16384 UTF-8 avgt 30 9.018 ? 0.179 us/op > CharsetEncodeDecode.encode 16384 BIG5 avgt 30 10.550 ? 0.470 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-15 avgt 30 8.843 ? 0.187 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-1 avgt 30 6.406 ? 0.155 us/op > CharsetEncodeDecode.encode 16384 ASCII avgt 30 8.822 ? 0.173 us/op > CharsetEncodeDecode.encode 16384 UTF-16 avgt 30 135.195 ? 1.432 us/op Its viable to commit to the main line and allow it to have some bake time before requesting it to be backported. That would allow some time to build confidence about the change; it might not make the first JDK 18 release but would come in later. There is no problem requesting approval for the change but it should go through that process. ------------- PR: https://git.openjdk.java.net/jdk18/pull/20 From phedlin at openjdk.java.net Thu Dec 16 11:37:02 2021 From: phedlin at openjdk.java.net (Patric Hedlin) Date: Thu, 16 Dec 2021 11:37:02 GMT Subject: [jdk18] RFR: 8274243: Implement fast-path for ASCII-compatible CharsetEncoders on aarch64 In-Reply-To: References: Message-ID: On Tue, 14 Dec 2021 10:45:28 GMT, Patric Hedlin wrote: > Implementation of ISO/ASCII char set encoding, extending current implementation with ASCII encoding support. > > The motivation is found in the original x86 issue ([JDK-8274242](https://bugs.openjdk.java.net/browse/JDK-8274242)). > > Implementation with slight focus on balance between footprint and efficiency, trying to utilise a dual SIMD path (e.g. Neoverse N1) for the additional Ascii-check and avoid performance loss in the ISO-only case. > > - Interleaved ISO and ASCII check code. > - Avoid 'umaxv' in the ISO main flow. > - Using post inc in main loop. > - Retain 8-char loop. > - Removing conditional prefetch (no upside). > - Adding ISO-8859-1 to encode-decode benchmark. > > Testing: tier1-6 > > The revised version compares like this (master vs. update). > > Benchmark (size) (type) Mode Cnt Score Error Units > CharsetEncodeDecode.encode 16384 UTF-8 avgt 30 17.920 ? 0.229 us/op > CharsetEncodeDecode.encode 16384 BIG5 avgt 30 18.867 ? 0.356 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-15 avgt 30 17.419 ? 0.220 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-1 avgt 30 6.200 ? 0.134 us/op > CharsetEncodeDecode.encode 16384 ASCII avgt 30 17.149 ? 0.219 us/op > CharsetEncodeDecode.encode 16384 UTF-16 avgt 30 135.115 ? 1.440 us/op > > > Benchmark (size) (type) Mode Cnt Score Error Units > CharsetEncodeDecode.encode 16384 UTF-8 avgt 30 9.018 ? 0.179 us/op > CharsetEncodeDecode.encode 16384 BIG5 avgt 30 10.550 ? 0.470 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-15 avgt 30 8.843 ? 0.187 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-1 avgt 30 6.406 ? 0.155 us/op > CharsetEncodeDecode.encode 16384 ASCII avgt 30 8.822 ? 0.173 us/op > CharsetEncodeDecode.encode 16384 UTF-16 avgt 30 135.195 ? 1.432 us/op So this will obviously be prolonged. Closing and moving to 19. ------------- PR: https://git.openjdk.java.net/jdk18/pull/20 From phedlin at openjdk.java.net Thu Dec 16 11:37:03 2021 From: phedlin at openjdk.java.net (Patric Hedlin) Date: Thu, 16 Dec 2021 11:37:03 GMT Subject: [jdk18] Withdrawn: 8274243: Implement fast-path for ASCII-compatible CharsetEncoders on aarch64 In-Reply-To: References: Message-ID: On Tue, 14 Dec 2021 10:45:28 GMT, Patric Hedlin wrote: > Implementation of ISO/ASCII char set encoding, extending current implementation with ASCII encoding support. > > The motivation is found in the original x86 issue ([JDK-8274242](https://bugs.openjdk.java.net/browse/JDK-8274242)). > > Implementation with slight focus on balance between footprint and efficiency, trying to utilise a dual SIMD path (e.g. Neoverse N1) for the additional Ascii-check and avoid performance loss in the ISO-only case. > > - Interleaved ISO and ASCII check code. > - Avoid 'umaxv' in the ISO main flow. > - Using post inc in main loop. > - Retain 8-char loop. > - Removing conditional prefetch (no upside). > - Adding ISO-8859-1 to encode-decode benchmark. > > Testing: tier1-6 > > The revised version compares like this (master vs. update). > > Benchmark (size) (type) Mode Cnt Score Error Units > CharsetEncodeDecode.encode 16384 UTF-8 avgt 30 17.920 ? 0.229 us/op > CharsetEncodeDecode.encode 16384 BIG5 avgt 30 18.867 ? 0.356 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-15 avgt 30 17.419 ? 0.220 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-1 avgt 30 6.200 ? 0.134 us/op > CharsetEncodeDecode.encode 16384 ASCII avgt 30 17.149 ? 0.219 us/op > CharsetEncodeDecode.encode 16384 UTF-16 avgt 30 135.115 ? 1.440 us/op > > > Benchmark (size) (type) Mode Cnt Score Error Units > CharsetEncodeDecode.encode 16384 UTF-8 avgt 30 9.018 ? 0.179 us/op > CharsetEncodeDecode.encode 16384 BIG5 avgt 30 10.550 ? 0.470 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-15 avgt 30 8.843 ? 0.187 us/op > CharsetEncodeDecode.encode 16384 ISO-8859-1 avgt 30 6.406 ? 0.155 us/op > CharsetEncodeDecode.encode 16384 ASCII avgt 30 8.822 ? 0.173 us/op > CharsetEncodeDecode.encode 16384 UTF-16 avgt 30 135.195 ? 1.432 us/op This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.java.net/jdk18/pull/20 From aph at openjdk.java.net Thu Dec 16 13:18:59 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Thu, 16 Dec 2021 13:18:59 GMT Subject: [jdk18] RFR: 8274243: Implement fast-path for ASCII-compatible CharsetEncoders on aarch64 In-Reply-To: References: Message-ID: <9yDa1xxnZh3yLMjNhmxlf4lPduEbNKWVKDJdNQ3_-s4=.4fd7a715-8258-4605-9f6f-c7865db99571@github.com> On Thu, 16 Dec 2021 11:33:11 GMT, Patric Hedlin wrote: > So this will obviously be prolonged. Closing and moving to 19. That sounds right. Let's get it into mainline soon, and we can do a backport to 18.1. ------------- PR: https://git.openjdk.java.net/jdk18/pull/20 From lancea at openjdk.java.net Thu Dec 16 15:42:56 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Thu, 16 Dec 2021 15:42:56 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: On Mon, 13 Dec 2021 17:26:39 GMT, Markus KARG wrote: >> This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > Ignoring the case of intermittend zero reads as proposed by Alan Bateman > > Signed-off-by: Markus Karg We will have to separate out the added changes into its own test as we are failing sporadically on some of the windows boxes due to the execution time. There is also further cleanup that can be done to existing test. Attached is an example diff based on the proposed changes in this PR [8278268-TransferTo-Cleanup.patch.txt](https://github.com/openjdk/jdk/files/7728608/8278268-TransferTo-Cleanup.patch.txt) ------------- Changes requested by lancea (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Thu Dec 16 16:11:56 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 16 Dec 2021 16:11:56 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: <4YP2VV-VMRwyj4WGI9cmaDIDmwwfdBWD3tkZdI_Cv1Y=.cc61c6b1-143d-4803-9e3f-1aa831125855@github.com> On Mon, 13 Dec 2021 17:26:39 GMT, Markus KARG wrote: >> This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > Ignoring the case of intermittend zero reads as proposed by Alan Bateman > > Signed-off-by: Markus Karg Is the cleanup essential for this PR? ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Thu Dec 16 16:30:58 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 16 Dec 2021 16:30:58 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: <1WeoNTsIGT9RN8XF4bxMILoBMiz6rOemX3kXP9eHW-c=.64cc066d-d3fa-4db1-bc64-3772394d22cf@github.com> On Thu, 16 Dec 2021 15:40:18 GMT, Lance Andersen wrote: > We will have to separate out the added changes into its own test as we are failing sporadically on some of the windows boxes due to the execution time. @LanceAndersen Do I understand correct, *ontop* of the diff you sent me, you also want me to strip all 2GB tests into a separate test class, correct? ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From lancea at openjdk.java.net Thu Dec 16 16:41:02 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Thu, 16 Dec 2021 16:41:02 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: On Mon, 13 Dec 2021 17:26:39 GMT, Markus KARG wrote: >> This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > Ignoring the case of intermittend zero reads as proposed by Alan Bateman > > Signed-off-by: Markus Karg > > We will have to separate out the added changes into its own test as we are failing sporadically on some of the windows boxes due to the execution time. > > @LanceAndersen Do I understand correct, _ontop_ of the diff you sent me, you also want me to strip all 2GB tests into a separate test class, correct? The updates that you made to add an additional test is resulting in sporadic timeouts on Windows. - Please revert TransferTo.Java to omit the changes you made to it as part of this PR - - Yes, we should cleanup TransferTo.java while we are addressing this test area - Add a new test file to address your changes to ChannelInputStream found in this PR ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Thu Dec 16 17:01:07 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 16 Dec 2021 17:01:07 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v9] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: - Ignoring the case of intermittend zero reads as proposed by Alan Bateman Signed-off-by: Markus Karg - Simplified using checked function Signed-off-by: Markus Karg - Common implementation for all 2 GB tests Signed-off-by: Markus Karg - Implementing 8278268 Signed-off-by: Markus Karg ------------- Changes: https://git.openjdk.java.net/jdk/pull/6711/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=08 Stats: 112 lines in 2 files changed: 84 ins; 5 del; 23 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Thu Dec 16 17:03:54 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 16 Dec 2021 17:03:54 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: On Thu, 16 Dec 2021 16:38:15 GMT, Lance Andersen wrote: >> Markus KARG has updated the pull request incrementally with one additional commit since the last revision: >> >> Ignoring the case of intermittend zero reads as proposed by Alan Bateman >> >> Signed-off-by: Markus Karg > >> > We will have to separate out the added changes into its own test as we are failing sporadically on some of the windows boxes due to the execution time. >> >> @LanceAndersen Do I understand correct, _ontop_ of the diff you sent me, you also want me to strip all 2GB tests into a separate test class, correct? > > The updates that you made to add an additional test is resulting in sporadic timeouts on Windows. > > - Please revert TransferTo.Java to omit the changes you made to it as part of this PR > - - Yes, we should cleanup TransferTo.java while we are addressing this test area > - Add a new test file to address your changes to ChannelInputStream found in this PR @LanceAndersen Are you sure that the sporadic timeout comes from the changes made by *this* PR? IIUC then the sporadic timeouts are found *in the master branch*, so reverting the changes from *this* PR do not bring a benefit. It would be better to separate out *all* (*even existing*) 2GB tests instead. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From lancea at openjdk.java.net Thu Dec 16 17:29:01 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Thu, 16 Dec 2021 17:29:01 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: On Thu, 16 Dec 2021 16:38:15 GMT, Lance Andersen wrote: >> Markus KARG has updated the pull request incrementally with one additional commit since the last revision: >> >> Ignoring the case of intermittend zero reads as proposed by Alan Bateman >> >> Signed-off-by: Markus Karg > >> > We will have to separate out the added changes into its own test as we are failing sporadically on some of the windows boxes due to the execution time. >> >> @LanceAndersen Do I understand correct, _ontop_ of the diff you sent me, you also want me to strip all 2GB tests into a separate test class, correct? > > The updates that you made to add an additional test is resulting in sporadic timeouts on Windows. > > - Please revert TransferTo.Java to omit the changes you made to it as part of this PR > - - Yes, we should cleanup TransferTo.java while we are addressing this test area > - Add a new test file to address your changes to ChannelInputStream found in this PR > @LanceAndersen Are you sure that the sporadic timeout comes from the changes made by _this_ PR? IIUC then the sporadic timeouts are found _in the master branch_, so reverting the changes from _this_ PR do not bring a benefit. It would be better to separate out _all_ (_even existing_) 2GB tests instead. As I mentioned in my earlier comment, we need to make the changes as outlined. We do have some Windows boxes which are older but we cannot afford to have sporadic test failures due to the disruption it causes. And yes, these failures are occurring with the changes made and do not occur if I back them out. So please follow the recommendations, I can then re-test the PR to validate we are clean (or as clean as we ca)n for sporadic failures. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Thu Dec 16 18:38:52 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 16 Dec 2021 18:38:52 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v10] In-Reply-To: References: Message-ID: <0YwYsYktVEIPPiCgJeyaUmSVEVrpIBYePKiaCqq7qs0=.ac226c5c-3981-40e8-bfa9-e59ad651074c@github.com> > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with five additional commits since the last revision: - separate out new tests into TransferTo2.java Signed-off-by: Markus Karg - Revert "Common implementation for all 2 GB tests" This reverts commit f8ba3a87221092ab3212732dd77a5d471ad1b13e. - Revert "Simplified using checked function" This reverts commit 4c2930fe71de4e29a779dea74edd9152676751f2. - No need to run test twice with same data point Signed-off-by: Markus Karg - Test cleanup proposed by Lance Andersen Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/2b72855f..709d258d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=09 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=08-09 Stats: 546 lines in 2 files changed: 422 ins; 57 del; 67 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Thu Dec 16 18:58:12 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 16 Dec 2021 18:58:12 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v11] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: separate out new tests into TransferTo2.java Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/709d258d..10431b44 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=10 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=09-10 Stats: 67 lines in 1 file changed: 0 ins; 67 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Thu Dec 16 18:58:13 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 16 Dec 2021 18:58:13 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: On Thu, 16 Dec 2021 17:26:04 GMT, Lance Andersen wrote: >>> > We will have to separate out the added changes into its own test as we are failing sporadically on some of the windows boxes due to the execution time. >>> >>> @LanceAndersen Do I understand correct, _ontop_ of the diff you sent me, you also want me to strip all 2GB tests into a separate test class, correct? >> >> The updates that you made to add an additional test is resulting in sporadic timeouts on Windows. >> >> - Please revert TransferTo.Java to omit the changes you made to it as part of this PR >> - - Yes, we should cleanup TransferTo.java while we are addressing this test area >> - Add a new test file to address your changes to ChannelInputStream found in this PR > >> @LanceAndersen Are you sure that the sporadic timeout comes from the changes made by _this_ PR? IIUC then the sporadic timeouts are found _in the master branch_, so reverting the changes from _this_ PR do not bring a benefit. It would be better to separate out _all_ (_even existing_) 2GB tests instead. > > As I mentioned in my earlier comment, we need to make the changes as outlined. We do have some Windows boxes which are older but we cannot afford to have sporadic test failures due to the disruption it causes. And yes, these failures are occurring with the changes made and do not occur if I back them out. > > So please follow the recommendations, I can then re-test the PR to validate we are clean (or as clean as we ca)n for sporadic failures. @LanceAndersen Thanks for the explanation, I (so I hope) did what you asked me for. The original TransferTo test now does not contain the new tests of *this* PR anymore, but there is a new TransferTo2 which *only* contain the new tests of *this* PR. Both contains your proposed cleanups. On my Windows laptop I never have nor had timeout problems. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Fri Dec 17 15:54:32 2021 From: duke at openjdk.java.net (Markus KARG) Date: Fri, 17 Dec 2021 15:54:32 GMT Subject: RFR: 8265891: (ch) InputStream returned by Channels.newInputStream should override transferTo [v13] In-Reply-To: References: Message-ID: On Thu, 12 Aug 2021 01:04:29 GMT, Brian Burkhalter wrote: >> Markus KARG has updated the pull request incrementally with two additional commits since the last revision: >> >> - Draft: Eliminated duplicate code using lambda expressions >> - Draft: Use blocking mode also for target channel > > src/java.base/share/classes/sun/nio/ch/ChannelInputStream.java line 249: > >> 247: } >> 248: >> 249: private static long transfer(ReadableByteChannel src, WritableByteChannel dst) throws IOException { > > Does this method have a measurable improvement in performance over `InputStream.transferTo()`? Measured it today, here are the actual results: Sandbox.transferTo thrpt 25 0,091 ? 0,011 ops/s -- using byte[] (original implementation) Sandbox.transferTo thrpt 25 0,095 ? 0,012 ops/s -- using utility temporary buffer (my contribution) Sandbox.transferTo thrpt 23 0,099 ? 0,012 ops/s -- FileChannel.transferTo(FileChannel) The improvement of this method actually is approx. 4% on my Windows laptop, which I would say is worth to do it. ------------- PR: https://git.openjdk.java.net/jdk/pull/4263 From jai.forums2013 at gmail.com Sat Dec 18 06:00:45 2021 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Sat, 18 Dec 2021 11:30:45 +0530 Subject: Seeking inputs on 8224794: ZipFile/JarFile should open zip/JAR file with FILE_SHARE_DELETE sharing mode on Windows In-Reply-To: References: Message-ID: <352c3fdb-e6e6-d03c-cea5-91ad2309b3f2@gmail.com> Would there be interest in moving this forward? Enabling that FILE_SHARE_DELETE option has opened up some questions that I noted in my previous mail below. So in order to move forward I would need some inputs. If we don't want to do this at this time, I'll close the draft PR that's currently open https://github.com/openjdk/jdk/pull/6477. -Jaikiran On 19/11/21 5:10 pm, Jaikiran Pai wrote: > I have been experimenting with > https://bugs.openjdk.java.net/browse/JDK-8224794 and have reached a > stage with my changes where I would like some inputs before I move > ahead or raise a PR for review. > > As noted in that JBS issue, the idea is to open the underlying file > using FILE_SHARE_DELETE when using java.uitl.zip.ZipFile and > java.util.jar.JarFile. From what I can infer from that JBS issue, the > motivation behind this seems to be to allow deleting that underlying > file when that file is currently opened and in use by the > ZipFile/JarFile instance. The linked github issues seem to suggest the > same. IMO, it's a reasonable expectation, given that such a deletion > works on *nix platform. It's on Windows where such a deletion fails > with "The process cannot access the file because it is being used by > another process" (the other process in many cases will be the current > JVM instance itself). So trying to get this to work on Windows by > setting the FILE_SHARE_DELETE share access mode seems reasonable. > > Coming to the implementation part, I've an initial draft version here > https://github.com/openjdk/jdk/pull/6477. It sets the > FILE_SHARE_DELETE share access mode on the file when constructed by > ZipFile/JarFile. It "works", in the sense that if you issue a delete > against this underlying file (path) after its been opened by > ZipFile/JarFile instance, you no longer see the error mentioned above > and the delete completes without any exception. However, my > experiments with this change have raised a bunch of questions related > to this: > > The crucial part of the "DeleteFile" API in Windows is that, as stated > in its documentation[1]: > > "The DeleteFile function marks a file for deletion on close. > Therefore, the file deletion does not occur until the last handle to > the file is closed. Subsequent calls to CreateFile to open the file > fail with ERROR_ACCESS_DENIED." > > So imagine the following pseudo-code: > > String path = "C:/foo.jar" > 1. JarFile jar = new JarFile(path); // create a jar instance > 2. var deleted = new File(path).delete(); // delete the underlying file > 3. var exists = new File(path).exists(); // check file existence > 4. FileOutputStream fos = new FileOutputStream(new File(path)); // > open a FileOutputStream for /tmp/foo.jar > 5. java.nio.file.Files.deleteIfExists(Path.of(path)); > > When this is run (on Windows) *without* the change that introduces > FILE_SHARE_DELETE, I notice that line#2 returns "false" for > File.delete() (since the file is in use by the JarFile instance) and > then line#3 returns "true" since it still exists and line#4 succeeds > and creates a new FileOutputStream and line#5 fails with an exception > stating "java.nio.file.FileSystemException: foo.jar: The process > cannot access the file because it is being used by another process > " > > Now when the same code is run (on Windows) *with* the proposed > FILE_SHARE_DELETE changes, I notice that line#2 returns "true" (i.e. > the file is considered deleted) and line#3 returns "true" (i.e. it's > considered the file still exists). So essentially the file is only > marked for deletion and hasn't been deleted at the OS level, since the > JarFile instance still has an handle open. Then line#4 unlike > previously, now throws an exception and fails to create the > FileOutputStream instance. The failure says > "java.nio.file.AccessDeniedException: foo.jar". This exception matches > the documentation from that Windows DeleteFile API. line#5 too fails > with this same AccessDeniedException. > > So what this means is, although we have managed to allow the deletion > to happen (i.e. the first call to delete does mark it for deletion at > OS level), IMO, it still doesn't improve much and raises other > difference in behaviour of APIs as seen above, unless the underlying > file handle held by ZipFile/JarFile is closed. So that brings me to > the next related issue. > > For this change to be really useful, the file handle (which is an > instance of RandomAccessFile) held by ZipFile/JarFile needs to be > closed. If a user application explicitly creates an instance of > ZipFile/JarFile, it is expected that they close it themselves. > However, the most prominent use of the JarFile creation is within the > JDK code where it uses the sun.net.www.protocol.jar.JarURLConnection > to construct the JarFile instances, typically from the URLClassLoader. > By default these JarFile instances that are created by the > sun.net.www.protocol.jar.JarURLConnection are cached and the close() > on those instances is called only when the URLClassLoader is > close()ed. That in itself doesn't guarantee that the underlying file > descriptor corresponding to the JarFile instance gets closed, because > there's one more level of reference counting "cache" in use in > java.util.zip.ZipFile$Source class where the same underlying file > descriptor (corresponding to the jar file) gets used by multiple > JarFile instances (irrespective of who created those). So in short, > the underlying file descriptor of a JarFile only gets closed when > every single instance of a JarFile pointing to the same jar file has > been close()ed within the JVM. Which effectively means that > introducing the FILE_SHARE_DELETE semantics/changes for Windows will > only allow the first delete() attempt to succeed (since that one marks > it for deletion), but any subsequent operations like the one noted in > the examples, that rely on that file being deleted will continue to > fail with Access Denied errors. Given all this caching of JarFile and > the reference count cache for the underlying jar file, is this change > of FILE_SHARE_DELETE adding much value and should I pursue it further? > > [1] > https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-deletefilew > > -Jaikiran > From duke at openjdk.java.net Sun Dec 19 08:38:25 2021 From: duke at openjdk.java.net (Markus KARG) Date: Sun, 19 Dec 2021 08:38:25 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: On Thu, 16 Dec 2021 17:26:04 GMT, Lance Andersen wrote: >>> > We will have to separate out the added changes into its own test as we are failing sporadically on some of the windows boxes due to the execution time. >>> >>> @LanceAndersen Do I understand correct, _ontop_ of the diff you sent me, you also want me to strip all 2GB tests into a separate test class, correct? >> >> The updates that you made to add an additional test is resulting in sporadic timeouts on Windows. >> >> - Please revert TransferTo.Java to omit the changes you made to it as part of this PR >> - - Yes, we should cleanup TransferTo.java while we are addressing this test area >> - Add a new test file to address your changes to ChannelInputStream found in this PR > >> @LanceAndersen Are you sure that the sporadic timeout comes from the changes made by _this_ PR? IIUC then the sporadic timeouts are found _in the master branch_, so reverting the changes from _this_ PR do not bring a benefit. It would be better to separate out _all_ (_even existing_) 2GB tests instead. > > As I mentioned in my earlier comment, we need to make the changes as outlined. We do have some Windows boxes which are older but we cannot afford to have sporadic test failures due to the disruption it causes. And yes, these failures are occurring with the changes made and do not occur if I back them out. > > So please follow the recommendations, I can then re-test the PR to validate we are clean (or as clean as we ca)n for sporadic failures. @LanceAndersen Requested changes are done since three days. Anything else you want me to change? ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Tue Dec 21 15:45:23 2021 From: duke at openjdk.java.net (Mike Hearn) Date: Tue, 21 Dec 2021 15:45:23 GMT Subject: RFR: 8273922: (fs) UserDefinedFileAttributeView doesn't handle file names that are just under the MAX_PATH limit (win) [v5] In-Reply-To: References: Message-ID: On Tue, 2 Nov 2021 23:37:44 GMT, Brian Burkhalter wrote: >> Modify `sun.nio.fs.WindowsUserDefinedFileAttributeView.join(WindowsPath,String)` to handle file names which exceed the limit. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8273922: Change getNameCount() != 1 tp getParent() != null Following up in case anyone else ever arrives here, the regression was fixed in https://github.com/openjdk/jdk/pull/6435 Thanks for the fixes. ------------- PR: https://git.openjdk.java.net/jdk/pull/5594 From duke at openjdk.java.net Wed Dec 22 08:33:15 2021 From: duke at openjdk.java.net (Markus KARG) Date: Wed, 22 Dec 2021 08:33:15 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v7] In-Reply-To: References: Message-ID: On Wed, 15 Dec 2021 14:25:10 GMT, Alan Bateman wrote: >> Ah, right, seems I missed that section about the one-byte-guarantee. Great, so I kindly request approval of this PR! > >> Ah, right, seems I missed that section about the one-byte-guarantee. Great, so I kindly request approval of this PR! > > I think Lance is going to help on this. We need to study the test update closely as this test has been causing a lot of issues in our CI (timeouts, hangs, ...). I think the main issue with the test right now is JDK-8278369 and it may only be on Windows. I haven't had time myself to study it closely. @AlanBateman Seems Lance has no time (or is possibly on holidays) so I kindly ask for approval again, as everything he asked me for is fixed in this PR since a week already. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From alanb at openjdk.java.net Wed Dec 22 13:34:11 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 22 Dec 2021 13:34:11 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v7] In-Reply-To: References: Message-ID: On Wed, 15 Dec 2021 14:25:10 GMT, Alan Bateman wrote: >> Ah, right, seems I missed that section about the one-byte-guarantee. Great, so I kindly request approval of this PR! > >> Ah, right, seems I missed that section about the one-byte-guarantee. Great, so I kindly request approval of this PR! > > I think Lance is going to help on this. We need to study the test update closely as this test has been causing a lot of issues in our CI (timeouts, hangs, ...). I think the main issue with the test right now is JDK-8278369 and it may only be on Windows. I haven't had time myself to study it closely. > @AlanBateman Seems Lance has no time (or is possibly on holidays) so I kindly ask for approval again, as everything he asked me for is fixed in this PR since a week already. I've been busy and haven't been following the recent discussions about the test. Lance has been busy too but I think he has been looking at the tests. He mentioned that the new TransferTo2 tests actually runs TransferTo due to an issue with the @run tag. I will try to get time to look at the test soon. Once this PR is in integrated then I think JDK-8278369 should be priority, I think that issue may be Windows specific. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Wed Dec 22 15:41:05 2021 From: duke at openjdk.java.net (Markus KARG) Date: Wed, 22 Dec 2021 15:41:05 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v12] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with one additional commit since the last revision: Fixed: TransferTo2 runs TransferTo Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/10431b44..fc7d00bf Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=11 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=10-11 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Wed Dec 22 15:41:05 2021 From: duke at openjdk.java.net (Markus KARG) Date: Wed, 22 Dec 2021 15:41:05 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v7] In-Reply-To: References: Message-ID: <2w59bsBKnfKFrJQ9qVouh8ok9dY30h7GoAMs-n6Z8kw=.774e0672-1c41-4e90-ae7e-019b180f8449@github.com> On Wed, 22 Dec 2021 13:30:40 GMT, Alan Bateman wrote: >>> Ah, right, seems I missed that section about the one-byte-guarantee. Great, so I kindly request approval of this PR! >> >> I think Lance is going to help on this. We need to study the test update closely as this test has been causing a lot of issues in our CI (timeouts, hangs, ...). I think the main issue with the test right now is JDK-8278369 and it may only be on Windows. I haven't had time myself to study it closely. > >> @AlanBateman Seems Lance has no time (or is possibly on holidays) so I kindly ask for approval again, as everything he asked me for is fixed in this PR since a week already. > > I've been busy and haven't been following the recent discussions about the test. Lance has been busy too but I think he has been looking at the tests. He mentioned that the new TransferTo2 tests actually runs TransferTo due to an issue with the @run tag. I will try to get time to look at the test soon. Once this PR is in integrated then I think JDK-8278369 should be priority, I think that issue may be Windows specific. Good catch! Silly me. Fixed in https://github.com/openjdk/jdk/pull/6711/commits/fc7d00bf20861276e11e755c6794999eca95049d. JDK-8278369: Tried several times on my Windows 11 laptop, but still cannot reproduce. Maybe Lance's boxes are simply overloaded / too slow? ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From alanb at openjdk.java.net Wed Dec 22 15:44:11 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 22 Dec 2021 15:44:11 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v7] In-Reply-To: <2w59bsBKnfKFrJQ9qVouh8ok9dY30h7GoAMs-n6Z8kw=.774e0672-1c41-4e90-ae7e-019b180f8449@github.com> References: <2w59bsBKnfKFrJQ9qVouh8ok9dY30h7GoAMs-n6Z8kw=.774e0672-1c41-4e90-ae7e-019b180f8449@github.com> Message-ID: On Wed, 22 Dec 2021 15:36:51 GMT, Markus KARG wrote: > JDK-8278369: Tried several times on my Windows 11 laptop, but still cannot reproduce. Maybe Lance's boxes are simply overloaded / too slow? I don't know if anyone has reproduced locally, instead it seems to be random Windows machines in our CI. There are differences between Windows client and server editions that periodically cause issues, I don't know if this is the case here. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From lancea at openjdk.java.net Wed Dec 22 17:32:10 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Wed, 22 Dec 2021 17:32:10 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v12] In-Reply-To: References: Message-ID: On Wed, 22 Dec 2021 15:41:05 GMT, Markus KARG wrote: >> This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > Fixed: TransferTo2 runs TransferTo > > Signed-off-by: Markus Karg There is additional clean up that can still be done. There is duplication of some methods and constants between both tests and this could be streamlined to further reduce duplication I have also attached a diff of minor cleanup with the tests as they are additional cleanup to the above work. As Alan mentioned we are seeing failures in approximately 3% of the runs on various windows boxes. It appears to be an issue with TransferTo.testStreamContents->TransferTo.checkTransferredContents and selectableChannelOutput() as it blocking. I have not been able to determine the cause. [testcleanup-2.patch.txt](https://github.com/openjdk/jdk/files/7764237/testcleanup-2.patch.txt) ------------- Changes requested by lancea (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Wed Dec 22 18:27:14 2021 From: duke at openjdk.java.net (Markus KARG) Date: Wed, 22 Dec 2021 18:27:14 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: On Thu, 16 Dec 2021 17:26:04 GMT, Lance Andersen wrote: >>> > We will have to separate out the added changes into its own test as we are failing sporadically on some of the windows boxes due to the execution time. >>> >>> @LanceAndersen Do I understand correct, _ontop_ of the diff you sent me, you also want me to strip all 2GB tests into a separate test class, correct? >> >> The updates that you made to add an additional test is resulting in sporadic timeouts on Windows. >> >> - Please revert TransferTo.Java to omit the changes you made to it as part of this PR >> - - Yes, we should cleanup TransferTo.java while we are addressing this test area >> - Add a new test file to address your changes to ChannelInputStream found in this PR > >> @LanceAndersen Are you sure that the sporadic timeout comes from the changes made by _this_ PR? IIUC then the sporadic timeouts are found _in the master branch_, so reverting the changes from _this_ PR do not bring a benefit. It would be better to separate out _all_ (_even existing_) 2GB tests instead. > > As I mentioned in my earlier comment, we need to make the changes as outlined. We do have some Windows boxes which are older but we cannot afford to have sporadic test failures due to the disruption it causes. And yes, these failures are occurring with the changes made and do not occur if I back them out. > > So please follow the recommendations, I can then re-test the PR to validate we are clean (or as clean as we ca)n for sporadic failures. @LanceAndersen I assumed the second test will exist only for the time until we fixed the Windows problems, so I did not invest time to reduce duplication so far. So you really want to keep the separated tests forever and want me to reduce duplication? ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From lancea at openjdk.java.net Wed Dec 22 18:38:08 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Wed, 22 Dec 2021 18:38:08 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: On Thu, 16 Dec 2021 17:26:04 GMT, Lance Andersen wrote: >>> > We will have to separate out the added changes into its own test as we are failing sporadically on some of the windows boxes due to the execution time. >>> >>> @LanceAndersen Do I understand correct, _ontop_ of the diff you sent me, you also want me to strip all 2GB tests into a separate test class, correct? >> >> The updates that you made to add an additional test is resulting in sporadic timeouts on Windows. >> >> - Please revert TransferTo.Java to omit the changes you made to it as part of this PR >> - - Yes, we should cleanup TransferTo.java while we are addressing this test area >> - Add a new test file to address your changes to ChannelInputStream found in this PR > >> @LanceAndersen Are you sure that the sporadic timeout comes from the changes made by _this_ PR? IIUC then the sporadic timeouts are found _in the master branch_, so reverting the changes from _this_ PR do not bring a benefit. It would be better to separate out _all_ (_even existing_) 2GB tests instead. > > As I mentioned in my earlier comment, we need to make the changes as outlined. We do have some Windows boxes which are older but we cannot afford to have sporadic test failures due to the disruption it causes. And yes, these failures are occurring with the changes made and do not occur if I back them out. > > So please follow the recommendations, I can then re-test the PR to validate we are clean (or as clean as we ca)n for sporadic failures. > @LanceAndersen I assumed the second test will exist only for the time until we fixed the Windows problems, so I did not invest time to reduce duplication so far. So you really want to keep the separated tests forever and want me to reduce duplication? I would not make any such assumptions especially for tests that create large files. So yes please address the duplication. It is not a huge amount of work to create a base class that the tests extend (you will see examples elsewhere within the various test directories). ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Wed Dec 22 18:38:08 2021 From: duke at openjdk.java.net (Markus KARG) Date: Wed, 22 Dec 2021 18:38:08 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: <77PmxY01wzavxG8uoYhVXoboSayGM8Bg01Dh8Ugsd9w=.b1c64653-93e4-4494-87e4-d9c68c21a27d@github.com> On Wed, 22 Dec 2021 18:34:09 GMT, Lance Andersen wrote: >>> @LanceAndersen Are you sure that the sporadic timeout comes from the changes made by _this_ PR? IIUC then the sporadic timeouts are found _in the master branch_, so reverting the changes from _this_ PR do not bring a benefit. It would be better to separate out _all_ (_even existing_) 2GB tests instead. >> >> As I mentioned in my earlier comment, we need to make the changes as outlined. We do have some Windows boxes which are older but we cannot afford to have sporadic test failures due to the disruption it causes. And yes, these failures are occurring with the changes made and do not occur if I back them out. >> >> So please follow the recommendations, I can then re-test the PR to validate we are clean (or as clean as we ca)n for sporadic failures. > >> @LanceAndersen I assumed the second test will exist only for the time until we fixed the Windows problems, so I did not invest time to reduce duplication so far. So you really want to keep the separated tests forever and want me to reduce duplication? > > I would not make any such assumptions especially for tests that create large files. So yes please address the duplication. It is not a huge amount of work to create a base class that the tests extend (you will see examples elsewhere within the various test directories). @LanceAndersen Ok, will do that. Stay tuned. :-) ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Wed Dec 22 19:01:04 2021 From: duke at openjdk.java.net (Markus KARG) Date: Wed, 22 Dec 2021 19:01:04 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v13] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with one additional commit since the last revision: Code style according to Lance Andersen's request Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/fc7d00bf..4bbce517 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=12 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=11-12 Stats: 18 lines in 2 files changed: 0 ins; 3 del; 15 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Wed Dec 22 22:11:06 2021 From: duke at openjdk.java.net (Markus KARG) Date: Wed, 22 Dec 2021 22:11:06 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v14] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with one additional commit since the last revision: Code Deduplication as requested by Lance Andersen Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/4bbce517..b243b68d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=13 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=12-13 Stats: 519 lines in 3 files changed: 203 ins; 314 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Thu Dec 23 09:48:00 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 23 Dec 2021 09:48:00 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v15] In-Reply-To: References: Message-ID: <4RcpFZMVKNqYuiowJOFwZuM_RDwKHAt8KyOPHz3S5c4=.23940329-cbf6-4f13-b044-eb91e4193f36@github.com> > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with one additional commit since the last revision: Removing unused imports Signed-off-by: Markus Karg ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/b243b68d..9ae35561 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=14 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=13-14 Stats: 35 lines in 3 files changed: 0 ins; 35 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Thu Dec 23 09:51:15 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 23 Dec 2021 09:51:15 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: On Wed, 22 Dec 2021 18:34:09 GMT, Lance Andersen wrote: >>> @LanceAndersen Are you sure that the sporadic timeout comes from the changes made by _this_ PR? IIUC then the sporadic timeouts are found _in the master branch_, so reverting the changes from _this_ PR do not bring a benefit. It would be better to separate out _all_ (_even existing_) 2GB tests instead. >> >> As I mentioned in my earlier comment, we need to make the changes as outlined. We do have some Windows boxes which are older but we cannot afford to have sporadic test failures due to the disruption it causes. And yes, these failures are occurring with the changes made and do not occur if I back them out. >> >> So please follow the recommendations, I can then re-test the PR to validate we are clean (or as clean as we ca)n for sporadic failures. > >> @LanceAndersen I assumed the second test will exist only for the time until we fixed the Windows problems, so I did not invest time to reduce duplication so far. So you really want to keep the separated tests forever and want me to reduce duplication? > > I would not make any such assumptions especially for tests that create large files. So yes please address the duplication. It is not a huge amount of work to create a base class that the tests extend (you will see examples elsewhere within the various test directories). @LanceAndersen I have refactored most of the shared code into a common super class. I deliberately left the 2GB+ tests as (mostly) duplicated code until we sorted out the actual origin of the sporadic failures. If you do not want to wait until then, just tell me, so I will consolidate both 2GB+ test functions into a single one. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Thu Dec 23 10:10:15 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 23 Dec 2021 10:10:15 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v12] In-Reply-To: References: Message-ID: <9URKIxzz3Ows22Nr8SCOx98xiEUE2ZEHl-GjsLQDt3c=.13f1a1ae-91a2-4f11-adfa-1009075f6acd@github.com> On Wed, 22 Dec 2021 17:29:22 GMT, Lance Andersen wrote: > As Alan mentioned we are seeing failures in approximately 3% of the runs on various windows boxes. It appears to be an issue with TransferTo.testStreamContents->TransferTo.checkTransferredContents and selectableChannelOutput() as it blocking. I have not been able to determine the cause. Looking at the fact that selectableChannelOutput() utilizes Channels.newInputStream() it comes to my mind that [Trisha Gee reported](https://trishagee.com/2021/12/21/simple-network-connections-with-java-a-problem/) that Channels.newInputStream() produced an endless hanging of her apparently correct code, and it was gone once she replaced it by Channels.newReader(). While she reproduced that on a Mac, it might be the same cause even on Windows. Trisha currently is working on a reproducer, but she already detected that *her* endless hanging is definitively a thread waiting for the blocking lock. This sounds similar to what you wrote above. So maybe the problem is not inside of TransferTo, but inside of ChannelInputStream? ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From lancea at openjdk.java.net Thu Dec 23 17:17:14 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Thu, 23 Dec 2021 17:17:14 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v15] In-Reply-To: <4RcpFZMVKNqYuiowJOFwZuM_RDwKHAt8KyOPHz3S5c4=.23940329-cbf6-4f13-b044-eb91e4193f36@github.com> References: <4RcpFZMVKNqYuiowJOFwZuM_RDwKHAt8KyOPHz3S5c4=.23940329-cbf6-4f13-b044-eb91e4193f36@github.com> Message-ID: On Thu, 23 Dec 2021 09:48:00 GMT, Markus KARG wrote: >> This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > Removing unused imports > > Signed-off-by: Markus Karg Thank you for the current updates. We are starting to move in the right direction. Alan and I had an offline conversation and concluded the following changes should be made: - Rename the base class as it is not an Abstract class so the name is confusing - The base class should only contain constants and helper/utility methods. The actual tests should reside in the test classes - That the tests should be broken out as - A separate test class for each large file test with a more descriptive class name - A primary TransferTo test class which includes the remaining tests After the next set of updates, I will kick off some additional Mach5 runs I realize that we are spending a lot of time on the tests but they are a key aspect of making sure we have adequate coverage for changes such as this and are easy to maintain and understand by future maintainers/developers ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Fri Dec 24 10:04:13 2021 From: duke at openjdk.java.net (Markus KARG) Date: Fri, 24 Dec 2021 10:04:13 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: On Wed, 22 Dec 2021 18:34:09 GMT, Lance Andersen wrote: >>> @LanceAndersen Are you sure that the sporadic timeout comes from the changes made by _this_ PR? IIUC then the sporadic timeouts are found _in the master branch_, so reverting the changes from _this_ PR do not bring a benefit. It would be better to separate out _all_ (_even existing_) 2GB tests instead. >> >> As I mentioned in my earlier comment, we need to make the changes as outlined. We do have some Windows boxes which are older but we cannot afford to have sporadic test failures due to the disruption it causes. And yes, these failures are occurring with the changes made and do not occur if I back them out. >> >> So please follow the recommendations, I can then re-test the PR to validate we are clean (or as clean as we ca)n for sporadic failures. > >> @LanceAndersen I assumed the second test will exist only for the time until we fixed the Windows problems, so I did not invest time to reduce duplication so far. So you really want to keep the separated tests forever and want me to reduce duplication? > > I would not make any such assumptions especially for tests that create large files. So yes please address the duplication. It is not a huge amount of work to create a base class that the tests extend (you will see examples elsewhere within the various test directories). @LanceAndersen I will do as you requested, but due to the holidays it will need a few days. Stay tuned and Merry Christmas! ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From alanb at openjdk.java.net Fri Dec 24 15:33:15 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 24 Dec 2021 15:33:15 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v15] In-Reply-To: <4RcpFZMVKNqYuiowJOFwZuM_RDwKHAt8KyOPHz3S5c4=.23940329-cbf6-4f13-b044-eb91e4193f36@github.com> References: <4RcpFZMVKNqYuiowJOFwZuM_RDwKHAt8KyOPHz3S5c4=.23940329-cbf6-4f13-b044-eb91e4193f36@github.com> Message-ID: On Thu, 23 Dec 2021 09:48:00 GMT, Markus KARG wrote: >> This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > Removing unused imports > > Signed-off-by: Markus Karg > Looking at the fact that selectableChannelOutput() utilizes Channels.newInputStream() it comes to my mind that [Trisha Gee reported](https://trishagee.com/2021/12/21/simple-network-connections-with-java-a-problem/) that Channels.newInputStream() produced an endless hanging of her apparently correct code, and it was gone once she replaced it by Channels.newReader(). While she reproduced that on a Mac, it might be the same cause even on Windows. Trisha currently is working on a reproducer, but she already detected that _her_ endless hanging is definitively a thread waiting for the blocking lock. This sounds similar to what you wrote above. So maybe the problem is not inside of TransferTo, but inside of ChannelInputStream? I think the hang in JDK-8278369 is specific to the Unix domain socket implementation of Pipe on Windows 10+ and Windows Server 2019+. More specifically, I think the issue is closing the sink after a large number of bytes has been written does not always cause a reader on the source side to wakeup with EOF. As a test, I changed the implementation to use TCP sockets unconditional (so it works like Windows 8 or Windows Server 2012 or 2016) and I cannot duplicate the issue. We need to look closer at this in the new year. I looked briefly at the chat server and it has a number of issues. There is a ArrayList mutated and read from several threads without synchronization. There are several issues that are possible when a chat client does not keep up, this will eventually cause the chat server to hang (a thread blocked on the PrintWriter) and all other threads will eventually blocking trying to echo the messages. So from a quick look, I don't think it is anything to do with the issues we are discussing here. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Sat Dec 25 08:21:11 2021 From: duke at openjdk.java.net (Markus KARG) Date: Sat, 25 Dec 2021 08:21:11 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v15] In-Reply-To: References: <4RcpFZMVKNqYuiowJOFwZuM_RDwKHAt8KyOPHz3S5c4=.23940329-cbf6-4f13-b044-eb91e4193f36@github.com> Message-ID: On Fri, 24 Dec 2021 15:29:43 GMT, Alan Bateman wrote: >> Markus KARG has updated the pull request incrementally with one additional commit since the last revision: >> >> Removing unused imports >> >> Signed-off-by: Markus Karg > >> Looking at the fact that selectableChannelOutput() utilizes Channels.newInputStream() it comes to my mind that [Trisha Gee reported](https://trishagee.com/2021/12/21/simple-network-connections-with-java-a-problem/) that Channels.newInputStream() produced an endless hanging of her apparently correct code, and it was gone once she replaced it by Channels.newReader(). While she reproduced that on a Mac, it might be the same cause even on Windows. Trisha currently is working on a reproducer, but she already detected that _her_ endless hanging is definitively a thread waiting for the blocking lock. This sounds similar to what you wrote above. So maybe the problem is not inside of TransferTo, but inside of ChannelInputStream? > > I think the hang in JDK-8278369 is specific to the Unix domain socket implementation of Pipe on Windows 10+ and Windows Server 2019+. More specifically, I think the issue is closing the sink after a large number of bytes has been written does not always cause a reader on the source side to wakeup with EOF. As a test, I changed the implementation to use TCP sockets unconditional (so it works like Windows 8 or Windows Server 2012 or 2016) and I cannot duplicate the issue. We need to look closer at this in the new year. > > I looked briefly at the chat server and it has a number of issues. There is a ArrayList mutated and read from several threads without synchronization. There are several issues that are possible when a chat client does not keep up, this will eventually cause the chat server to hang (a thread blocked on the PrintWriter) and all other threads will eventually blocking trying to echo the messages. So from a quick look, I don't think it is anything to do with the issues we are discussing here. @AlanBateman Thanks a lot for investing your time. At least it is good news for TransferTo. ;-) @trishagee Can you please double-check that your problem actually is solved *solely* by replacing streams by writers? ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Sat Dec 25 16:10:37 2021 From: duke at openjdk.java.net (Markus KARG) Date: Sat, 25 Dec 2021 16:10:37 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v16] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with three additional commits since the last revision: - separate test class for each large file test - Base class only contains utilities and helpers - Renamed AbstractTransferTo to TransferToBase as it is not abstract ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6711/files - new: https://git.openjdk.java.net/jdk/pull/6711/files/9ae35561..c79fc0b1 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=15 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6711&range=14-15 Stats: 725 lines in 6 files changed: 402 ins; 263 del; 60 mod Patch: https://git.openjdk.java.net/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Sat Dec 25 16:10:38 2021 From: duke at openjdk.java.net (Markus KARG) Date: Sat, 25 Dec 2021 16:10:38 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: On Wed, 22 Dec 2021 18:34:09 GMT, Lance Andersen wrote: >>> @LanceAndersen Are you sure that the sporadic timeout comes from the changes made by _this_ PR? IIUC then the sporadic timeouts are found _in the master branch_, so reverting the changes from _this_ PR do not bring a benefit. It would be better to separate out _all_ (_even existing_) 2GB tests instead. >> >> As I mentioned in my earlier comment, we need to make the changes as outlined. We do have some Windows boxes which are older but we cannot afford to have sporadic test failures due to the disruption it causes. And yes, these failures are occurring with the changes made and do not occur if I back them out. >> >> So please follow the recommendations, I can then re-test the PR to validate we are clean (or as clean as we ca)n for sporadic failures. > >> @LanceAndersen I assumed the second test will exist only for the time until we fixed the Windows problems, so I did not invest time to reduce duplication so far. So you really want to keep the separated tests forever and want me to reduce duplication? > > I would not make any such assumptions especially for tests that create large files. So yes please address the duplication. It is not a huge amount of work to create a base class that the tests extend (you will see examples elsewhere within the various test directories). @LanceAndersen I'm done with the requested changes. For "the remaining tests" I kept the separation into TransferTo (existing tests) and TransferTo2 (new tests of this PR) -- or do you want me to consolidate TransferTo2 into TransferTo now that Alan identified the source of trouble being the Unix Domain Sockets? ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Sun Dec 26 20:10:10 2021 From: duke at openjdk.java.net (=?UTF-8?B?0KHQtdGA0LPQtdC5?= =?UTF-8?B?IA==?= =?UTF-8?B?0KbRi9C/0LDQvdC+0LI=?=) Date: Sun, 26 Dec 2021 20:10:10 GMT Subject: RFR: 8274811: Remove superfluous use of boxing in java.base In-Reply-To: References: Message-ID: On Sat, 11 Sep 2021 12:11:50 GMT, Andrey Turbanov wrote: > Usages of primitive types should be preferred and makes code easier to read. > Similar cleanups: > 1. [JDK-8273168](https://bugs.openjdk.java.net/browse/JDK-8273168) java.desktop > 2. [JDK-8274234](https://bugs.openjdk.java.net/browse/JDK-8274234) java.sql.rowset Marked as reviewed by stsypanov at github.com (no known OpenJDK username). ------------- PR: https://git.openjdk.java.net/jdk/pull/5481 From duke at openjdk.java.net Mon Dec 27 17:28:10 2021 From: duke at openjdk.java.net (Trisha Gee) Date: Mon, 27 Dec 2021 17:28:10 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v8] In-Reply-To: References: Message-ID: On Sat, 25 Dec 2021 16:06:05 GMT, Markus KARG wrote: >>> @LanceAndersen I assumed the second test will exist only for the time until we fixed the Windows problems, so I did not invest time to reduce duplication so far. So you really want to keep the separated tests forever and want me to reduce duplication? >> >> I would not make any such assumptions especially for tests that create large files. So yes please address the duplication. It is not a huge amount of work to create a base class that the tests extend (you will see examples elsewhere within the various test directories). > > @LanceAndersen I'm done with the requested changes. For "the remaining tests" I kept the separation into TransferTo (existing tests) and TransferTo2 (new tests of this PR) -- or do you want me to consolidate TransferTo2 into TransferTo now that Alan identified the source of trouble being the Unix Domain Sockets? @mkarg yes, I can confirm the only change I made was to substitute Channels.newOutputStream with Channels.newWriter, and similarly InputStream/Reader. The code can be found here: [socket-locking](https://github.com/trishagee/socket-locking/). There's two different server classes, and a test case - the test works as expected with Writer/Reader and blocks with the Streams implementation (MacOS Big Sur, adopt-openjdk-11.0.11/) ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From lancea at openjdk.java.net Mon Dec 27 18:12:12 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Mon, 27 Dec 2021 18:12:12 GMT Subject: RFR: 8274811: Remove superfluous use of boxing in java.base In-Reply-To: References: Message-ID: On Sat, 11 Sep 2021 12:11:50 GMT, Andrey Turbanov wrote: > Usages of primitive types should be preferred and makes code easier to read. > Similar cleanups: > 1. [JDK-8273168](https://bugs.openjdk.java.net/browse/JDK-8273168) java.desktop > 2. [JDK-8274234](https://bugs.openjdk.java.net/browse/JDK-8274234) java.sql.rowset Marked as reviewed by lancea (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/5481 From duke at openjdk.java.net Mon Dec 27 20:07:10 2021 From: duke at openjdk.java.net (=?UTF-8?B?0KHQtdGA0LPQtdC5?= =?UTF-8?B?IA==?= =?UTF-8?B?0KbRi9C/0LDQvdC+0LI=?=) Date: Mon, 27 Dec 2021 20:07:10 GMT Subject: RFR: 8274811: Remove superfluous use of boxing in java.base In-Reply-To: References: Message-ID: <8IjGbdX2WK-ehSvj8t2tOXFwizmWpqX_iXIvErThatQ=.e201c353-ed8a-4789-9c8e-3d6b37097b42@github.com> On Sat, 27 Nov 2021 17:41:58 GMT, Andrey Turbanov wrote: >> Usages of primitive types should be preferred and makes code easier to read. >> Similar cleanups: >> 1. [JDK-8273168](https://bugs.openjdk.java.net/browse/JDK-8273168) java.desktop >> 2. [JDK-8274234](https://bugs.openjdk.java.net/browse/JDK-8274234) java.sql.rowset > > let's wait for review a bit more @turbanoff I think now you can integrate the PR ------------- PR: https://git.openjdk.java.net/jdk/pull/5481 From duke at openjdk.java.net Tue Dec 28 18:00:14 2021 From: duke at openjdk.java.net (Markus KARG) Date: Tue, 28 Dec 2021 18:00:14 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v15] In-Reply-To: References: <4RcpFZMVKNqYuiowJOFwZuM_RDwKHAt8KyOPHz3S5c4=.23940329-cbf6-4f13-b044-eb91e4193f36@github.com> Message-ID: On Fri, 24 Dec 2021 15:29:43 GMT, Alan Bateman wrote: > I looked briefly at the chat server and it has a number of issues. There is a ArrayList mutated and read from several threads without synchronization. There are several issues that are possible when a chat client does not keep up, this will eventually cause the chat server to hang (a thread blocked on the PrintWriter) and all other threads will eventually blocking trying to echo the messages. So from a quick look, I don't think it is anything to do with the issues we are discussing here. @AlanBateman I found [the cause of the problem](https://github.com/trishagee/socket-locking/pull/1#issuecomment-1002187644). The problem is that ChannelInputStream and ChannelOutputStream share the same blocking lock when created for the same IP connection, which is pretty straightforward, as a SocketChannel is bidirectional by nature, but has only one blocking setting for both directions. This shared lock is not documented anywhere, so how should an application programmer know about this? This should *either* be documented in the Channels JavaDocs, or (even better) there should be separate blocking modes per direction. FYI, the channel reader does not produce the deadlock as it does not care about the blocking mode at all but simply expects non-blocking mode so it runs in an active loop (squandering power). Maybe we should align both implementations, reader and stream? WDYT? ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From alanb at openjdk.java.net Wed Dec 29 07:51:09 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 29 Dec 2021 07:51:09 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v15] In-Reply-To: References: <4RcpFZMVKNqYuiowJOFwZuM_RDwKHAt8KyOPHz3S5c4=.23940329-cbf6-4f13-b044-eb91e4193f36@github.com> Message-ID: On Fri, 24 Dec 2021 15:29:43 GMT, Alan Bateman wrote: >> Markus KARG has updated the pull request incrementally with one additional commit since the last revision: >> >> Removing unused imports >> >> Signed-off-by: Markus Karg > >> Looking at the fact that selectableChannelOutput() utilizes Channels.newInputStream() it comes to my mind that [Trisha Gee reported](https://trishagee.com/2021/12/21/simple-network-connections-with-java-a-problem/) that Channels.newInputStream() produced an endless hanging of her apparently correct code, and it was gone once she replaced it by Channels.newReader(). While she reproduced that on a Mac, it might be the same cause even on Windows. Trisha currently is working on a reproducer, but she already detected that _her_ endless hanging is definitively a thread waiting for the blocking lock. This sounds similar to what you wrote above. So maybe the problem is not inside of TransferTo, but inside of ChannelInputStream? > > I think the hang in JDK-8278369 is specific to the Unix domain socket implementation of Pipe on Windows 10+ and Windows Server 2019+. More specifically, I think the issue is closing the sink after a large number of bytes has been written does not always cause a reader on the source side to wakeup with EOF. As a test, I changed the implementation to use TCP sockets unconditional (so it works like Windows 8 or Windows Server 2012 or 2016) and I cannot duplicate the issue. We need to look closer at this in the new year. > > I looked briefly at the chat server and it has a number of issues. There is a ArrayList mutated and read from several threads without synchronization. There are several issues that are possible when a chat client does not keep up, this will eventually cause the chat server to hang (a thread blocked on the PrintWriter) and all other threads will eventually blocking trying to echo the messages. So from a quick look, I don't think it is anything to do with the issues we are discussing here. > @AlanBateman I found [the cause of the problem](https://github.com/trishagee/socket-locking/pull/1#issuecomment-1002187644). The problem is that ChannelInputStream and ChannelOutputStream share the same blocking lock when created for the same IP connection, which is pretty straightforward, as a SocketChannel is bidirectional by nature, but has only one blocking setting for both directions. This shared lock is not documented anywhere, so how should an application programmer know about this? This should _either_ be documented in the Channels JavaDocs, or (even better) there should be separate blocking modes per direction. FYI, the channel reader does not produce the deadlock as it does not care about the blocking mode at all but simply expects non-blocking mode so it runs in an active loop (squandering power). Maybe we should align both implementations, reader and stream? WDYT? Good sleuthing! This is a design bug that goes back to JSR-51/Java 1.4 (https://bugs.openjdk.java.net/browse/JDK-4509080). SelectableChannel specifies blockingLock, which is okay for adaptors and the like when they are readable or writable, but not both. The socket adaptors have been re-implemented so as not to require the blockingLock during read/write operations so this limits the issue to the interop support that Channels provides. It's fixable with a bit of surgery. A first attempt would be for Channels.newInputStream to return sc.socket().getInputStream() when the channel is a SocketChannel. This requires dealing with cases where someone attempts to get an input stream from an unconnected or closed channel. A second attempt would be to add sun.nio.ch.SocketInputStream package sun.nio.ch; import java.io.IOException; import java.io.InputStream; public class SocketInputStream extends InputStream { private final SocketChannelImpl sc; public SocketInputStream(SocketChannelImpl sc) { this.sc = sc; } @Override public int read() throws IOException { byte[] a = new byte[1]; int n = read(a, 0, 1); return (n > 0) ? (a[0] & 0xff) : -1; } @Override public int read(byte[] b, int off, int len) throws IOException { return sc.blockingRead(b, off, len, 0); } @Override public int available() throws IOException { return sc.available(); } @Override public void close() throws IOException { sc.close(); } } The main question is whether it's worth doing. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Wed Dec 29 09:01:12 2021 From: duke at openjdk.java.net (Markus KARG) Date: Wed, 29 Dec 2021 09:01:12 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v16] In-Reply-To: References: Message-ID: On Sat, 25 Dec 2021 16:10:37 GMT, Markus KARG wrote: >> This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. > > Markus KARG has updated the pull request incrementally with three additional commits since the last revision: > > - separate test class for each large file test > - Base class only contains utilities and helpers > - Renamed AbstractTransferTo to TransferToBase as it is not abstract > Good sleuthing! This is a design bug that goes back to JSR-51/Java 1.4 (https://bugs.openjdk.java.net/browse/JDK-4509080) ... The main question is whether it's worth doing. Thank you for your ideas. On a gut level I would say I will try it out and when the result smells fine I will open a PR for further discussion. What intrigues me most is that solving this problem also would effectively result in potentially higher throughput of the overall system, as read and write are not mutual exclusive anymore. So from a first glimpse I would say it is worth doing it, but I need to give it more in-depth analysis. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From duke at openjdk.java.net Thu Dec 30 10:15:17 2021 From: duke at openjdk.java.net (Markus KARG) Date: Thu, 30 Dec 2021 10:15:17 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v15] In-Reply-To: References: <4RcpFZMVKNqYuiowJOFwZuM_RDwKHAt8KyOPHz3S5c4=.23940329-cbf6-4f13-b044-eb91e4193f36@github.com> Message-ID: <3y4DdQ1hOUjOKWxuZ4JCwl9abqXbjqXpf-Awx9qLoek=.7d21aa99-9527-4c0f-b4a8-148073f98e71@github.com> On Wed, 29 Dec 2021 07:47:58 GMT, Alan Bateman wrote: >>> Looking at the fact that selectableChannelOutput() utilizes Channels.newInputStream() it comes to my mind that [Trisha Gee reported](https://trishagee.com/2021/12/21/simple-network-connections-with-java-a-problem/) that Channels.newInputStream() produced an endless hanging of her apparently correct code, and it was gone once she replaced it by Channels.newReader(). While she reproduced that on a Mac, it might be the same cause even on Windows. Trisha currently is working on a reproducer, but she already detected that _her_ endless hanging is definitively a thread waiting for the blocking lock. This sounds similar to what you wrote above. So maybe the problem is not inside of TransferTo, but inside of ChannelInputStream? >> >> I think the hang in JDK-8278369 is specific to the Unix domain socket implementation of Pipe on Windows 10+ and Windows Server 2019+. More specifically, I think the issue is closing the sink after a large number of bytes has been written does not always cause a reader on the source side to wakeup with EOF. As a test, I changed the implementation to use TCP sockets unconditional (so it works like Windows 8 or Windows Server 2012 or 2016) and I cannot duplicate the issue. We need to look closer at this in the new year. >> >> I looked briefly at the chat server and it has a number of issues. There is a ArrayList mutated and read from several threads without synchronization. There are several issues that are possible when a chat client does not keep up, this will eventually cause the chat server to hang (a thread blocked on the PrintWriter) and all other threads will eventually blocking trying to echo the messages. So from a quick look, I don't think it is anything to do with the issues we are discussing here. > >> @AlanBateman I found [the cause of the problem](https://github.com/trishagee/socket-locking/pull/1#issuecomment-1002187644). The problem is that ChannelInputStream and ChannelOutputStream share the same blocking lock when created for the same IP connection, which is pretty straightforward, as a SocketChannel is bidirectional by nature, but has only one blocking setting for both directions. This shared lock is not documented anywhere, so how should an application programmer know about this? This should _either_ be documented in the Channels JavaDocs, or (even better) there should be separate blocking modes per direction. FYI, the channel reader does not produce the deadlock as it does not care about the blocking mode at all but simply expects non-blocking mode so it runs in an active loop (squandering power). Maybe we should align both implementations, reader and stream? WDYT? > > Good sleuthing! This is a design bug that goes back to JSR-51/Java 1.4 (https://bugs.openjdk.java.net/browse/JDK-4509080). SelectableChannel specifies blockingLock, which is okay for adaptors and the like when they are readable or writable, but not both. The socket adaptors have been re-implemented so as not to require the blockingLock during read/write operations so this limits the issue to the interop support that Channels provides. It's fixable with a bit of surgery. A first attempt would be for Channels.newInputStream to return sc.socket().getInputStream() when the channel is a SocketChannel. This requires dealing with cases where someone attempts to get an input stream from an unconnected or closed channel. A second attempt would be to add sun.nio.ch.SocketInputStream > > package sun.nio.ch; > > import java.io.IOException; > import java.io.InputStream; > > public class SocketInputStream extends InputStream { > private final SocketChannelImpl sc; > public SocketInputStream(SocketChannelImpl sc) { > this.sc = sc; > } > @Override > public int read() throws IOException { > byte[] a = new byte[1]; > int n = read(a, 0, 1); > return (n > 0) ? (a[0] & 0xff) : -1; > } > @Override > public int read(byte[] b, int off, int len) throws IOException { > return sc.blockingRead(b, off, len, 0); > } > @Override > public int available() throws IOException { > return sc.available(); > } > @Override > public void close() throws IOException { > sc.close(); > } > } > > The main question is whether it's worth doing. @AlanBateman @LanceAndersen Back to the original thread of this PR, are there more change requests or is it fine for merging? ------------- PR: https://git.openjdk.java.net/jdk/pull/6711 From alanb at openjdk.java.net Thu Dec 30 16:19:12 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 30 Dec 2021 16:19:12 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v15] In-Reply-To: References: <4RcpFZMVKNqYuiowJOFwZuM_RDwKHAt8KyOPHz3S5c4=.23940329-cbf6-4f13-b044-eb91e4193f36@github.com> Message-ID: On Wed, 29 Dec 2021 07:47:58 GMT, Alan Bateman wrote: >>> Looking at the fact that selectableChannelOutput() utilizes Channels.newInputStream() it comes to my mind that [Trisha Gee reported](https://trishagee.com/2021/12/21/simple-network-connections-with-java-a-problem/) that Channels.newInputStream() produced an endless hanging of her apparently correct code, and it was gone once she replaced it by Channels.newReader(). While she reproduced that on a Mac, it might be the same cause even on Windows. Trisha currently is working on a reproducer, but she already detected that _her_ endless hanging is definitively a thread waiting for the blocking lock. This sounds similar to what you wrote above. So maybe the problem is not inside of TransferTo, but inside of ChannelInputStream? >> >> I think the hang in JDK-8278369 is specific to the Unix domain socket implementation of Pipe on Windows 10+ and Windows Server 2019+. More specifically, I think the issue is closing the sink after a large number of bytes has been written does not always cause a reader on the source side to wakeup with EOF. As a test, I changed the implementation to use TCP sockets unconditional (so it works like Windows 8 or Windows Server 2012 or 2016) and I cannot duplicate the issue. We need to look closer at this in the new year. >> >> I looked briefly at the chat server and it has a number of issues. There is a ArrayList mutated and read from several threads without synchronization. There are several issues that are possible when a chat client does not keep up, this will eventually cause the chat server to hang (a thread blocked on the PrintWriter) and all other threads will eventually blocking trying to echo the messages. So from a quick look, I don't think it is anything to do with the issues we are discussing here. > >> @AlanBateman I found [the cause of the problem](https://github.com/trishagee/socket-locking/pull/1#issuecomment-1002187644). The problem is that ChannelInputStream and ChannelOutputStream share the same blocking lock when created for the same IP connection, which is pretty straightforward, as a SocketChannel is bidirectional by nature, but has only one blocking setting for both directions. This shared lock is not documented anywhere, so how should an application programmer know about this? This should _either_ be documented in the Channels JavaDocs, or (even better) there should be separate blocking modes per direction. FYI, the channel reader does not produce the deadlock as it does not care about the blocking mode at all but simply expects non-blocking mode so it runs in an active loop (squandering power). Maybe we should align both implementations, reader and stream? WDYT? > > Good sleuthing! This is a design bug that goes back to JSR-51/Java 1.4 (https://bugs.openjdk.java.net/browse/JDK-4509080). SelectableChannel specifies blockingLock, which is okay for adaptors and the like when they are readable or writable, but not both. The socket adaptors have been re-implemented so as not to require the blockingLock during read/write operations so this limits the issue to the interop support that Channels provides. It's fixable with a bit of surgery. A first attempt would be for Channels.newInputStream to return sc.socket().getInputStream() when the channel is a SocketChannel. This requires dealing with cases where someone attempts to get an input stream from an unconnected or closed channel. A second attempt would be to add sun.nio.ch.SocketInputStream > > package sun.nio.ch; > > import java.io.IOException; > import java.io.InputStream; > > public class SocketInputStream extends InputStream { > private final SocketChannelImpl sc; > public SocketInputStream(SocketChannelImpl sc) { > this.sc = sc; > } > @Override > public int read() throws IOException { > byte[] a = new byte[1]; > int n = read(a, 0, 1); > return (n > 0) ? (a[0] & 0xff) : -1; > } > @Override > public int read(byte[] b, int off, int len) throws IOException { > return sc.blockingRead(b, off, len, 0); > } > @Override > public int available() throws IOException { > return sc.available(); > } > @Override > public void close() throws IOException { > sc.close(); > } > } > > The main question is whether it's worth doing. > @AlanBateman @LanceAndersen Back to the original thread of this PR, are there more change requests or is it fine for merging? Lance was looking at the test update, I'm sure he'll get back to it after the holidays. ------------- PR: https://git.openjdk.java.net/jdk/pull/6711