From alanb at openjdk.org Thu Nov 2 15:02:12 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 2 Nov 2023 15:02:12 GMT Subject: RFR: 8318422: Allow poller threads be virtual threads Message-ID: Virtual threads doing blocking I/O rely on poller threads to unpark virtual threads when sockets are ready for I/O. Right now, the poller threads are dedicated platform threads that block waiting for I/O events. In some environments it would be better to have the poller thread be virtual threads so that handling of I/O events integrates better with the virtual thread scheduler and doesn't steal cycles from the carrier threads. The changes to support this are straight forward but it has required refactoring the internal Poller class so the changes might look more than they actually are. The changes mean the Poller implementation supports two modes. The default is changed on Linux to use the "virtual thread poller" mode. The default on other platforms is to use platform threads as before. As part of the refactor, the so-called "indirect" mode with updater threads is removed. This mode came from experimenting in the loom repo a long time ago and probably should have been removed before integration. ------------- Commit messages: - Sync up changes from loom repo - Merge - Merge - Initial commit Changes: https://git.openjdk.org/jdk/pull/16243/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16243&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8318422 Stats: 542 lines in 12 files changed: 167 ins; 198 del; 177 mod Patch: https://git.openjdk.org/jdk/pull/16243.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16243/head:pull/16243 PR: https://git.openjdk.org/jdk/pull/16243 From michaelm at openjdk.org Thu Nov 2 15:02:14 2023 From: michaelm at openjdk.org (Michael McMahon) Date: Thu, 2 Nov 2023 15:02:14 GMT Subject: RFR: 8318422: Allow poller threads be virtual threads In-Reply-To: References: Message-ID: On Wed, 18 Oct 2023 10:31:32 GMT, Alan Bateman wrote: > Virtual threads doing blocking I/O rely on poller threads to unpark virtual threads when sockets are ready for I/O. Right now, the poller threads are dedicated platform threads that block waiting for I/O events. In some environments it would be better to have the poller thread be virtual threads so that handling of I/O events integrates better with the virtual thread scheduler and doesn't steal cycles from the carrier threads. > > The changes to support this are straight forward but it has required refactoring the internal Poller class so the changes might look more than they actually are. The changes mean the Poller implementation supports two modes. The default is changed on Linux to use the "virtual thread poller" mode. The default on other platforms is to use platform threads as before. As part of the refactor, the so-called "indirect" mode with updater threads is removed. This mode came from experimenting in the loom repo a long time ago and probably should have been removed before integration. src/java.base/share/classes/sun/nio/ch/Poller.java line 44: > 42: */ > 43: abstract class Poller { > 44: private static Pollers POLLERS; Suggestion: private static final Pollers POLLERS; ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16243#discussion_r1378920960 From alanb at openjdk.org Thu Nov 2 15:23:03 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 2 Nov 2023 15:23:03 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions In-Reply-To: References: Message-ID: On Wed, 18 Oct 2023 16:20:55 GMT, Pavel Rappo wrote: > There are a few builder style APIs for immutable objects that have `withXXX` methods that return an instance with the change requested. A `withExtension(String ext)` could handle the common cases for removing, replacing, and adding an extension. This looks like a good direction to try out, seems better than cluttering Path with several methods to aid adding or stripping of an extension. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1790944279 From michaelm at openjdk.org Thu Nov 2 15:40:02 2023 From: michaelm at openjdk.org (Michael McMahon) Date: Thu, 2 Nov 2023 15:40:02 GMT Subject: RFR: 8318422: Allow poller threads be virtual threads In-Reply-To: References: Message-ID: On Wed, 18 Oct 2023 10:31:32 GMT, Alan Bateman wrote: > Virtual threads doing blocking I/O rely on poller threads to unpark virtual threads when sockets are ready for I/O. Right now, the poller threads are dedicated platform threads that block waiting for I/O events. In some environments it would be better to have the poller thread be virtual threads so that handling of I/O events integrates better with the virtual thread scheduler and doesn't steal cycles from the carrier threads. > > The changes to support this are straight forward but it has required refactoring the internal Poller class so the changes might look more than they actually are. The changes mean the Poller implementation supports two modes. The default is changed on Linux to use the "virtual thread poller" mode. The default on other platforms is to use platform threads as before. As part of the refactor, the so-called "indirect" mode with updater threads is removed. This mode came from experimenting in the loom repo a long time ago and probably should have been removed before integration. src/java.base/share/classes/sun/nio/ch/Poller.java line 89: > 87: * Returns the poller's file descriptor, used when the read and write poller threads > 88: * are virtual threads. > 89: * Would it be useful to add some more explanation here like: "When there is no activity on any of a subpoller's descriptors, the descriptor of the subpoller is registered with the master poller" ? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16243#discussion_r1380349217 From alanb at openjdk.org Thu Nov 2 16:31:10 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 2 Nov 2023 16:31:10 GMT Subject: RFR: 8318422: Allow poller threads be virtual threads In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 15:37:41 GMT, Michael McMahon wrote: > Would it be useful to add some more explanation here like: "When there is no activity on any of a subpoller's descriptors, the descriptor of the subpoller is registered with the master poller" ? The moment on Mode.VTHREAD_POLLERS explains this. It also links to the Karsten/Barghi paper where a similar scheme is presented. Do you think that's enough? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16243#discussion_r1380426367 From rriggs at openjdk.org Thu Nov 2 17:34:02 2023 From: rriggs at openjdk.org (Roger Riggs) Date: Thu, 2 Nov 2023 17:34:02 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions In-Reply-To: References: Message-ID: On Tue, 17 Oct 2023 19:52:14 GMT, Brian Burkhalter wrote: > Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. As suggested, I would rename `removeExtension` to `withoutExtension` and replace `addExtension` with `withExtension`. Please rephrase or reword as you see fit: /** * Returns a Path with the {@code extension}, replacing the existing extension, if any. * If the extension is non-null and non-empty, the "." and the extension * is appended to the path returned by {@link #withoutExtension()}, * otherwise the path returned by {@link #withoutExtension()} is returned. * * @implSpec * The default implementation is equivalent for this path to: * * {@snippet lang="java" : * Path p = withoutExtension(); * if (extension == null || extension.isEmpty()) { * return p; * } else { * return p.resolveSibling(p.getFileName() + "." + extension); * } *} * @param extension * the extension to add, may be {@code null} * * @return a Path with the {@code extension}, replacing the existing extension, if any * * @see #getExtension * @see #withoutExtension * * @since 22 */ default Path withExtension(String extension) { Path path = withoutExtension(); if (extension == null || extension.isEmpty()) return path; return path.resolveSibling(path.getFileName() + "." + extension); } Likely there is a more efficient implementation that does not create unnecessary Path intermediate objects. This is the same as suggested : https://github.com/openjdk/jdk/pull/16226#issuecomment-1768659315 ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1791210875 PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1791216671 From bpb at openjdk.org Thu Nov 2 17:58:01 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 2 Nov 2023 17:58:01 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions In-Reply-To: References: Message-ID: On Tue, 17 Oct 2023 19:52:14 GMT, Brian Burkhalter wrote: > Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. Thanks for the suggestion; I think it looks promising. I wonder however whether the notion of `withoutExtension` needs to be explicit. * Path p = withoutExtension(); * if (extension == null || extension.isEmpty()) { * return p; * ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1791264920 From rriggs at openjdk.org Thu Nov 2 19:11:03 2023 From: rriggs at openjdk.org (Roger Riggs) Date: Thu, 2 Nov 2023 19:11:03 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions In-Reply-To: References: Message-ID: On Tue, 17 Oct 2023 19:52:14 GMT, Brian Burkhalter wrote: > Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. The use of `withExtension("")` would be equivalent to `withoutExtension()` so the later could be omitted for a simpler API. And I think that's a reasonable simplification; replacing the extension is much more frequent operation than than dropping the extension. The spec will need to explain/define how to trim off the extension, so the new extension can be added. But it could be defined in terms of `path.getFileName()` and perhaps `path.resolveSibling`. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1791393901 From michaelm at openjdk.org Fri Nov 3 09:39:02 2023 From: michaelm at openjdk.org (Michael McMahon) Date: Fri, 3 Nov 2023 09:39:02 GMT Subject: RFR: 8318422: Allow poller threads be virtual threads In-Reply-To: References: Message-ID: On Wed, 18 Oct 2023 10:31:32 GMT, Alan Bateman wrote: > Virtual threads doing blocking I/O rely on poller threads to unpark virtual threads when sockets are ready for I/O. Right now, the poller threads are dedicated platform threads that block waiting for I/O events. In some environments it would be better to have the poller thread be virtual threads so that handling of I/O events integrates better with the virtual thread scheduler and doesn't steal cycles from the carrier threads. > > The changes to support this are straight forward but it has required refactoring the internal Poller class so the changes might look more than they actually are. The changes mean the Poller implementation supports two modes. The default is changed on Linux to use the "virtual thread poller" mode. The default on other platforms is to use platform threads as before. As part of the refactor, the so-called "indirect" mode with updater threads is removed. This mode came from experimenting in the loom repo a long time ago and probably should have been removed before integration. Looks good to me. ------------- Marked as reviewed by michaelm (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16243#pullrequestreview-1712200713 From alanb at openjdk.org Fri Nov 3 14:01:36 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 3 Nov 2023 14:01:36 GMT Subject: RFR: 8318422: Allow poller threads be virtual threads [v2] In-Reply-To: References: Message-ID: > Virtual threads doing blocking I/O rely on poller threads to unpark virtual threads when sockets are ready for I/O. Right now, the poller threads are dedicated platform threads that block waiting for I/O events. In some environments it would be better to have the poller thread be virtual threads so that handling of I/O events integrates better with the virtual thread scheduler and doesn't steal cycles from the carrier threads. > > The changes to support this are straight forward but it has required refactoring the internal Poller class so the changes might look more than they actually are. The changes mean the Poller implementation supports two modes. The default is changed on Linux to use the "virtual thread poller" mode. The default on other platforms is to use platform threads as before. As part of the refactor, the so-called "indirect" mode with updater threads is removed. This mode came from experimenting in the loom repo a long time ago and probably should have been removed before integration. Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: - VMOutOfMemoryException001.java should be excluded on all platforms - Merge - Sync up changes from loom repo - Merge - Merge - Initial commit ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16243/files - new: https://git.openjdk.org/jdk/pull/16243/files/9588be88..87136dde Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16243&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16243&range=00-01 Stats: 6428 lines in 139 files changed: 1949 ins; 2117 del; 2362 mod Patch: https://git.openjdk.org/jdk/pull/16243.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16243/head:pull/16243 PR: https://git.openjdk.org/jdk/pull/16243 From alanb at openjdk.org Fri Nov 3 14:01:38 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 3 Nov 2023 14:01:38 GMT Subject: RFR: 8318422: Allow poller threads be virtual threads [v2] In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 09:35:52 GMT, Michael McMahon wrote: > Looks good to me. Thanks. I've had to one additional update to ProblemList-Virtual.txt to excludevmTestbase/nsk/jdi/VMOutOfMemoryException/VMOutOfMemoryException001/VMOutOfMemoryException001.java on all platforms. It had been excluded on Windows only but it's quite fragile and we tickle it to fail on Linux with these changes. I had a brief exchange with @plummercj in JDK-8285417 about this a few days ago. It's excluded on all platforms in the loom repo already so I should have remembered to include it in this update. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16243#issuecomment-1792487367 From michaelm at openjdk.org Fri Nov 3 15:05:05 2023 From: michaelm at openjdk.org (Michael McMahon) Date: Fri, 3 Nov 2023 15:05:05 GMT Subject: RFR: 8318422: Allow poller threads be virtual threads [v2] In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 14:01:36 GMT, Alan Bateman wrote: >> Virtual threads doing blocking I/O rely on poller threads to unpark virtual threads when sockets are ready for I/O. Right now, the poller threads are dedicated platform threads that block waiting for I/O events. In some environments it would be better to have the poller thread be virtual threads so that handling of I/O events integrates better with the virtual thread scheduler and doesn't steal cycles from the carrier threads. >> >> The changes to support this are straight forward but it has required refactoring the internal Poller class so the changes might look more than they actually are. The changes mean the Poller implementation supports two modes. The default is changed on Linux to use the "virtual thread poller" mode. The default on other platforms is to use platform threads as before. As part of the refactor, the so-called "indirect" mode with updater threads is removed. This mode came from experimenting in the loom repo a long time ago and probably should have been removed before integration. > > Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: > > - VMOutOfMemoryException001.java should be excluded on all platforms > - Merge > - Sync up changes from loom repo > - Merge > - Merge > - Initial commit Latest change is good. ------------- Marked as reviewed by michaelm (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16243#pullrequestreview-1712913029 From bpb at openjdk.org Fri Nov 3 18:38:05 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 3 Nov 2023 18:38:05 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 19:08:45 GMT, Roger Riggs wrote: > The spec will need to explain/define how to trim off the extension, so the new extension can be added. Something like this perhaps for the nominal case? Path p; String name = p.getFileName().toString(); String ext = p.getExtension(); String s = name.substring(0, name.lastIndexOf(ext) - 1); Path stripped = p.resolveSibling(s); ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1792947777 From rriggs at openjdk.org Fri Nov 3 19:31:02 2023 From: rriggs at openjdk.org (Roger Riggs) Date: Fri, 3 Nov 2023 19:31:02 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions In-Reply-To: References: Message-ID: <1TGXNr0UsTo_R0ejnqpnElGbgO9iBbz6WznZmdiO8ug=.bd09168c-7199-4b4e-8911-5d3387762b30@github.com> On Tue, 17 Oct 2023 19:52:14 GMT, Brian Burkhalter wrote: > Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. Almost, except for the detail that both "photo" and "photo." have an empty string as the extension so the trailing ".", if any, should be dropped. Path p; String name = p.getFileName().toString(); String ext = p.getExtension(); int index = name.length() - ext.length(); if (name.getChar(index - 1) == DOT) index--; String s = name.substring(0, indexOfExt); Path stripped = p.resolveSibling(s); Keeping the `withoutExtension` method provides a place for that as a building block for `withExtension`. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1792986358 From bpb at openjdk.org Fri Nov 3 20:29:07 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 3 Nov 2023 20:29:07 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions In-Reply-To: <1TGXNr0UsTo_R0ejnqpnElGbgO9iBbz6WznZmdiO8ug=.bd09168c-7199-4b4e-8911-5d3387762b30@github.com> References: <1TGXNr0UsTo_R0ejnqpnElGbgO9iBbz6WznZmdiO8ug=.bd09168c-7199-4b4e-8911-5d3387762b30@github.com> Message-ID: <1th_8HqpGYo4y1R8lBHL9JKgXMplEwWk-1cKPBna5gQ=.f217d83b-46e9-407e-8f2a-5e8141866b52@github.com> On Fri, 3 Nov 2023 19:28:24 GMT, Roger Riggs wrote: > Keeping the `withoutExtension` method provides a place for that as a building block for `withExtension`. Might be a good reason to keep it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1793054250 From alanb at openjdk.org Sat Nov 4 06:55:21 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sat, 4 Nov 2023 06:55:21 GMT Subject: Integrated: 8318422: Allow poller threads be virtual threads In-Reply-To: References: Message-ID: On Wed, 18 Oct 2023 10:31:32 GMT, Alan Bateman wrote: > Virtual threads doing blocking I/O rely on poller threads to unpark virtual threads when sockets are ready for I/O. Right now, the poller threads are dedicated platform threads that block waiting for I/O events. In some environments it would be better to have the poller thread be virtual threads so that handling of I/O events integrates better with the virtual thread scheduler and doesn't steal cycles from the carrier threads. > > The changes to support this are straight forward but it has required refactoring the internal Poller class so the changes might look more than they actually are. The changes mean the Poller implementation supports two modes. The default is changed on Linux to use the "virtual thread poller" mode. The default on other platforms is to use platform threads as before. As part of the refactor, the so-called "indirect" mode with updater threads is removed. This mode came from experimenting in the loom repo a long time ago and probably should have been removed before integration. This pull request has now been integrated. Changeset: c099cf53 Author: Alan Bateman URL: https://git.openjdk.org/jdk/commit/c099cf53f25496c99629dc578045aa5186e1109d Stats: 545 lines in 13 files changed: 167 ins; 198 del; 180 mod 8318422: Allow poller threads be virtual threads Reviewed-by: michaelm ------------- PR: https://git.openjdk.org/jdk/pull/16243 From lancea at openjdk.org Tue Nov 7 16:55:41 2023 From: lancea at openjdk.org (Lance Andersen) Date: Tue, 7 Nov 2023 16:55:41 GMT Subject: Integrated: 8314891: Additional Zip64 extra header validation In-Reply-To: References: Message-ID: On Sat, 9 Sep 2023 14:33:53 GMT, Lance Andersen wrote: > Please review this PR which improves the Zip64 extra header validation: > > - Throw a ZipException If the extra len field is 0 and : > -- size, csize, or loc offset are set to 0xFFFFFFFF > -- disk starting number is set to 0xFFFF > > - We have a valid size for the Zip64 extra header but we are missing the csize or loc fields if they are expected to be part of the header > > Mach5 tiers 1-3 are clean This pull request has now been integrated. Changeset: 82747132 Author: Lance Andersen URL: https://git.openjdk.org/jdk/commit/82747132b07e288c1e6c2ba3392748c7718e994a Stats: 1122 lines in 3 files changed: 1037 ins; 51 del; 34 mod 8314891: Additional Zip64 extra header validation Reviewed-by: coffeys ------------- PR: https://git.openjdk.org/jdk/pull/15650 From duke at openjdk.org Wed Nov 8 16:31:16 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Wed, 8 Nov 2023 16:31:16 GMT Subject: RFR: 8314891: Additional Zip64 extra header validation [v8] In-Reply-To: References: Message-ID: On Mon, 23 Oct 2023 19:12:57 GMT, Lance Andersen wrote: >> Please review this PR which improves the Zip64 extra header validation: >> >> - Throw a ZipException If the extra len field is 0 and : >> -- size, csize, or loc offset are set to 0xFFFFFFFF >> -- disk starting number is set to 0xFFFF >> >> - We have a valid size for the Zip64 extra header but we are missing the csize or loc fields if they are expected to be part of the header >> >> Mach5 tiers 1-3 are clean > > Lance Andersen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision: > > - Merge branch 'master' into JDK-8314891 > - Add missing space > - Revamp isZip64ExtBlockSizeValid > - Merge branch 'master' into JDK-8314891 > - Merge branch 'master' into JDK-8314891 > - Remove tab(s) from comment > - Added additional tests, along with additional cleanup and refactoring > - Clean up some minor formatting issues > - Additional Zip64 extra header validation @LanceAndersen I noticed that this PR did not update `ZipInputStream.readLOC` to perform consistency validation between expected and actual extra field size and values. Any particular reason why processing of LOC headers was not made consistent with CEN? ------------- PR Comment: https://git.openjdk.org/jdk/pull/15650#issuecomment-1802241544 From lancea at openjdk.org Wed Nov 8 17:30:12 2023 From: lancea at openjdk.org (Lance Andersen) Date: Wed, 8 Nov 2023 17:30:12 GMT Subject: RFR: 8314891: Additional Zip64 extra header validation [v8] In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 16:27:56 GMT, Eirik Bjorsnos wrote: > @LanceAndersen > > I noticed that this PR did not update `ZipInputStream.readLOC` to perform consistency validation between expected and actual extra field size and values. Any particular reason why processing of LOC headers was not made consistent with CEN? Intentional, as this was a follow on to the updates which were done previously to the CEN work in August, this is follow on cleanup. Updates to ZipInputStream would be done separately under a separate PR or could be done via your work on 8303866 ------------- PR Comment: https://git.openjdk.org/jdk/pull/15650#issuecomment-1802340986 From lancea at openjdk.org Wed Nov 8 20:06:37 2023 From: lancea at openjdk.org (Lance Andersen) Date: Wed, 8 Nov 2023 20:06:37 GMT Subject: RFR: 8316141: Improve CEN header validation checking Message-ID: Please review this PR which enhances the existing CEN header validation checking to ensure that the size of the CEN Header + name length + comment length + extra length do not exceed 65,535 bytes per the PKWare APP.NOTE 4.4.10, 4.4.11, & 4.4.12. Also check that current CEN header will not exceed the length of the CEN array. Mach 5 tiers 1-3 are clean with this change. ------------- Commit messages: - Initial changes for JDK-8316141 Changes: https://git.openjdk.org/jdk/pull/16570/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16570&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8316141 Stats: 335 lines in 4 files changed: 322 ins; 4 del; 9 mod Patch: https://git.openjdk.org/jdk/pull/16570.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16570/head:pull/16570 PR: https://git.openjdk.org/jdk/pull/16570 From duke at openjdk.org Wed Nov 8 20:26:56 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Wed, 8 Nov 2023 20:26:56 GMT Subject: RFR: 8316141: Improve CEN header validation checking In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 19:59:34 GMT, Lance Andersen wrote: > Please review this PR which enhances the existing CEN header validation checking to ensure that the > size of the CEN Header + name length + comment length + extra length do not exceed 65,535 bytes per the PKWare APP.NOTE 4.4.10, 4.4.11, & 4.4.12. Also check that current CEN header will not exceed the length of the CEN array. > > Mach 5 tiers 1-3 are clean with this change. Perhaps the PR/issue title could be more specific in describing what is being validated? Something like "Validate the combined length of CEN header fields"? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16570#issuecomment-1802596701 From duke at openjdk.org Wed Nov 8 21:27:58 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Wed, 8 Nov 2023 21:27:58 GMT Subject: RFR: 8316141: Improve CEN header validation checking In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 19:59:34 GMT, Lance Andersen wrote: > Please review this PR which enhances the existing CEN header validation checking to ensure that the > size of the CEN Header + name length + comment length + extra length do not exceed 65,535 bytes per the PKWare APP.NOTE 4.4.10, 4.4.11, & 4.4.12. Also check that current CEN header will not exceed the length of the CEN array. > > Mach 5 tiers 1-3 are clean with this change. LGTM, although I'm not sure I follow the underlying motivation of this stricter validation. What problem is being solved here? APPNOTE.TXT uses the phrase `SHOULD NOT`. Even if the spec is not an RFC, RFC2119 defines `SHOULD NOT` as: ```This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label. I would expect our producer `ZipOutputStream` to be stricter than our consumers in this case, honoring Postel's law. From a implementation robustness perspective, the individual lengths are already validated, it's just the combined clause that is now enforced in this PR. That said, here are some comments inline: src/java.base/share/classes/java/util/zip/ZipFile.java line 1225: > 1223: int elen = CENEXT(cen, pos); > 1224: int clen = CENCOM(cen, pos); > 1225: long headerSize = (long)CENHDR + nlen + clen + elen; Since CENHDR is 46 and nlen, clen, elen are all unsigned shorts, this sum cannot possibly overflow an int. Is the long conversion necessary? The specification uses the term "combined length", would it be better to name the local variable `combinedLength` instead? src/java.base/share/classes/java/util/zip/ZipFile.java line 1235: > 1233: > 1234: if (elen > 0 && !DISABLE_ZIP64_EXTRA_VALIDATION) { > 1235: checkExtraFields(pos, entryPos + nlen, elen); The naming of `entryPos` confused me here. The name indicates it is the position where the CEN header starts, but we already have `pos` for that. (It actually contains the position where the encoded name starts) So perhaps it should be renamed to `namePos` or `npos` to make future maintenance less confusing? Also, I actually liked the `extraStartingOffset` local variable, having a name makes the code easier to follow than just `entryPos + nlen`. But perhaps `extraPos` is shorter and more consistent with other uses of `pos`? So perhaps: `long extraPos = pos + CENHDR + nlen` ? src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java line 1596: > 1594: throw new ZipException("invalid CEN header (unsupported compression method: " + method + ")"); > 1595: } > 1596: long headerSize = (long)CENHDR + nlen + clen + elen; If the corresponding ZipFile local variable is renamed, this should also be updated. src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java line 1668: > 1666: int tagBlockSize = SH(cen, currentOffset); > 1667: currentOffset += Short.BYTES; > 1668: long tagBlockEndingOffset = (long)currentOffset + tagBlockSize; I think my ZipFile comment also applies here. ------------- PR Review: https://git.openjdk.org/jdk/pull/16570#pullrequestreview-1721212606 PR Review Comment: https://git.openjdk.org/jdk/pull/16570#discussion_r1387163396 PR Review Comment: https://git.openjdk.org/jdk/pull/16570#discussion_r1387177874 PR Review Comment: https://git.openjdk.org/jdk/pull/16570#discussion_r1387194439 PR Review Comment: https://git.openjdk.org/jdk/pull/16570#discussion_r1387195085 From smarks at openjdk.org Thu Nov 9 05:18:57 2023 From: smarks at openjdk.org (Stuart Marks) Date: Thu, 9 Nov 2023 05:18:57 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions In-Reply-To: References: Message-ID: On Tue, 17 Oct 2023 19:52:14 GMT, Brian Burkhalter wrote: > Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. The current state of the `removeExtension` code drops the parent part of the path if there is no extension or if there is an empty extension (i.e., trailing dot): jshell> Path.of("a/foo").removeExtension() $3 ==> foo jshell> Path.of("a/foo.").removeExtension() $4 ==> foo. Probably need to add test cases for this. I have additional questions about whether a trailing dot should or shouldn't be removed in this case; but maybe that's still currently being discussed? Not sure where that stands at the moment. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1803176423 From lancea at openjdk.org Thu Nov 9 17:24:58 2023 From: lancea at openjdk.org (Lance Andersen) Date: Thu, 9 Nov 2023 17:24:58 GMT Subject: RFR: 8316141: Improve CEN header validation checking In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 19:59:34 GMT, Lance Andersen wrote: > Please review this PR which enhances the existing CEN header validation checking to ensure that the > size of the CEN Header + name length + comment length + extra length do not exceed 65,535 bytes per the PKWare APP.NOTE 4.4.10, 4.4.11, & 4.4.12. Also check that current CEN header will not exceed the length of the CEN array. > > Mach 5 tiers 1-3 are clean with this change. Thank you for the comments. See my replies below. Regarding you comment about checking whether or not to check if the combined length of the CEN header + name length + comment length + extra length > 65K bytes, I chose to add this given the strong wording given this a really old spec. That being said, I do not object to removing the validation if that is the overall preference. zerror("invalid CEN header (bad header size)"); } ------------- PR Review: https://git.openjdk.org/jdk/pull/16570#pullrequestreview-1723102540 From lancea at openjdk.org Thu Nov 9 17:25:02 2023 From: lancea at openjdk.org (Lance Andersen) Date: Thu, 9 Nov 2023 17:25:02 GMT Subject: RFR: 8316141: Improve CEN header validation checking In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 20:26:32 GMT, Eirik Bjorsnos wrote: >> Please review this PR which enhances the existing CEN header validation checking to ensure that the >> size of the CEN Header + name length + comment length + extra length do not exceed 65,535 bytes per the PKWare APP.NOTE 4.4.10, 4.4.11, & 4.4.12. Also check that current CEN header will not exceed the length of the CEN array. >> >> Mach 5 tiers 1-3 are clean with this change. > > src/java.base/share/classes/java/util/zip/ZipFile.java line 1225: > >> 1223: int elen = CENEXT(cen, pos); >> 1224: int clen = CENCOM(cen, pos); >> 1225: long headerSize = (long)CENHDR + nlen + clen + elen; > > Since CENHDR is 46 and nlen, clen, elen are all unsigned shorts, this sum cannot possibly overflow an int. Is the long conversion necessary? > > The specification uses the term "combined length", would it be better to name the local variable `combinedLength` instead? I can remove the cast as that was a holdover. I chose to make this a long knowing that it would not overflow but an overflow while unlikely could occur depending on the value of `pos` in the statement below if (headerSize > 0xFFFF || pos + headerSize > cen.length - ENDHDR) { zerror("invalid CEN header (bad header size)"); } I could keep headerSize an int and then move the cast to the if statement: if (headerSize > 0xFFFF || (long)pos + headerSize > cen.length - ENDHDR) { zerror("invalid CEN header (bad header size)"); } I decided making headerSize a long might be clearer but do not have a strong preference and will go with the consensus As far as the name, I don't have a strong preference, but not sure combinedLength is any better > src/java.base/share/classes/java/util/zip/ZipFile.java line 1235: > >> 1233: >> 1234: if (elen > 0 && !DISABLE_ZIP64_EXTRA_VALIDATION) { >> 1235: checkExtraFields(pos, entryPos + nlen, elen); > > The naming of `entryPos` confused me here. The name indicates it is the position where the CEN header starts, but we already have `pos` for that. (It actually contains the position where the encoded name starts) > > So perhaps it should be renamed to `namePos` or `npos` to make future maintenance less confusing? > > Also, I actually liked the `extraStartingOffset` local variable, having a name makes the code easier to follow than just `entryPos + nlen`. But perhaps `extraPos` is shorter and more consistent with other uses of `pos`? > > So perhaps: `long extraPos = pos + CENHDR + nlen` ? `entryPos` was the name of the field from a previous PR so I did not see a need to change it and decided there was no need to keep extraStartingOffset given the change in validation above. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16570#discussion_r1388318963 PR Review Comment: https://git.openjdk.org/jdk/pull/16570#discussion_r1388313503 From bpb at openjdk.org Thu Nov 9 17:33:03 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 9 Nov 2023 17:33:03 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions In-Reply-To: References: Message-ID: <2n3hwHxmYjxrQy_ydW2yrV-wj50lEvC3buv5o-tUod4=.ccad2e2f-d710-4a47-93a9-acc1882a8287@github.com> On Thu, 9 Nov 2023 05:16:22 GMT, Stuart Marks wrote: > Probably need to add test cases for this. Testing definitely needs to be broadened, but I think it should wait until there is consensus on the proposed API(s). > I have additional questions about whether a trailing dot should or shouldn't be removed in this case; but maybe that's still currently being discussed? Not sure where that stands at the moment. At the moment I think that everything except perhaps `Path.getExtension` is still pretty fluid. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1804261278 From dfuchs at openjdk.org Tue Nov 14 17:56:38 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Tue, 14 Nov 2023 17:56:38 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out Message-ID: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> Please find here a fix for a timeout issue observed in DatagramChannel/Connect.java The suspicion is that the responder may have received some message from some other source and replied to that, leaving the initiator waiting forever. The proposed fix makes sure the messages exchanged are unique and identifiable and ensure that both the initiator and responder will discard any message that do not have the expected content. Additional logging should help with diagnosis if the test fails again. ------------- Commit messages: - Minor cleanup 2 - Minor cleanup - 8310807 Changes: https://git.openjdk.org/jdk/pull/16661/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16661&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8310807 Stats: 80 lines in 1 file changed: 42 ins; 0 del; 38 mod Patch: https://git.openjdk.org/jdk/pull/16661.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16661/head:pull/16661 PR: https://git.openjdk.org/jdk/pull/16661 From dfuchs at openjdk.org Tue Nov 14 18:01:44 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Tue, 14 Nov 2023 18:01:44 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out [v2] In-Reply-To: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> Message-ID: > Please find here a fix for a timeout issue observed in DatagramChannel/Connect.java > > The suspicion is that the responder may have received some message from some other source and replied to that, leaving the initiator waiting forever. > > The proposed fix makes sure the messages exchanged are unique and identifiable and ensure that both the initiator and responder will discard any message that do not have the expected content. > > Additional logging should help with diagnosis if the test fails again. Daniel Fuchs has updated the pull request incrementally with one additional commit since the last revision: Minor cleanup 3 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16661/files - new: https://git.openjdk.org/jdk/pull/16661/files/b659b368..958d5b76 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16661&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16661&range=00-01 Stats: 5 lines in 1 file changed: 1 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16661.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16661/head:pull/16661 PR: https://git.openjdk.org/jdk/pull/16661 From Alan.Bateman at oracle.com Tue Nov 14 19:02:13 2023 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 14 Nov 2023 19:02:13 +0000 Subject: return-value check of setsockopt ? In-Reply-To: References: Message-ID: <637ce73e-8ad0-4617-88f0-012ab2171325@oracle.com> On 14/11/2023 15:16, Baesken, Matthias wrote: > > Hi, while looking at some net/nio coding, I noticed? that at a lot of > places the return value of setsockopt is checked : > > https://github.com/search?q=repo%3Aopenjdk%2Fjdk+%22setsockopt%22&type=code > > > but the check is omitted at some places . > The setsockopt usages in ping4 and ping6 (Inet4Address.c and Inet6Address.c) should probably have checks. There are a few others where they are set when supported. An argument could be made to have these handle other errors. So I think it means working through them on a case-by-case basis. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From bpb at openjdk.org Tue Nov 14 19:44:55 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 14 Nov 2023 19:44:55 GMT Subject: RFR: 8315273: (fs) Path.toRealPath(LinkOption.NOFOLLOW_LINKS) fails when "../../" follows a link (win) [v4] In-Reply-To: <9TrqNiqFM-WUzVO2G_pQVtAeI06TwRt1dR1cO2zNemk=.580d210b-e5a2-4b5d-956f-ca5d286844e1@github.com> References: <9TrqNiqFM-WUzVO2G_pQVtAeI06TwRt1dR1cO2zNemk=.580d210b-e5a2-4b5d-956f-ca5d286844e1@github.com> Message-ID: <2rdSqg9wJiKXpdxhatwIprCSFmosoDaABVF8A8mkDjI=.bfdfe79a-afef-4c94-8945-4993ab5bba9e@github.com> > Windows implementation of integrated pull request #15397. The test java/nio/file/Path/ToRealPath.java is also removed from the problem list. Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: - Merge - Merge - 8315273: Add bug ID to test - 8315273: (fs) Path.toRealPath(LinkOption.NOFOLLOW_LINKS) fails when "../../" follows a link (win) ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15525/files - new: https://git.openjdk.org/jdk/pull/15525/files/dd3ce6d1..e1248412 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15525&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15525&range=02-03 Stats: 639352 lines in 1404 files changed: 93828 ins; 478008 del; 67516 mod Patch: https://git.openjdk.org/jdk/pull/15525.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15525/head:pull/15525 PR: https://git.openjdk.org/jdk/pull/15525 From bpb at openjdk.org Wed Nov 15 01:41:53 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 15 Nov 2023 01:41:53 GMT Subject: RFR: 4860681: (bf) Add CharBuffer absolute bulk put method for CharSequence [v7] In-Reply-To: References: Message-ID: > Add to `java.nio.CharBuffer` an absolute bulk put method which takes a `CharSequence` as its source of `char`s. Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 10 additional commits since the last revision: - 4860681: Fix java/nio/Buffer/CopyDirectMemory.java build failure - Merge - 4860681: Make test use RandomFactory - Merge - 4860681: Heap and direct buffer specializations - 4860681: Change some {@code ...} to ... to prevent line breaking - 4860681: Modify existing test - Merge - 4860681: Add spaces to parameter lists in @see tag - 4860681: (bf) Add CharBuffer absolute bulk put method for CharSequence ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13744/files - new: https://git.openjdk.org/jdk/pull/13744/files/2ea96eed..34c828a4 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13744&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13744&range=05-06 Stats: 684320 lines in 3152 files changed: 120896 ins; 487279 del; 76145 mod Patch: https://git.openjdk.org/jdk/pull/13744.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13744/head:pull/13744 PR: https://git.openjdk.org/jdk/pull/13744 From bpb at openjdk.org Wed Nov 15 01:41:55 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 15 Nov 2023 01:41:55 GMT Subject: RFR: 4860681: (bf) Add CharBuffer absolute bulk put method for CharSequence [v5] In-Reply-To: References: Message-ID: <1elxdTpUiMPi3w6-o1ulvNkjSMhAaSzSWPLG2Zfmd78=.1f3ca5dd-e290-4c16-946a-33bb6c10138e@github.com> On Mon, 9 Oct 2023 21:15:43 GMT, Brian Burkhalter wrote: >> src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template line 542: >> >>> 540: sb.getChars(start, start + count, buf, 0); >>> 541: } else if (csq instanceof StringBuffer sb) { >>> 542: sb.getChars(start, start + count, buf, 0); >> >> Can the allocation and double copy be avoided? >> But I guess the combinatorics of latin1/utf16 string encoding and the memory and byte order make it difficult. > > Back when I was originally developing this change I could not see any better way to do this part. > Can the allocation and double copy be avoided? The performance improvement in the heap and direct implementations versus the `X-Buffer` base implementation comes from using `getChars` to process a chunk of `char`s at once instead of doing them one at a time with `charAt`. The heap implementation can avoid the allocation and double copy by passing in its array to `getChars` directly, but in the direct implementation this is not possible. I could not identify a better way for the direct version. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13744#discussion_r1393525298 From msheppar at openjdk.org Wed Nov 15 14:43:31 2023 From: msheppar at openjdk.org (Mark Sheppard) Date: Wed, 15 Nov 2023 14:43:31 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out [v2] In-Reply-To: References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> Message-ID: On Tue, 14 Nov 2023 18:01:44 GMT, Daniel Fuchs wrote: >> Please find here a fix for a timeout issue observed in DatagramChannel/Connect.java >> >> The suspicion is that the responder may have received some message from some other source and replied to that, leaving the initiator waiting forever. >> >> The proposed fix makes sure the messages exchanged are unique and identifiable and ensure that both the initiator and responder will discard any message that do not have the expected content. >> >> Additional logging should help with diagnosis if the test fails again. > > Daniel Fuchs has updated the pull request incrementally with one additional commit since the last revision: > > Minor cleanup 3 very good enhancements ... because of the inherent unreliability of UDP, even for local systems, there is the possibility that the initial send may not reach the responder and vice verse ... but that's for another story. LGTM ------------- Marked as reviewed by msheppar (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16661#pullrequestreview-1732190091 From mbaesken at openjdk.org Thu Nov 16 08:47:48 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Thu, 16 Nov 2023 08:47:48 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values Message-ID: There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. ------------- Commit messages: - JDK-8320168 Changes: https://git.openjdk.org/jdk/pull/16684/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16684&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320168 Stats: 70 lines in 9 files changed: 47 ins; 0 del; 23 mod Patch: https://git.openjdk.org/jdk/pull/16684.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16684/head:pull/16684 PR: https://git.openjdk.org/jdk/pull/16684 From Alan.Bateman at oracle.com Thu Nov 16 08:51:10 2023 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 16 Nov 2023 08:51:10 +0000 Subject: return-value check of setsockopt ? In-Reply-To: References: <637ce73e-8ad0-4617-88f0-012ab2171325@oracle.com> Message-ID: <965c50a9-ba31-4a74-aea6-19ccc029d70e@oracle.com> On 16/11/2023 08:40, Baesken, Matthias wrote: > > Hi Alan, thanks for the comment. > > I created https://bugs.openjdk.org/browse/JDK-8320168? . > > Okay, but please note that putting checks into some of these places will require testing across a very wide range of systems. There are at least some of these that are opt-in where supported on the kernel. So we will have be very cautious with some of this. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From alanb at openjdk.org Thu Nov 16 09:03:47 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 16 Nov 2023 09:03:47 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 08:39:32 GMT, Matthias Baesken wrote: > There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. src/java.base/unix/native/libnet/SdpSupport.c line 100: > 98: if (getsockopt(fd, SOL_SOCKET, SO_REUSEADDR, (char*)&arg, &len) == 0) { > 99: res = setsockopt(s, SOL_SOCKET, SO_REUSEADDR, (char*)&arg, len); > 100: if (res < 0) JNU_ThrowIOExceptionWithLastError(env, "setsockopt SO_REUSEADDR"); OFED deprecated SDP a long time and I think we should just remove this code from the JDK. The copying of the socket options was always a best effort basis. It requires Infiniband config to be able to test these changes so maybe it would be better to just drop it from this change as it can't be tested. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16684#discussion_r1395364985 From alanb at openjdk.org Thu Nov 16 09:07:33 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 16 Nov 2023 09:07:33 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 08:39:32 GMT, Matthias Baesken wrote: > There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. src/java.base/windows/native/libnio/ch/Net.c line 160: > 158: > 159: /* IPV6_V6ONLY is true by default */ > 160: /* attempt to disable IPV6_V6ONLY to ensure dual-socket support; ignore errors */ You can the previous comment at L159. src/java.base/windows/native/libnio/ch/WindowsAsynchronousServerSocketChannelImpl.c line 134: > 132: SOCKET s2 = (SOCKET)jlong_to_ptr(acceptSocket); > 133: > 134: if (setsockopt(s2, SOL_SOCKET, SO_UPDATE_ACCEPT_CONTEXT, (char *)&s1, sizeof(s1)) < 0) { I think this goes back to when the JDK targeted Windows releases where this wasn't supported or there were non-MS LSPs installed. It should be okay to change now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16684#discussion_r1395369394 PR Review Comment: https://git.openjdk.org/jdk/pull/16684#discussion_r1395368258 From mbaesken at openjdk.org Thu Nov 16 13:16:32 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Thu, 16 Nov 2023 13:16:32 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 09:01:02 GMT, Alan Bateman wrote: >> There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. > > src/java.base/unix/native/libnet/SdpSupport.c line 100: > >> 98: if (getsockopt(fd, SOL_SOCKET, SO_REUSEADDR, (char*)&arg, &len) == 0) { >> 99: res = setsockopt(s, SOL_SOCKET, SO_REUSEADDR, (char*)&arg, len); >> 100: if (res < 0) JNU_ThrowIOExceptionWithLastError(env, "setsockopt SO_REUSEADDR"); > > OFED deprecated SDP a long time and I think we should just remove this code from the JDK. The copying of the socket options was always a best effort basis. It requires Infiniband config to be able to test these changes so maybe it would be better to just drop it from this change as it can't be tested. Hi Alan, do you mean "drop this chunk from the change" or "drop the coding that is trying to copy the socket options" ? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16684#discussion_r1395674845 From mbaesken at openjdk.org Thu Nov 16 13:24:45 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Thu, 16 Nov 2023 13:24:45 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v2] In-Reply-To: References: Message-ID: > There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: adjust comment in Net.c ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16684/files - new: https://git.openjdk.org/jdk/pull/16684/files/f5cf86db..687ec6e3 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16684&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16684&range=00-01 Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16684.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16684/head:pull/16684 PR: https://git.openjdk.org/jdk/pull/16684 From mbaesken at openjdk.org Thu Nov 16 13:24:48 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Thu, 16 Nov 2023 13:24:48 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v2] In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 09:04:48 GMT, Alan Bateman wrote: >> Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: >> >> adjust comment in Net.c > > src/java.base/windows/native/libnio/ch/Net.c line 160: > >> 158: >> 159: /* IPV6_V6ONLY is true by default */ >> 160: /* attempt to disable IPV6_V6ONLY to ensure dual-socket support; ignore errors */ > > You can remove the previous comment at L159. I adjusted the comment. > src/java.base/windows/native/libnio/ch/WindowsAsynchronousServerSocketChannelImpl.c line 134: > >> 132: SOCKET s2 = (SOCKET)jlong_to_ptr(acceptSocket); >> 133: >> 134: if (setsockopt(s2, SOL_SOCKET, SO_UPDATE_ACCEPT_CONTEXT, (char *)&s1, sizeof(s1)) < 0) { > > I think this goes back to when the JDK targeted Windows releases where this wasn't supported or there were non-MS LSPs installed. It should be okay to change now. Hi Alan, in the MS resources https://learn.microsoft.com/en-us/windows/win32/api/mswsock/nf-mswsock-acceptex I find some info about 'Windows XP or later' regarding the SO_UPDATE_ACCEPT_CONTEXT option. So maybe this was the Windows release you are referring to. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16684#discussion_r1395687380 PR Review Comment: https://git.openjdk.org/jdk/pull/16684#discussion_r1395683693 From jpai at openjdk.org Thu Nov 16 13:27:37 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Thu, 16 Nov 2023 13:27:37 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out [v2] In-Reply-To: References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> Message-ID: On Tue, 14 Nov 2023 18:01:44 GMT, Daniel Fuchs wrote: >> Please find here a fix for a timeout issue observed in DatagramChannel/Connect.java >> >> The suspicion is that the responder may have received some message from some other source and replied to that, leaving the initiator waiting forever. >> >> The proposed fix makes sure the messages exchanged are unique and identifiable and ensure that both the initiator and responder will discard any message that do not have the expected content. >> >> Additional logging should help with diagnosis if the test fails again. > > Daniel Fuchs has updated the pull request incrementally with one additional commit since the last revision: > > Minor cleanup 3 test/jdk/java/nio/channels/DatagramChannel/Connect.java line 116: > 114: int port = dc.socket().getLocalPort(); > 115: InetAddress loopback = InetAddress.getLoopbackAddress(); > 116: InetSocketAddress otherAddress = new InetSocketAddress(loopback, (port == 3333 ? 3332 : 3333)); Hello Daniel, It was a bit surprising to see a specific port value being used here (before the changes in the PR). So I looked at the history of this test to see if it was intentional and was there for some specific reason. It appears that this was an unintentional leftover from one of the previous refactoring of this test. Using an ephemeral port for this `otherAddress`, like you now do in this PR, looks good to me then. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16661#discussion_r1395690955 From jpai at openjdk.org Thu Nov 16 13:45:36 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Thu, 16 Nov 2023 13:45:36 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out [v2] In-Reply-To: References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> Message-ID: <9Y9ihD4X-k2lAaaSnzBO8UQv49yMQh_OcW-7KHo7A4g=.438a4e8c-de66-4194-85ea-cdde8258ff3a@github.com> On Tue, 14 Nov 2023 18:01:44 GMT, Daniel Fuchs wrote: >> Please find here a fix for a timeout issue observed in DatagramChannel/Connect.java >> >> The suspicion is that the responder may have received some message from some other source and replied to that, leaving the initiator waiting forever. >> >> The proposed fix makes sure the messages exchanged are unique and identifiable and ensure that both the initiator and responder will discard any message that do not have the expected content. >> >> Additional logging should help with diagnosis if the test fails again. > > Daniel Fuchs has updated the pull request incrementally with one additional commit since the last revision: > > Minor cleanup 3 test/jdk/java/nio/channels/DatagramChannel/Connect.java line 160: > 158: if (!RESPONSE.equals(cb.toString())) { > 159: log.println("Initiator received unexpected message: continue waiting"); > 160: continue; `DatagramChannel.read(ByteBuffer)`, through the contract of `ReadableByteChannel.read(ByteBuffer)`, says that the number of bytes added into the passed `ByteBuffer` may not always be the remaining space in the buffer. i.e. `dc.read(bb)` may just read in few bytes at a time. Do you think that's a practical possibility in the context of this test and would that then mean, in this current proposed form, the `RESPONSE.equals(cb.toString())` will never evaluate to `true` and we would end up looping forever? Perhaps we should use the return value of `dc.read(bb)` to determine when to do this response equality check and until then keep accumulating the received bytes? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16661#discussion_r1395712782 From alanb at openjdk.org Thu Nov 16 14:00:32 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 16 Nov 2023 14:00:32 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out [v2] In-Reply-To: References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> Message-ID: On Tue, 14 Nov 2023 18:01:44 GMT, Daniel Fuchs wrote: >> Please find here a fix for a timeout issue observed in DatagramChannel/Connect.java >> >> The suspicion is that the responder may have received some message from some other source and replied to that, leaving the initiator waiting forever. >> >> The proposed fix makes sure the messages exchanged are unique and identifiable and ensure that both the initiator and responder will discard any message that do not have the expected content. >> >> Additional logging should help with diagnosis if the test fails again. > > Daniel Fuchs has updated the pull request incrementally with one additional commit since the last revision: > > Minor cleanup 3 test/jdk/java/nio/channels/DatagramChannel/Connect.java line 46: > 44: > 45: static final PrintStream log = System.err; > 46: static final String NOW = Instant.now().toString(); Would it be possible to rename "log" to "err" so it's a bit clearer at the use-sites where the output is going? Also I think we should rename "NOW" as it's the time that the class is initialized so very confusing at every usage. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16661#discussion_r1395735788 From alanb at openjdk.org Thu Nov 16 14:46:32 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 16 Nov 2023 14:46:32 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v2] In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 13:14:16 GMT, Matthias Baesken wrote: > Hi Alan, do you mean "drop this chunk from the change" or "drop the coding that is trying to copy the socket options" ? Yes, I think drop SdpSupport.c from the PR. We probably should have removed the SDP support a long time ago. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16684#discussion_r1395818177 From msheppar at openjdk.org Thu Nov 16 15:46:32 2023 From: msheppar at openjdk.org (Mark Sheppard) Date: Thu, 16 Nov 2023 15:46:32 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out [v2] In-Reply-To: <9Y9ihD4X-k2lAaaSnzBO8UQv49yMQh_OcW-7KHo7A4g=.438a4e8c-de66-4194-85ea-cdde8258ff3a@github.com> References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> <9Y9ihD4X-k2lAaaSnzBO8UQv49yMQh_OcW-7KHo7A4g=.438a4e8c-de66-4194-85ea-cdde8258ff3a@github.com> Message-ID: On Thu, 16 Nov 2023 13:42:36 GMT, Jaikiran Pai wrote: >> Daniel Fuchs has updated the pull request incrementally with one additional commit since the last revision: >> >> Minor cleanup 3 > > test/jdk/java/nio/channels/DatagramChannel/Connect.java line 160: > >> 158: if (!RESPONSE.equals(cb.toString())) { >> 159: log.println("Initiator received unexpected message: continue waiting"); >> 160: continue; > > `DatagramChannel.read(ByteBuffer)`, through the contract of `ReadableByteChannel.read(ByteBuffer)`, says that the number of bytes added into the passed `ByteBuffer` may not always be the remaining space in the buffer. i.e. `dc.read(bb)` may just read in few bytes at a time. > > Do you think that's a practical possibility in the context of this test and would that then mean, in this current proposed form, the `RESPONSE.equals(cb.toString())` will never evaluate to `true` and we would end up looping forever? > > Perhaps we should use the return value of `dc.read(bb)` to determine when to do this response equality check and until then keep accumulating the received bytes? UDP send and receive is atomic, there are no partial send or receive of a datagram (unlike a stream). Thus the initiator send its message as a complete datagram, then when the responder invokes receive and if the send has been successful, the receive will be the complete datagram sent by the initiator -- no partial datagram. That is unless the datagram is a stray datagram from another source, in which the responder continues to wait for the expected datagram. The reverse is true for the responder, it will receive a complete (atomic) message and echo that back to the initiator ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16661#discussion_r1395931273 From mbaesken at openjdk.org Thu Nov 16 15:52:02 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Thu, 16 Nov 2023 15:52:02 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v3] In-Reply-To: References: Message-ID: <1qQ4txOhk5OEHr3fh_PmAGA-Sk-RSXD-U4A3oupjv_I=.8feb9f23-488d-4b8d-b370-55ec2e314f3e@github.com> > There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: do not change old legacy src/java.base/unix/native/libnet/SdpSupport.c ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16684/files - new: https://git.openjdk.org/jdk/pull/16684/files/687ec6e3..ecd11c71 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16684&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16684&range=01-02 Stats: 17 lines in 1 file changed: 0 ins; 8 del; 9 mod Patch: https://git.openjdk.org/jdk/pull/16684.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16684/head:pull/16684 PR: https://git.openjdk.org/jdk/pull/16684 From mbaesken at openjdk.org Thu Nov 16 15:52:04 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Thu, 16 Nov 2023 15:52:04 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v2] In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 13:24:45 GMT, Matthias Baesken wrote: >> There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > adjust comment in Net.c > Yes, I think drop SdpSupport.c from the PR. We probably should have removed the SDP support a long time ago. Hi Alan, I dropped SdpSupport.c from the PR. Removing SDP support from the JDK is probably something for another PR (or even JEP ;-) ) ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16684#issuecomment-1814727807 From dfuchs at openjdk.org Thu Nov 16 16:41:47 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 16 Nov 2023 16:41:47 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out [v3] In-Reply-To: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> Message-ID: > Please find here a fix for a timeout issue observed in DatagramChannel/Connect.java > > The suspicion is that the responder may have received some message from some other source and replied to that, leaving the initiator waiting forever. > > The proposed fix makes sure the messages exchanged are unique and identifiable and ensure that both the initiator and responder will discard any message that do not have the expected content. > > Additional logging should help with diagnosis if the test fails again. Daniel Fuchs has updated the pull request incrementally with one additional commit since the last revision: Review feedback ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16661/files - new: https://git.openjdk.org/jdk/pull/16661/files/958d5b76..ba2ee580 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16661&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16661&range=01-02 Stats: 22 lines in 1 file changed: 0 ins; 0 del; 22 mod Patch: https://git.openjdk.org/jdk/pull/16661.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16661/head:pull/16661 PR: https://git.openjdk.org/jdk/pull/16661 From dfuchs at openjdk.org Thu Nov 16 16:41:51 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 16 Nov 2023 16:41:51 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out [v2] In-Reply-To: References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> Message-ID: On Thu, 16 Nov 2023 13:57:56 GMT, Alan Bateman wrote: >> Daniel Fuchs has updated the pull request incrementally with one additional commit since the last revision: >> >> Minor cleanup 3 > > test/jdk/java/nio/channels/DatagramChannel/Connect.java line 46: > >> 44: >> 45: static final PrintStream log = System.err; >> 46: static final String NOW = Instant.now().toString(); > > Would it be possible to rename "log" to "err" so it's a bit clearer at the use-sites where the output is going? > > Also I think we should rename "NOW" as it's the time that the class is initialized so very confusing at every usage. done ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16661#discussion_r1396019501 From dfuchs at openjdk.org Thu Nov 16 16:41:52 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 16 Nov 2023 16:41:52 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out [v2] In-Reply-To: References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> <9Y9ihD4X-k2lAaaSnzBO8UQv49yMQh_OcW-7KHo7A4g=.438a4e8c-de66-4194-85ea-cdde8258ff3a@github.com> Message-ID: On Thu, 16 Nov 2023 15:44:11 GMT, Mark Sheppard wrote: >> test/jdk/java/nio/channels/DatagramChannel/Connect.java line 160: >> >>> 158: if (!RESPONSE.equals(cb.toString())) { >>> 159: log.println("Initiator received unexpected message: continue waiting"); >>> 160: continue; >> >> `DatagramChannel.read(ByteBuffer)`, through the contract of `ReadableByteChannel.read(ByteBuffer)`, says that the number of bytes added into the passed `ByteBuffer` may not always be the remaining space in the buffer. i.e. `dc.read(bb)` may just read in few bytes at a time. >> >> Do you think that's a practical possibility in the context of this test and would that then mean, in this current proposed form, the `RESPONSE.equals(cb.toString())` will never evaluate to `true` and we would end up looping forever? >> >> Perhaps we should use the return value of `dc.read(bb)` to determine when to do this response equality check and until then keep accumulating the received bytes? > > UDP send and receive is atomic, there are no partial send or receive of a datagram (unlike a stream). Thus the initiator send its message as a complete datagram, then when the responder invokes receive and if the send has been successful, the receive will be the complete datagram sent by the initiator -- no partial datagram. That is unless the datagram is a stray datagram from another source, in which the responder continues to wait for the expected datagram. The reverse is true for the responder, it will receive a complete (atomic) message and echo that back to the initiator Yes - for DatagramChannel the whole UDP payload is copied in the ByteBuffer. It may get truncated if the ByteBuffer doesn't have enough remaining bytes, but I made sure that the ByteBuffers used in the test are big enough to contain the UDP payload of the datagrams sent by the test. Bytes that don't fit in the ByteBuffer are discarded. Two calls to read would get you two payloads from two different datagrams. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16661#discussion_r1396016255 From dfuchs at openjdk.org Thu Nov 16 16:49:49 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 16 Nov 2023 16:49:49 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out [v3] In-Reply-To: References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> Message-ID: <0ILNlbNDL3MU-sfWHzKGOmZGa4dqN1IFZ-gRmdgK_fo=.84784513-8065-445e-b00c-639c78877ea2@github.com> On Thu, 16 Nov 2023 16:41:47 GMT, Daniel Fuchs wrote: >> Please find here a fix for a timeout issue observed in DatagramChannel/Connect.java >> >> The suspicion is that the responder may have received some message from some other source and replied to that, leaving the initiator waiting forever. >> >> The proposed fix makes sure the messages exchanged are unique and identifiable and ensure that both the initiator and responder will discard any message that do not have the expected content. >> >> Additional logging should help with diagnosis if the test fails again. > > Daniel Fuchs has updated the pull request incrementally with one additional commit since the last revision: > > Review feedback Pushed review feedback. Also refactored Responder to put try { } finally { } around the while loop, instead of inside. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16661#issuecomment-1814832078 From dfuchs at openjdk.org Thu Nov 16 16:49:46 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 16 Nov 2023 16:49:46 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out [v4] In-Reply-To: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> Message-ID: > Please find here a fix for a timeout issue observed in DatagramChannel/Connect.java > > The suspicion is that the responder may have received some message from some other source and replied to that, leaving the initiator waiting forever. > > The proposed fix makes sure the messages exchanged are unique and identifiable and ensure that both the initiator and responder will discard any message that do not have the expected content. > > Additional logging should help with diagnosis if the test fails again. Daniel Fuchs has updated the pull request incrementally with two additional commits since the last revision: - Responder: try { } finally { } should be around while () { } - Misplaced trace in finally block ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16661/files - new: https://git.openjdk.org/jdk/pull/16661/files/ba2ee580..6667720b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16661&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16661&range=02-03 Stats: 12 lines in 1 file changed: 5 ins; 5 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16661.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16661/head:pull/16661 PR: https://git.openjdk.org/jdk/pull/16661 From jvernee at openjdk.org Thu Nov 16 19:23:44 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Thu, 16 Nov 2023 19:23:44 GMT Subject: RFR: 8318966: Some methods make promises about Java array element alignment that are too strong Message-ID: See the JBS issue for an extended problem description. This patch changes the specification and implementation of `MethodHandles::byteArrayViewVarHandle`, `MethodHandles::byteBufferViewVarHandle`, `ByteBuffer::alignedSlice`, and `ByteBuffer::alignmentOffset` to weaken the guarantees they make about the alignment of Java array elements, in order to bring them in line with the guarantees made by an arbitrary JVM implementation (which makes no guarantees about array element alignment beyond them being aligned to their natural alignment). - `MethodHandles::byteArrayViewVarHandle`: we can not guarantee any alignment for the accesses. Which means we can only reliably support plain get and set access modes. The javadoc text explaining which other access modes are supported, or how to compute aligned offsets into the array is dropped, as it is not guaranteed to be correct on all JVM implementations. The implementation of the returned VarHandle is changed to throw an `UnsupportedOperationException` for the unsupported access modes, as mandated by the spec of `VarHandle` [1]. - `MethodHandles::byteBufferViewVarHandle`: the implementation/spec is incorrect when accessing a heap buffer (wrapping a byte[]), for the same reasons as `byteArrayViewVarHandle`. The spec is changed to specify that when accessing a _heap buffer_, only plain get and set access modes are supported. The implementation of the returned var handle is changed to throw an `IllegalStateException` when an access is attempted on a heap buffer using an access mode other than plain get or set. Note that we don't throw an outright `UnsupportedOperationException` for this case, since whether the access modes are supported depends on the byte buffer instance being used. - `ByteBuffer::alignedSlice` and `ByteBuffer::alignmentOffset`: The former method depends directly on the latter for all its alignment computations. We change the implementation of the latter method to throw an `UnsupportedOperationException` for all unit sizes greater than 1, when the buffer is non-direct. This change is largely covered by the existing specification: * @throws UnsupportedOperationException * If the native platform does not guarantee stable alignment offset * values for the given unit size when managing the memory regions * of buffers of the same kind as this buffer (direct or * non-direct). For example, if garbage collection would result * in the moving of a memory region covered by a non-direct buffer * from one location to another and both locations have different * alignment characteristics. However, the `@implNote` mentions that an `UnsupportedOperationException` will be thrown for unit sizes greater than 8. This is updated to say unit sizes greater than 1. Note that the testing code for these APIs is tightly coupled, so it is practically convenient to address all these issues together. Testing: local `java/lang/invoke/VarHandles`, more TODO [1]: https://github.com/openjdk/jdk/blob/ffa35d8cf181cfbcb54497e997dbd18a9b62b97e/src/java.base/share/classes/java/lang/invoke/VarHandle.java#L189-L191 ------------- Commit messages: - Add api note pointing to alternatives for client that require non-plain access - simplify spec for alignmentOffset and alignedSlice - Merge note about misaligned access in byteBufferViewVarHandle - updated alignedSlice implNote as well - updated alignedOffset implNote - Use ISE for bbvvh instead of UOE - restore some tests for direct buffers - fix BAVV and BBVV impl and tests - regen test files - fix BasicByte test Changes: https://git.openjdk.org/jdk/pull/16681/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16681&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8318966 Stats: 5133 lines in 19 files changed: 612 ins; 3440 del; 1081 mod Patch: https://git.openjdk.org/jdk/pull/16681.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16681/head:pull/16681 PR: https://git.openjdk.org/jdk/pull/16681 From psandoz at openjdk.org Thu Nov 16 19:23:48 2023 From: psandoz at openjdk.org (Paul Sandoz) Date: Thu, 16 Nov 2023 19:23:48 GMT Subject: RFR: 8318966: Some methods make promises about Java array element alignment that are too strong In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 22:46:03 GMT, Jorn Vernee wrote: > See the JBS issue for an extended problem description. > > This patch changes the specification and implementation of `MethodHandles::byteArrayViewVarHandle`, `MethodHandles::byteBufferViewVarHandle`, `ByteBuffer::alignedSlice`, and `ByteBuffer::alignmentOffset` to weaken the guarantees they make about the alignment of Java array elements, in order to bring them in line with the guarantees made by an arbitrary JVM implementation (which makes no guarantees about array element alignment beyond them being aligned to their natural alignment). > > - `MethodHandles::byteArrayViewVarHandle`: we can not guarantee any alignment for the accesses. Which means we can only reliably support plain get and set access modes. The javadoc text explaining which other access modes are supported, or how to compute aligned offsets into the array is dropped, as it is not guaranteed to be correct on all JVM implementations. The implementation of the returned VarHandle is changed to throw an `UnsupportedOperationException` for the unsupported access modes, as mandated by the spec of `VarHandle` [1]. > > - `MethodHandles::byteBufferViewVarHandle`: the implementation/spec is incorrect when accessing a heap buffer (wrapping a byte[]), for the same reasons as `byteArrayViewVarHandle`. The spec is changed to specify that when accessing a _heap buffer_, only plain get and set access modes are supported. The implementation of the returned var handle is changed to throw an `IllegalStateException` when an access is attempted on a heap buffer using an access mode other than plain get or set. Note that we don't throw an outright `UnsupportedOperationException` for this case, since whether the access modes are supported depends on the byte buffer instance being used. > > - `ByteBuffer::alignedSlice` and `ByteBuffer::alignmentOffset`: The former method depends directly on the latter for all its alignment computations. We change the implementation of the latter method to throw an `UnsupportedOperationException` for all unit sizes greater than 1, when the buffer is non-direct. This change is largely covered by the existing specification: > > > * @throws UnsupportedOperationException > * If the native platform does not guarantee stable alignment offset > * values for the given unit size when managing the memory regions > * of buffers of the same kind as this buffer (direct or > * non-direct). For example, if garbage collection would result > * in the mo... src/java.base/share/classes/java/lang/invoke/MethodHandles.java line 4518: > 4516: * Only plain {@linkplain VarHandle.AccessMode#GET get} and {@linkplain VarHandle.AccessMode#SET set} > 4517: * access modes are supported by the returned var handle. For all other access modes, an > 4518: * {@link UnsupportedOperationException} will be thrown. I recommend adding an api note explaining that native memory segments, direct byte buffers, or heap memory segments backed by long[] should be used if support for other access modes are required. src/java.base/share/classes/java/lang/invoke/MethodHandles.java line 4610: > 4608: * {@link Double#doubleToRawLongBits}, respectively). > 4609: *

> 4610: * Access to heap byte buffers is always unaligned. I recommend merging that sentence into the paragraph on heap byte buffers e.g.: > For direct buffers, access of the bytes at an index is always misaligned. As a result only the plain... src/java.base/share/classes/java/nio/X-Buffer.java.template line 2218: > 2216: * @implNote > 2217: * This implementation throws {@code UnsupportedOperationException} for > 2218: * non-direct buffers when the given unit size is greater then {@code 1}. This is no longer an implementation note, its now part of the specified API. So i think we can simplify the text of the `@throws UOE ...` to just say: @throws UOE if the buffer is non-direct and the unit size > 1 Same for the other method. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16681#discussion_r1396097577 PR Review Comment: https://git.openjdk.org/jdk/pull/16681#discussion_r1396101675 PR Review Comment: https://git.openjdk.org/jdk/pull/16681#discussion_r1396109655 From jvernee at openjdk.org Thu Nov 16 19:23:48 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Thu, 16 Nov 2023 19:23:48 GMT Subject: RFR: 8318966: Some methods make promises about Java array element alignment that are too strong In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 17:31:46 GMT, Paul Sandoz wrote: >> See the JBS issue for an extended problem description. >> >> This patch changes the specification and implementation of `MethodHandles::byteArrayViewVarHandle`, `MethodHandles::byteBufferViewVarHandle`, `ByteBuffer::alignedSlice`, and `ByteBuffer::alignmentOffset` to weaken the guarantees they make about the alignment of Java array elements, in order to bring them in line with the guarantees made by an arbitrary JVM implementation (which makes no guarantees about array element alignment beyond them being aligned to their natural alignment). >> >> - `MethodHandles::byteArrayViewVarHandle`: we can not guarantee any alignment for the accesses. Which means we can only reliably support plain get and set access modes. The javadoc text explaining which other access modes are supported, or how to compute aligned offsets into the array is dropped, as it is not guaranteed to be correct on all JVM implementations. The implementation of the returned VarHandle is changed to throw an `UnsupportedOperationException` for the unsupported access modes, as mandated by the spec of `VarHandle` [1]. >> >> - `MethodHandles::byteBufferViewVarHandle`: the implementation/spec is incorrect when accessing a heap buffer (wrapping a byte[]), for the same reasons as `byteArrayViewVarHandle`. The spec is changed to specify that when accessing a _heap buffer_, only plain get and set access modes are supported. The implementation of the returned var handle is changed to throw an `IllegalStateException` when an access is attempted on a heap buffer using an access mode other than plain get or set. Note that we don't throw an outright `UnsupportedOperationException` for this case, since whether the access modes are supported depends on the byte buffer instance being used. >> >> - `ByteBuffer::alignedSlice` and `ByteBuffer::alignmentOffset`: The former method depends directly on the latter for all its alignment computations. We change the implementation of the latter method to throw an `UnsupportedOperationException` for all unit sizes greater than 1, when the buffer is non-direct. This change is largely covered by the existing specification: >> >> >> * @throws UnsupportedOperationException >> * If the native platform does not guarantee stable alignment offset >> * values for the given unit size when managing the memory regions >> * of buffers of the same kind as this buffer (direct or >> * non-direct). For example, if garbage collection would... > > src/java.base/share/classes/java/lang/invoke/MethodHandles.java line 4518: > >> 4516: * Only plain {@linkplain VarHandle.AccessMode#GET get} and {@linkplain VarHandle.AccessMode#SET set} >> 4517: * access modes are supported by the returned var handle. For all other access modes, an >> 4518: * {@link UnsupportedOperationException} will be thrown. > > I recommend adding an api note explaining that native memory segments, direct byte buffers, or heap memory segments backed by long[] should be used if support for other access modes are required. Good idea. Thanks > src/java.base/share/classes/java/nio/X-Buffer.java.template line 2218: > >> 2216: * @implNote >> 2217: * This implementation throws {@code UnsupportedOperationException} for >> 2218: * non-direct buffers when the given unit size is greater then {@code 1}. > > This is no longer an implementation note, its now part of the specified API. So i think we can simplify the text of the `@throws UOE ...` to just say: > > @throws UOE if the buffer is non-direct and the unit size > 1 > > Same for the other method. Right. Good idea ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16681#discussion_r1396147281 PR Review Comment: https://git.openjdk.org/jdk/pull/16681#discussion_r1396147495 From mcimadamore at openjdk.org Thu Nov 16 19:23:49 2023 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 16 Nov 2023 19:23:49 GMT Subject: RFR: 8318966: Some methods make promises about Java array element alignment that are too strong In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 17:34:40 GMT, Paul Sandoz wrote: >> See the JBS issue for an extended problem description. >> >> This patch changes the specification and implementation of `MethodHandles::byteArrayViewVarHandle`, `MethodHandles::byteBufferViewVarHandle`, `ByteBuffer::alignedSlice`, and `ByteBuffer::alignmentOffset` to weaken the guarantees they make about the alignment of Java array elements, in order to bring them in line with the guarantees made by an arbitrary JVM implementation (which makes no guarantees about array element alignment beyond them being aligned to their natural alignment). >> >> - `MethodHandles::byteArrayViewVarHandle`: we can not guarantee any alignment for the accesses. Which means we can only reliably support plain get and set access modes. The javadoc text explaining which other access modes are supported, or how to compute aligned offsets into the array is dropped, as it is not guaranteed to be correct on all JVM implementations. The implementation of the returned VarHandle is changed to throw an `UnsupportedOperationException` for the unsupported access modes, as mandated by the spec of `VarHandle` [1]. >> >> - `MethodHandles::byteBufferViewVarHandle`: the implementation/spec is incorrect when accessing a heap buffer (wrapping a byte[]), for the same reasons as `byteArrayViewVarHandle`. The spec is changed to specify that when accessing a _heap buffer_, only plain get and set access modes are supported. The implementation of the returned var handle is changed to throw an `IllegalStateException` when an access is attempted on a heap buffer using an access mode other than plain get or set. Note that we don't throw an outright `UnsupportedOperationException` for this case, since whether the access modes are supported depends on the byte buffer instance being used. >> >> - `ByteBuffer::alignedSlice` and `ByteBuffer::alignmentOffset`: The former method depends directly on the latter for all its alignment computations. We change the implementation of the latter method to throw an `UnsupportedOperationException` for all unit sizes greater than 1, when the buffer is non-direct. This change is largely covered by the existing specification: >> >> >> * @throws UnsupportedOperationException >> * If the native platform does not guarantee stable alignment offset >> * values for the given unit size when managing the memory regions >> * of buffers of the same kind as this buffer (direct or >> * non-direct). For example, if garbage collection would... > > src/java.base/share/classes/java/lang/invoke/MethodHandles.java line 4610: > >> 4608: * {@link Double#doubleToRawLongBits}, respectively). >> 4609: *

>> 4610: * Access to heap byte buffers is always unaligned. > > I recommend merging that sentence into the paragraph on heap byte buffers e.g.: >> For direct buffers, access of the bytes at an index is always misaligned. As a result only the plain... But... surely if I'm just accessing bytes the access cannot possibly be mis-aligned? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16681#discussion_r1396191981 From jvernee at openjdk.org Thu Nov 16 19:23:50 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Thu, 16 Nov 2023 19:23:50 GMT Subject: RFR: 8318966: Some methods make promises about Java array element alignment that are too strong In-Reply-To: References: Message-ID: <4HJwUwQZOI4JTxrF76V8ZXqrRQs1756uM_vw12YmkFI=.543d94ea-dc74-4c1d-9ca8-6314eb9b7831@github.com> On Thu, 16 Nov 2023 18:52:03 GMT, Maurizio Cimadamore wrote: >> src/java.base/share/classes/java/lang/invoke/MethodHandles.java line 4610: >> >>> 4608: * {@link Double#doubleToRawLongBits}, respectively). >>> 4609: *

>>> 4610: * Access to heap byte buffers is always unaligned. >> >> I recommend merging that sentence into the paragraph on heap byte buffers e.g.: >>> For direct buffers, access of the bytes at an index is always misaligned. As a result only the plain... > > But... surely if I'm just accessing bytes the access cannot possibly be mis-aligned? Right. I think there is a difference between 'unaligned' and 'misaligned'. Unaligned means there is no effort made to align the access (so it _may_ be misaligned). Misaligned means _definitely not_ aligned. At least, that is my interpretation. This is what I wrote in the latest version: * For heap byte buffers, access is always unaligned. As a result, only the plain * {@linkplain VarHandle.AccessMode#GET get} * and {@linkplain VarHandle.AccessMode#SET set} access modes are supported by the * returned var handle. For all other access modes, an {@link IllegalStateException} * will be thrown. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16681#discussion_r1396215821 From mcimadamore at openjdk.org Thu Nov 16 19:23:51 2023 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 16 Nov 2023 19:23:51 GMT Subject: RFR: 8318966: Some methods make promises about Java array element alignment that are too strong In-Reply-To: <4HJwUwQZOI4JTxrF76V8ZXqrRQs1756uM_vw12YmkFI=.543d94ea-dc74-4c1d-9ca8-6314eb9b7831@github.com> References: <4HJwUwQZOI4JTxrF76V8ZXqrRQs1756uM_vw12YmkFI=.543d94ea-dc74-4c1d-9ca8-6314eb9b7831@github.com> Message-ID: On Thu, 16 Nov 2023 19:15:25 GMT, Jorn Vernee wrote: >> But... surely if I'm just accessing bytes the access cannot possibly be mis-aligned? > > Right. I think there is a difference between 'unaligned' and 'misaligned'. Unaligned means there is no effort made to align the access (so it _may_ be misaligned). Misaligned means _definitely not_ aligned. At least, that is my interpretation. > > This is what I wrote in the latest version: > > > * For heap byte buffers, access is always unaligned. As a result, only the plain > * {@linkplain VarHandle.AccessMode#GET get} > * and {@linkplain VarHandle.AccessMode#SET set} access modes are supported by the > * returned var handle. For all other access modes, an {@link IllegalStateException} > * will be thrown. Uhm, I think my assumption was wrong - the `byte`/`boolean` carrier is not supported by this method, so there's no case where heap access would be guaranteed to be aligned. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16681#discussion_r1396221179 From jvernee at openjdk.org Thu Nov 16 19:23:53 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Thu, 16 Nov 2023 19:23:53 GMT Subject: RFR: 8318966: Some methods make promises about Java array element alignment that are too strong In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 22:46:03 GMT, Jorn Vernee wrote: > See the JBS issue for an extended problem description. > > This patch changes the specification and implementation of `MethodHandles::byteArrayViewVarHandle`, `MethodHandles::byteBufferViewVarHandle`, `ByteBuffer::alignedSlice`, and `ByteBuffer::alignmentOffset` to weaken the guarantees they make about the alignment of Java array elements, in order to bring them in line with the guarantees made by an arbitrary JVM implementation (which makes no guarantees about array element alignment beyond them being aligned to their natural alignment). > > - `MethodHandles::byteArrayViewVarHandle`: we can not guarantee any alignment for the accesses. Which means we can only reliably support plain get and set access modes. The javadoc text explaining which other access modes are supported, or how to compute aligned offsets into the array is dropped, as it is not guaranteed to be correct on all JVM implementations. The implementation of the returned VarHandle is changed to throw an `UnsupportedOperationException` for the unsupported access modes, as mandated by the spec of `VarHandle` [1]. > > - `MethodHandles::byteBufferViewVarHandle`: the implementation/spec is incorrect when accessing a heap buffer (wrapping a byte[]), for the same reasons as `byteArrayViewVarHandle`. The spec is changed to specify that when accessing a _heap buffer_, only plain get and set access modes are supported. The implementation of the returned var handle is changed to throw an `IllegalStateException` when an access is attempted on a heap buffer using an access mode other than plain get or set. Note that we don't throw an outright `UnsupportedOperationException` for this case, since whether the access modes are supported depends on the byte buffer instance being used. > > - `ByteBuffer::alignedSlice` and `ByteBuffer::alignmentOffset`: The former method depends directly on the latter for all its alignment computations. We change the implementation of the latter method to throw an `UnsupportedOperationException` for all unit sizes greater than 1, when the buffer is non-direct. This change is largely covered by the existing specification: > > > * @throws UnsupportedOperationException > * If the native platform does not guarantee stable alignment offset > * values for the given unit size when managing the memory regions > * of buffers of the same kind as this buffer (direct or > * non-direct). For example, if garbage collection would result > * in the mo... src/java.base/share/classes/java/lang/invoke/X-VarHandleByteArrayView.java.template line 739: > 737: ByteBufferHandle handle = (ByteBufferHandle)ob; > 738: ByteBuffer bb = (ByteBuffer) Objects.requireNonNull(obb); > 739: #if[Object] Note that I removed this 'if Object' block (and the one below), as the public API does not support object access any way: https://github.com/openjdk/jdk/blob/9727f4bdddc071e6f59806087339f345405ab004/src/java.base/share/classes/java/lang/invoke/MethodHandles.java#L4591-L4593 test/jdk/java/nio/Buffer/BasicChar.java line 530: > 528: > 529: > 530: These white space changes come from re-generating the test files using the script included in the repo. I've kept the changes so that the next contributor won't have to remove them again. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16681#discussion_r1396208790 PR Review Comment: https://git.openjdk.org/jdk/pull/16681#discussion_r1396210579 From alanb at openjdk.org Thu Nov 16 19:23:55 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 16 Nov 2023 19:23:55 GMT Subject: RFR: 8318966: Some methods make promises about Java array element alignment that are too strong In-Reply-To: References: Message-ID: <3RnzrWDM5vzcZDh8KOjf5PnOlj6xa-cbbXEoXPPbYzA=.6d34ae5c-34f6-4cce-a210-5f41caddeeb4@github.com> On Wed, 15 Nov 2023 22:46:03 GMT, Jorn Vernee wrote: > See the JBS issue for an extended problem description. > > This patch changes the specification and implementation of `MethodHandles::byteArrayViewVarHandle`, `MethodHandles::byteBufferViewVarHandle`, `ByteBuffer::alignedSlice`, and `ByteBuffer::alignmentOffset` to weaken the guarantees they make about the alignment of Java array elements, in order to bring them in line with the guarantees made by an arbitrary JVM implementation (which makes no guarantees about array element alignment beyond them being aligned to their natural alignment). > > - `MethodHandles::byteArrayViewVarHandle`: we can not guarantee any alignment for the accesses. Which means we can only reliably support plain get and set access modes. The javadoc text explaining which other access modes are supported, or how to compute aligned offsets into the array is dropped, as it is not guaranteed to be correct on all JVM implementations. The implementation of the returned VarHandle is changed to throw an `UnsupportedOperationException` for the unsupported access modes, as mandated by the spec of `VarHandle` [1]. > > - `MethodHandles::byteBufferViewVarHandle`: the implementation/spec is incorrect when accessing a heap buffer (wrapping a byte[]), for the same reasons as `byteArrayViewVarHandle`. The spec is changed to specify that when accessing a _heap buffer_, only plain get and set access modes are supported. The implementation of the returned var handle is changed to throw an `IllegalStateException` when an access is attempted on a heap buffer using an access mode other than plain get or set. Note that we don't throw an outright `UnsupportedOperationException` for this case, since whether the access modes are supported depends on the byte buffer instance being used. > > - `ByteBuffer::alignedSlice` and `ByteBuffer::alignmentOffset`: The former method depends directly on the latter for all its alignment computations. We change the implementation of the latter method to throw an `UnsupportedOperationException` for all unit sizes greater than 1, when the buffer is non-direct. This change is largely covered by the existing specification: > > > * @throws UnsupportedOperationException > * If the native platform does not guarantee stable alignment offset > * values for the given unit size when managing the memory regions > * of buffers of the same kind as this buffer (direct or > * non-direct). For example, if garbage collection would result > * in the mo... src/java.base/share/classes/java/nio/X-Buffer.java.template line 2250: > 2248: if (unitSize < 1 || (unitSize & (unitSize - 1)) != 0) > 2249: throw new IllegalArgumentException("Unit size not a power of two: " + unitSize); > 2250: if (unitSize > 1 && !isDirect()) This will require an update to the implNote as it documents 8. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16681#discussion_r1395205617 From jvernee at openjdk.org Thu Nov 16 19:23:55 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Thu, 16 Nov 2023 19:23:55 GMT Subject: RFR: 8318966: Some methods make promises about Java array element alignment that are too strong In-Reply-To: <3RnzrWDM5vzcZDh8KOjf5PnOlj6xa-cbbXEoXPPbYzA=.6d34ae5c-34f6-4cce-a210-5f41caddeeb4@github.com> References: <3RnzrWDM5vzcZDh8KOjf5PnOlj6xa-cbbXEoXPPbYzA=.6d34ae5c-34f6-4cce-a210-5f41caddeeb4@github.com> Message-ID: On Thu, 16 Nov 2023 06:29:20 GMT, Alan Bateman wrote: >> See the JBS issue for an extended problem description. >> >> This patch changes the specification and implementation of `MethodHandles::byteArrayViewVarHandle`, `MethodHandles::byteBufferViewVarHandle`, `ByteBuffer::alignedSlice`, and `ByteBuffer::alignmentOffset` to weaken the guarantees they make about the alignment of Java array elements, in order to bring them in line with the guarantees made by an arbitrary JVM implementation (which makes no guarantees about array element alignment beyond them being aligned to their natural alignment). >> >> - `MethodHandles::byteArrayViewVarHandle`: we can not guarantee any alignment for the accesses. Which means we can only reliably support plain get and set access modes. The javadoc text explaining which other access modes are supported, or how to compute aligned offsets into the array is dropped, as it is not guaranteed to be correct on all JVM implementations. The implementation of the returned VarHandle is changed to throw an `UnsupportedOperationException` for the unsupported access modes, as mandated by the spec of `VarHandle` [1]. >> >> - `MethodHandles::byteBufferViewVarHandle`: the implementation/spec is incorrect when accessing a heap buffer (wrapping a byte[]), for the same reasons as `byteArrayViewVarHandle`. The spec is changed to specify that when accessing a _heap buffer_, only plain get and set access modes are supported. The implementation of the returned var handle is changed to throw an `IllegalStateException` when an access is attempted on a heap buffer using an access mode other than plain get or set. Note that we don't throw an outright `UnsupportedOperationException` for this case, since whether the access modes are supported depends on the byte buffer instance being used. >> >> - `ByteBuffer::alignedSlice` and `ByteBuffer::alignmentOffset`: The former method depends directly on the latter for all its alignment computations. We change the implementation of the latter method to throw an `UnsupportedOperationException` for all unit sizes greater than 1, when the buffer is non-direct. This change is largely covered by the existing specification: >> >> >> * @throws UnsupportedOperationException >> * If the native platform does not guarantee stable alignment offset >> * values for the given unit size when managing the memory regions >> * of buffers of the same kind as this buffer (direct or >> * non-direct). For example, if garbage collection would... > > src/java.base/share/classes/java/nio/X-Buffer.java.template line 2250: > >> 2248: if (unitSize < 1 || (unitSize & (unitSize - 1)) != 0) >> 2249: throw new IllegalArgumentException("Unit size not a power of two: " + unitSize); >> 2250: if (unitSize > 1 && !isDirect()) > > This will require an update to the implNote as it documents 8. Thanks, good catch ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16681#discussion_r1395682013 From duke at openjdk.org Thu Nov 16 20:54:30 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Thu, 16 Nov 2023 20:54:30 GMT Subject: RFR: 8316141: Improve CEN header validation checking In-Reply-To: References: Message-ID: On Thu, 9 Nov 2023 17:22:39 GMT, Lance Andersen wrote: > Regarding you comment about checking whether or not to check if the combined length of the CEN header + name length + comment length + extra length > 65K bytes, I chose to add this given the strong wording given this is a really old spec. That being said, I do not object to removing the validation if that is the overall preference. I can't claim to have a particularly strong opinion on this, the following is mostly me thinking aloud: - Given Hyrum's Law, it is conceivable that someone is currently using the extra or comment fields to attach up to 65535+65535 bytes of metadata for entires. The proposed validation will break such schemes. Does Oracle have a ZIP file corpus which could be used to identify files currently exceeding the combined length clause, just to get a sense of how rare or common this is? - The actual benefits of adding this validation after all these years is not quite clear to me. I don't see how this improves security, robustness, compatibility, maintainability or other ilities (apart from strictly-following-the-spec-ility :-) - I created a ZIP file with an entry with an extra field of the maximal expressable length of 0xFFFF. Info-ZIP's `unzip` command on MacOS did not output any warning or error when processing this file. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16570#issuecomment-1815293978 From mbaesken at openjdk.org Fri Nov 17 08:09:31 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Fri, 17 Nov 2023 08:09:31 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v3] In-Reply-To: <1qQ4txOhk5OEHr3fh_PmAGA-Sk-RSXD-U4A3oupjv_I=.8feb9f23-488d-4b8d-b370-55ec2e314f3e@github.com> References: <1qQ4txOhk5OEHr3fh_PmAGA-Sk-RSXD-U4A3oupjv_I=.8feb9f23-488d-4b8d-b370-55ec2e314f3e@github.com> Message-ID: On Thu, 16 Nov 2023 15:52:02 GMT, Matthias Baesken wrote: >> There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > do not change old legacy src/java.base/unix/native/libnet/SdpSupport.c Hi Alan, do you want an additional change to src/java.base/windows/native/libnio/ch/WindowsAsynchronousServerSocketChannelImpl.c ? Was not really sure about this after your comment about this chunk . ------------- PR Comment: https://git.openjdk.org/jdk/pull/16684#issuecomment-1815907466 From alanb at openjdk.org Fri Nov 17 09:55:31 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 17 Nov 2023 09:55:31 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v3] In-Reply-To: References: <1qQ4txOhk5OEHr3fh_PmAGA-Sk-RSXD-U4A3oupjv_I=.8feb9f23-488d-4b8d-b370-55ec2e314f3e@github.com> Message-ID: On Fri, 17 Nov 2023 08:07:02 GMT, Matthias Baesken wrote: > Hi Alan, do you want an additional change to src/java.base/windows/native/libnio/ch/WindowsAsynchronousServerSocketChannelImpl.c ? Was not really sure about this after your comment about this chunk . I think it's okay. The only concern now will be third party LSPs but it would be months or years before such issues come out of the woodwork, so I think what you have in the WindowsAsynchronous* files is okay, just check for SOCKET_ERROR instead of < 0. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16684#issuecomment-1816055621 From jpai at openjdk.org Fri Nov 17 10:32:34 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 17 Nov 2023 10:32:34 GMT Subject: RFR: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out [v4] In-Reply-To: References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> Message-ID: On Thu, 16 Nov 2023 16:49:46 GMT, Daniel Fuchs wrote: >> Please find here a fix for a timeout issue observed in DatagramChannel/Connect.java >> >> The suspicion is that the responder may have received some message from some other source and replied to that, leaving the initiator waiting forever. >> >> The proposed fix makes sure the messages exchanged are unique and identifiable and ensure that both the initiator and responder will discard any message that do not have the expected content. >> >> Additional logging should help with diagnosis if the test fails again. > > Daniel Fuchs has updated the pull request incrementally with two additional commits since the last revision: > > - Responder: try { } finally { } should be around while () { } > - Misplaced trace in finally block Thank you Mark and Daniel for those inputs. The changes look good to me. ------------- Marked as reviewed by jpai (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16661#pullrequestreview-1736582958 From lucy at openjdk.org Fri Nov 17 11:33:30 2023 From: lucy at openjdk.org (Lutz Schmidt) Date: Fri, 17 Nov 2023 11:33:30 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v3] In-Reply-To: <1qQ4txOhk5OEHr3fh_PmAGA-Sk-RSXD-U4A3oupjv_I=.8feb9f23-488d-4b8d-b370-55ec2e314f3e@github.com> References: <1qQ4txOhk5OEHr3fh_PmAGA-Sk-RSXD-U4A3oupjv_I=.8feb9f23-488d-4b8d-b370-55ec2e314f3e@github.com> Message-ID: On Thu, 16 Nov 2023 15:52:02 GMT, Matthias Baesken wrote: >> There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > do not change old legacy src/java.base/unix/native/libnet/SdpSupport.c LGTM with all of Alan's comments considered. ------------- Marked as reviewed by lucy (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16684#pullrequestreview-1736701866 From naoto at openjdk.org Fri Nov 17 19:07:44 2023 From: naoto at openjdk.org (Naoto Sato) Date: Fri, 17 Nov 2023 19:07:44 GMT Subject: RFR: 8319817: Charset constructor should make defensive copy of aliases Message-ID: Fixing `Charset(String, String[])` to make a defensive copy of the given `aliases` when necessary. ------------- Commit messages: - Minor fix to test - Used convenient method in VM - check done early, trusted passes skipped - initial commit Changes: https://git.openjdk.org/jdk/pull/16715/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16715&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319817 Stats: 76 lines in 2 files changed: 74 ins; 1 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16715.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16715/head:pull/16715 PR: https://git.openjdk.org/jdk/pull/16715 From rriggs at openjdk.org Fri Nov 17 19:12:30 2023 From: rriggs at openjdk.org (Roger Riggs) Date: Fri, 17 Nov 2023 19:12:30 GMT Subject: RFR: 8319817: Charset constructor should make defensive copy of aliases In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 19:02:44 GMT, Naoto Sato wrote: > Fixing `Charset(String, String[])` to make a defensive copy of the given `aliases` when necessary. Marked as reviewed by rriggs (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16715#pullrequestreview-1737707755 From alanb at openjdk.org Fri Nov 17 19:19:28 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 17 Nov 2023 19:19:28 GMT Subject: RFR: 8319817: Charset constructor should make defensive copy of aliases In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 19:02:44 GMT, Naoto Sato wrote: > Fixing `Charset(String, String[])` to make a defensive copy of the given `aliases` when necessary. Marked as reviewed by alanb (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16715#pullrequestreview-1737726371 From bpb at openjdk.org Fri Nov 17 20:25:28 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 17 Nov 2023 20:25:28 GMT Subject: RFR: 8319817: Charset constructor should make defensive copy of aliases In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 19:02:44 GMT, Naoto Sato wrote: > Fixing `Charset(String, String[])` to make a defensive copy of the given `aliases` when necessary. +1 ------------- Marked as reviewed by bpb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16715#pullrequestreview-1737858454 From iris at openjdk.org Fri Nov 17 21:36:29 2023 From: iris at openjdk.org (Iris Clark) Date: Fri, 17 Nov 2023 21:36:29 GMT Subject: RFR: 8319817: Charset constructor should make defensive copy of aliases In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 19:02:44 GMT, Naoto Sato wrote: > Fixing `Charset(String, String[])` to make a defensive copy of the given `aliases` when necessary. Marked as reviewed by iris (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16715#pullrequestreview-1737966683 From alanb at openjdk.org Sat Nov 18 08:05:32 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sat, 18 Nov 2023 08:05:32 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v3] In-Reply-To: <1qQ4txOhk5OEHr3fh_PmAGA-Sk-RSXD-U4A3oupjv_I=.8feb9f23-488d-4b8d-b370-55ec2e314f3e@github.com> References: <1qQ4txOhk5OEHr3fh_PmAGA-Sk-RSXD-U4A3oupjv_I=.8feb9f23-488d-4b8d-b370-55ec2e314f3e@github.com> Message-ID: On Thu, 16 Nov 2023 15:52:02 GMT, Matthias Baesken wrote: >> There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > do not change old legacy src/java.base/unix/native/libnet/SdpSupport.c Looks fine, if you are doing any further edits then I think the Windows code should check for SOCKET_ERROR rather than < 0. ------------- PR Review: https://git.openjdk.org/jdk/pull/16684#pullrequestreview-1738299737 From alanb at openjdk.org Sat Nov 18 08:05:35 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sat, 18 Nov 2023 08:05:35 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v3] In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 13:22:10 GMT, Matthias Baesken wrote: >> src/java.base/windows/native/libnio/ch/Net.c line 160: >> >>> 158: >>> 159: /* IPV6_V6ONLY is true by default */ >>> 160: /* attempt to disable IPV6_V6ONLY to ensure dual-socket support; ignore errors */ >> >> You can remove the previous comment at L159. > > I adjusted the comment. I mean L159 can be removed, I don't think we need that now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16684#discussion_r1398127577 From jpai at openjdk.org Sat Nov 18 10:57:30 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Sat, 18 Nov 2023 10:57:30 GMT Subject: RFR: 8319817: Charset constructor should make defensive copy of aliases In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 19:02:44 GMT, Naoto Sato wrote: > Fixing `Charset(String, String[])` to make a defensive copy of the given `aliases` when necessary. Thank you Naoto. This looks good to me. ------------- Marked as reviewed by jpai (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16715#pullrequestreview-1738379196 From mbaesken at openjdk.org Mon Nov 20 08:23:51 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Mon, 20 Nov 2023 08:23:51 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v4] In-Reply-To: References: Message-ID: > There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: Windows: check for SOCKET_ERROR; remove comment line ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16684/files - new: https://git.openjdk.org/jdk/pull/16684/files/ecd11c71..55abd44f Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16684&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16684&range=02-03 Stats: 7 lines in 5 files changed: 0 ins; 2 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/16684.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16684/head:pull/16684 PR: https://git.openjdk.org/jdk/pull/16684 From mbaesken at openjdk.org Mon Nov 20 08:23:53 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Mon, 20 Nov 2023 08:23:53 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v3] In-Reply-To: References: <1qQ4txOhk5OEHr3fh_PmAGA-Sk-RSXD-U4A3oupjv_I=.8feb9f23-488d-4b8d-b370-55ec2e314f3e@github.com> Message-ID: On Sat, 18 Nov 2023 08:02:23 GMT, Alan Bateman wrote: > Looks fine, if you are doing any further edits then I think the Windows code should check for SOCKET_ERROR rather than < 0. I adjusted the checks to SOCKET_ERROR. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16684#issuecomment-1818435490 From alanb at openjdk.org Mon Nov 20 08:38:31 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 20 Nov 2023 08:38:31 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v4] In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 08:23:51 GMT, Matthias Baesken wrote: >> There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > Windows: check for SOCKET_ERROR; remove comment line Marked as reviewed by alanb (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16684#pullrequestreview-1739200427 From vtewari at openjdk.org Mon Nov 20 10:00:33 2023 From: vtewari at openjdk.org (Vyom Tewari) Date: Mon, 20 Nov 2023 10:00:33 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v4] In-Reply-To: References: Message-ID: <0NdUhX6NKFnJ3QNV9mhvHe21GzWQk8-8-0pbQAc88LI=.b555953b-c3a7-4637-88dc-e210ba97be01@github.com> On Mon, 20 Nov 2023 08:23:51 GMT, Matthias Baesken wrote: >> There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > Windows: check for SOCKET_ERROR; remove comment line Looks OK to me. ------------- Marked as reviewed by vtewari (Committer). PR Review: https://git.openjdk.org/jdk/pull/16684#pullrequestreview-1739387761 From alanb at openjdk.org Mon Nov 20 10:33:28 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 20 Nov 2023 10:33:28 GMT Subject: RFR: 8320386: Remove java/nio/channels/vthread/BlockingChannelOps.java#direct-register from ProblemList In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 10:09:36 GMT, Daniel Jeli?ski wrote: > The direct-register test variant was removed in c099cf53f25496c99629dc578045aa5186e1109d. The failing test case was also modified - the socket timeout is much shorter now, so even if the receive operation gets stuck for the full duration, the test will still pass. Yes, this exclude list was missed with the poller update so the test id no longer matches. It's okay to remove now. ------------- Marked as reviewed by alanb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16732#pullrequestreview-1739458698 From mbaesken at openjdk.org Mon Nov 20 11:31:44 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Mon, 20 Nov 2023 11:31:44 GMT Subject: RFR: JDK-8320168: handle setsocktopt return values [v4] In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 08:23:51 GMT, Matthias Baesken wrote: >> There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > Windows: check for SOCKET_ERROR; remove comment line Thanks for the reviews ! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16684#issuecomment-1818870914 From mbaesken at openjdk.org Mon Nov 20 11:31:45 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Mon, 20 Nov 2023 11:31:45 GMT Subject: Integrated: JDK-8320168: handle setsocktopt return values In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 08:39:32 GMT, Matthias Baesken wrote: > There are a few places in the JDK C coding where the setsocktopt return value is not handled but better should be handled. This pull request has now been integrated. Changeset: db1d8234 Author: Matthias Baesken URL: https://git.openjdk.org/jdk/commit/db1d82347bb18e21c4c6a18076ffdaf17724c733 Stats: 53 lines in 8 files changed: 38 ins; 0 del; 15 mod 8320168: handle setsocktopt return values Reviewed-by: lucy, alanb, vtewari ------------- PR: https://git.openjdk.org/jdk/pull/16684 From djelinski at openjdk.org Mon Nov 20 12:49:44 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 20 Nov 2023 12:49:44 GMT Subject: RFR: 8320386: Remove java/nio/channels/vthread/BlockingChannelOps.java#direct-register from ProblemList In-Reply-To: References: Message-ID: <2iTYgMqcdyUHbkNu43RWMYvMsCHoWYagaIFK0Om-0gs=.12f16611-3621-4fa6-b4aa-82ff2896df52@github.com> On Mon, 20 Nov 2023 10:09:36 GMT, Daniel Jeli?ski wrote: > The direct-register test variant was removed in c099cf53f25496c99629dc578045aa5186e1109d. The failing test case was also modified - the socket timeout is much shorter now, so even if the receive operation gets stuck for the full duration, the test will still pass. Thanks for the quick review. Given the trivial nature of this change, I'm going to integrate it now. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16732#issuecomment-1818996881 From djelinski at openjdk.org Mon Nov 20 12:49:45 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 20 Nov 2023 12:49:45 GMT Subject: Integrated: 8320386: Remove java/nio/channels/vthread/BlockingChannelOps.java#direct-register from ProblemList In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 10:09:36 GMT, Daniel Jeli?ski wrote: > The direct-register test variant was removed in c099cf53f25496c99629dc578045aa5186e1109d. The failing test case was also modified - the socket timeout is much shorter now, so even if the receive operation gets stuck for the full duration, the test will still pass. This pull request has now been integrated. Changeset: 9f6a4a3f Author: Daniel Jeli?ski URL: https://git.openjdk.org/jdk/commit/9f6a4a3f9c5ee0fbb65c794d657fffca7d19927f Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod 8320386: Remove java/nio/channels/vthread/BlockingChannelOps.java#direct-register from ProblemList Reviewed-by: alanb ------------- PR: https://git.openjdk.org/jdk/pull/16732 From naoto at openjdk.org Mon Nov 20 17:35:25 2023 From: naoto at openjdk.org (Naoto Sato) Date: Mon, 20 Nov 2023 17:35:25 GMT Subject: Integrated: 8319817: Charset constructor should make defensive copy of aliases In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 19:02:44 GMT, Naoto Sato wrote: > Fixing `Charset(String, String[])` to make a defensive copy of the given `aliases` when necessary. This pull request has now been integrated. Changeset: d6d7bdc7 Author: Naoto Sato URL: https://git.openjdk.org/jdk/commit/d6d7bdc7748c10963c3e58c0287b2472646bf36f Stats: 76 lines in 2 files changed: 74 ins; 1 del; 1 mod 8319817: Charset constructor should make defensive copy of aliases Reviewed-by: rriggs, alanb, bpb, iris, jpai ------------- PR: https://git.openjdk.org/jdk/pull/16715 From bpb at openjdk.org Mon Nov 20 23:30:45 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 20 Nov 2023 23:30:45 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions [v2] In-Reply-To: References: Message-ID: > Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - 8298318: Correct type in path.getExtension spec - Merge - 8298318: (fs) APIs for handling filename extensions ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16226/files - new: https://git.openjdk.org/jdk/pull/16226/files/b34c9378..5a926bdc Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16226&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16226&range=00-01 Stats: 685919 lines in 3006 files changed: 119418 ins; 488709 del; 77792 mod Patch: https://git.openjdk.org/jdk/pull/16226.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16226/head:pull/16226 PR: https://git.openjdk.org/jdk/pull/16226 From bpb at openjdk.org Tue Nov 21 00:08:06 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 21 Nov 2023 00:08:06 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions In-Reply-To: <1TGXNr0UsTo_R0ejnqpnElGbgO9iBbz6WznZmdiO8ug=.bd09168c-7199-4b4e-8911-5d3387762b30@github.com> References: <1TGXNr0UsTo_R0ejnqpnElGbgO9iBbz6WznZmdiO8ug=.bd09168c-7199-4b4e-8911-5d3387762b30@github.com> Message-ID: On Fri, 3 Nov 2023 19:28:24 GMT, Roger Riggs wrote: > String s = name.substring(0, indexOfExt); I assume that `indexOfExt` is a typo which should be simply `index`. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1820003358 From alanb at openjdk.org Tue Nov 21 06:44:14 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 21 Nov 2023 06:44:14 GMT Subject: RFR: 8282726: java/net/vthread/BlockingSocketOps.java timeout/hang intermittently on Windows Message-ID: This is a Windows specific issue where a virtual thread attempting to establish a connection appears to hang, and the test eventually times out. It initially looked like an issue on older releases of Windows but it turns out to be an issue that arises when a SOCKET handle is recycled quickly. Thanks to Daniel Jeli?ski for spending time to find a way to reproduce this quickly, up to now it was too intermittent and needed tens of thousands of runs to having some chance of reproducing. When a SOCKET has been polled, the event is disabled rather than removed from wepoll / AFD poll. When the SOCKET is closed and immediately recycled then there is a race with the close event and registration the "new" socket for events. The change that we've tested is to remove the event after polling. This is done in the client thread rather than the poller to avoid racing with an async close. As part of this change I've removed a temporary change to the Windows Pipe implementation and also remove a connect timeout that went into a few tests to workaround the issue. ------------- Commit messages: - Fix AIX - Initial commit Changes: https://git.openjdk.org/jdk/pull/16742/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16742&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8282726 Stats: 42 lines in 8 files changed: 9 ins; 11 del; 22 mod Patch: https://git.openjdk.org/jdk/pull/16742.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16742/head:pull/16742 PR: https://git.openjdk.org/jdk/pull/16742 From djelinski at openjdk.org Tue Nov 21 09:36:10 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Tue, 21 Nov 2023 09:36:10 GMT Subject: RFR: 8282726: java/net/vthread/BlockingSocketOps.java timeout/hang intermittently on Windows In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 16:08:10 GMT, Alan Bateman wrote: > This is a Windows specific issue where a virtual thread attempting to establish a connection appears to hang, and the test eventually times out. It initially looked like an issue on older releases of Windows but it turns out to be an issue that arises when a SOCKET handle is recycled quickly. Thanks to Daniel Jeli?ski for spending time to find a way to reproduce this quickly, up to now it was too intermittent and needed tens of thousands of runs to have some chance of reproducing. > > When a SOCKET has been polled, the event is disabled rather than removed from wepoll / AFD poll. When the SOCKET is closed and immediately recycled then there is a race with the close event and registration of the "new" socket for events. The change that we've tested is to remove the event after polling. This is done virtual thread "client" rather than the poller to avoid racing with an async close. > > As part of this change I've removed a temporary change to the Windows Pipe implementation and also remove the connect timeout that went into a few tests to workaround the issue. LGTM. Thanks for fixing this! ------------- Marked as reviewed by djelinski (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16742#pullrequestreview-1741507555 From alanb at openjdk.org Tue Nov 21 14:08:16 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 21 Nov 2023 14:08:16 GMT Subject: RFR: 8282726: java/net/vthread/BlockingSocketOps.java timeout/hang intermittently on Windows In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 16:08:10 GMT, Alan Bateman wrote: > This is a Windows specific issue where a virtual thread attempting to establish a connection appears to hang, and the test eventually times out. It initially looked like an issue on older releases of Windows but it turns out to be an issue that arises when a SOCKET handle is recycled quickly. Thanks to Daniel Jeli?ski for spending time to find a way to reproduce this quickly, up to now it was too intermittent and needed tens of thousands of runs to have some chance of reproducing. > > When a SOCKET has been polled, the event is disabled rather than removed from wepoll / AFD poll. When the SOCKET is closed and immediately recycled then there is a race with the close event and registration of the "new" socket for events. The change that we've tested is to remove the event after polling. This is done virtual thread "client" rather than the poller to avoid racing with an async close. > > As part of this change I've removed a temporary change to the Windows Pipe implementation and also remove the connect timeout that went into a few tests to workaround the issue. Matthias Baesken has confirmed that this builds on AIX and that the BlockingChannelOps tests passes, thanks Matthias. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16742#issuecomment-1820986083 From alanb at openjdk.org Tue Nov 21 14:08:17 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 21 Nov 2023 14:08:17 GMT Subject: Integrated: 8282726: java/net/vthread/BlockingSocketOps.java timeout/hang intermittently on Windows In-Reply-To: References: Message-ID: <4C6OufeFaFqbEpoH8br1TLbFSryRtAjc2Y0cW0oZ-zk=.761932a5-e36b-42eb-823f-32a283377538@github.com> On Mon, 20 Nov 2023 16:08:10 GMT, Alan Bateman wrote: > This is a Windows specific issue where a virtual thread attempting to establish a connection appears to hang, and the test eventually times out. It initially looked like an issue on older releases of Windows but it turns out to be an issue that arises when a SOCKET handle is recycled quickly. Thanks to Daniel Jeli?ski for spending time to find a way to reproduce this quickly, up to now it was too intermittent and needed tens of thousands of runs to have some chance of reproducing. > > When a SOCKET has been polled, the event is disabled rather than removed from wepoll / AFD poll. When the SOCKET is closed and immediately recycled then there is a race with the close event and registration of the "new" socket for events. The change that we've tested is to remove the event after polling. This is done virtual thread "client" rather than the poller to avoid racing with an async close. > > As part of this change I've removed a temporary change to the Windows Pipe implementation and also remove the connect timeout that went into a few tests to workaround the issue. This pull request has now been integrated. Changeset: 21a59b9f Author: Alan Bateman URL: https://git.openjdk.org/jdk/commit/21a59b9f4e46ebd32cff8f1000fe9ad56c918431 Stats: 42 lines in 8 files changed: 9 ins; 11 del; 22 mod 8282726: java/net/vthread/BlockingSocketOps.java timeout/hang intermittently on Windows Reviewed-by: djelinski ------------- PR: https://git.openjdk.org/jdk/pull/16742 From dfuchs at openjdk.org Tue Nov 21 14:12:24 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Tue, 21 Nov 2023 14:12:24 GMT Subject: Integrated: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out In-Reply-To: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> References: <9_YEuKX3kjiLv4lv9FeijmWtqna8JYW1LaBFtN6BYaw=.40e6c6be-bce1-4584-86a4-b0771602b3af@github.com> Message-ID: On Tue, 14 Nov 2023 17:49:30 GMT, Daniel Fuchs wrote: > Please find here a fix for a timeout issue observed in DatagramChannel/Connect.java > > The suspicion is that the responder may have received some message from some other source and replied to that, leaving the initiator waiting forever. > > The proposed fix makes sure the messages exchanged are unique and identifiable and ensure that both the initiator and responder will discard any message that do not have the expected content. > > Additional logging should help with diagnosis if the test fails again. This pull request has now been integrated. Changeset: 570dffb1 Author: Daniel Fuchs URL: https://git.openjdk.org/jdk/commit/570dffb104fc37f053fcdf38a24aa2cabdc921c0 Stats: 79 lines in 1 file changed: 43 ins; 0 del; 36 mod 8310807: java/nio/channels/DatagramChannel/Connect.java timed out Reviewed-by: msheppar, jpai ------------- PR: https://git.openjdk.org/jdk/pull/16661 From duke at openjdk.org Tue Nov 21 17:41:11 2023 From: duke at openjdk.org (Anthony Vanelverdinghe) Date: Tue, 21 Nov 2023 17:41:11 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions [v2] In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 23:30:45 GMT, Brian Burkhalter wrote: >> Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - 8298318: Correct type in path.getExtension spec > - Merge > - 8298318: (fs) APIs for handling filename extensions My two cents: * for `foo.`, wouldn't `removeExtension().addExtension(getExtension())` result in `foo`, i.e. the invariant be broken? * for `.gitconfig`, the current implementation of `getExtension()` returns `""` , but according to Windows conventions, the extension is `gitconfig` * both PowerShell and Python (just 2 I tried) include the dot in the extension, and it seems natural to me to have an invariant that `base + extension = filename`. I assume this was discussed at length already though, so I'm just curious about the rationale for excluding the dot? * has the idea of introducing a new class and adding a method to Path (`getFileName()` is already taken, so I'd propose `filename()`) been explored? For example, recently I needed to work with extensions and ended up with something like below. final class Filename implements CharSequence { enum Convention { DEFAULT, WINDOWS } String base() String base(Convention convention) String extension() String extension(Convention convention) LocalizedBase localizedBase() {} LocalizedBase localizedBase(Convention convention) { // e.g. Java .properties files or .srt subtitle files record LocalizedBase(String root, Locale locale) {} } ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1821371321 From bpb at openjdk.org Tue Nov 21 19:00:10 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 21 Nov 2023 19:00:10 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions [v2] In-Reply-To: References: Message-ID: <2C98C_sdB7aswkyMIQ49i3r9bWSmpr5mV9TPLDvPvs8=.2c6f42ee-17c6-40e0-8200-43f94a9f7b58@github.com> On Tue, 21 Nov 2023 17:38:44 GMT, Anthony Vanelverdinghe wrote: > My two cents: Thanks for the comments. > * for `foo.`, wouldn't `removeExtension().addExtension(getExtension())` result in `foo`, i.e. the invariant be broken? It looks like it would be. Another problematic corner case. > * for `.gitconfig`, the current implementation of `getExtension()` returns `""` , but according to Windows conventions, the extension is `gitconfig` And according to the UNIX `FILE(1)` command $ file --extension .gitconfig .gitconfig: ??? there is no extension. Many different tools and APIs were examined in the past and there is no consensus as to whether to include the period character. It's almost a 50-50 split. > * both PowerShell and Python (just 2 I tried) include the dot in the extension, and it seems natural to me to have an invariant that `base + extension = filename`. I assume this was discussed at length already though, so I'm just curious about the rationale for excluding the dot? It has been discussed at length in prior incarnations of this effort. The invariant is an appealing idea. _My_ rationale for exlcuding the dot is that is analogous to a separator such as `/` or `` and conveys no information on its own. In my own experience in other areas such as media (imaging and video) I have rarely seen the period character included in the extension, and if it was detected it was removed. > * has the idea of introducing a new class and adding a method to Path (`getFileName()` is already taken, so I'd propose `filename()`) been explored? For example, recently I needed to work with extensions and ended up with something like below. This has not been discussed but I doubt it would be a very popular idea. One thing is that it would add complexity and API surface. Another is that OS-specific behavior is not usually codified. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1821482681 From tprinzing at openjdk.org Tue Nov 21 20:25:15 2023 From: tprinzing at openjdk.org (Tim Prinzing) Date: Tue, 21 Nov 2023 20:25:15 GMT Subject: RFR: 8310994: Add JFR event for selection operations Message-ID: Added mirror event with static methods: jdk.internal.event.SelectionEvent that provides the duration of select calls and the count of how many keys are available. Emit the event from SelectorImpl::lockAndDoSelect Test at jdk.jfr.event.io.TestSelectionEvents ------------- Commit messages: - remove trailing whitespace - event logic outside of the lock, selector in try block - remove unused import - fix TestConfigure failure - add event defaults - Merge branch 'master' into JDK-8310994 - minor test cleanup - Merge branch 'master' into JDK-8310994 - 8310994: Add JFR event for selection operations Changes: https://git.openjdk.org/jdk/pull/16710/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16710&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8310994 Stats: 265 lines in 9 files changed: 264 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16710.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16710/head:pull/16710 PR: https://git.openjdk.org/jdk/pull/16710 From alanb at openjdk.org Tue Nov 21 20:25:16 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 21 Nov 2023 20:25:16 GMT Subject: RFR: 8310994: Add JFR event for selection operations In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 16:22:55 GMT, Tim Prinzing wrote: > Added mirror event with static methods: jdk.internal.event.SelectionEvent that provides the duration of select calls and the count of how many keys are available. > > Emit the event from SelectorImpl::lockAndDoSelect > > Test at jdk.jfr.event.io.TestSelectionEvents src/java.base/share/classes/sun/nio/ch/SelectorImpl.java line 141: > 139: SelectionEvent.commit(start, duration, n); > 140: } > 141: return n; I'd prefer if the existing code moved to implLockAndDoSelect so that lockAndDoSelect commits the event outside of the locked region. test/jdk/jdk/jfr/event/io/TestSelectionEvents.java line 70: > 68: SocketChannel sc2 = ssc.accept()) { > 69: > 70: Selector sel = Selector.open(); You can add this to the try block so that the selector is closed too. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16710#discussion_r1397596209 PR Review Comment: https://git.openjdk.org/jdk/pull/16710#discussion_r1397593894 From duke at openjdk.org Wed Nov 22 08:04:10 2023 From: duke at openjdk.org (Anthony Vanelverdinghe) Date: Wed, 22 Nov 2023 08:04:10 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions [v2] In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 23:30:45 GMT, Brian Burkhalter wrote: >> Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - 8298318: Correct type in path.getExtension spec > - Merge > - 8298318: (fs) APIs for handling filename extensions For the `.gitconfig` example, I just wanted to point out that the whole concept of a filename extension is merely an OS-specific convention. So I'd argue that the JDK should be neutral, rather than pick one specific convention. Since you mentioned "OS-specific behavior is not usually codified", the `Convention` enum I used is not an option, but the `getExtension` implementation could behave differently depending on its environment and specify something like `the exact definition of "extension" is implementation specific`. About whether to include the dot: I'm not a mathematician, but intuitively I'd say that excluding it makes it impossible to implement the API in such a way that `equals(removeExtension().addExtension(getExtension()))` always holds. On the other hand, including it would make it trivial to do so. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1822276138 From dfuchs at openjdk.org Wed Nov 22 12:13:11 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Wed, 22 Nov 2023 12:13:11 GMT Subject: RFR: 8310994: Add JFR event for selection operations In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 16:22:55 GMT, Tim Prinzing wrote: > Added mirror event with static methods: jdk.internal.event.SelectionEvent that provides the duration of select calls and the count of how many keys are available. > > Emit the event from SelectorImpl::lockAndDoSelect > > Test at jdk.jfr.event.io.TestSelectionEvents src/java.base/share/classes/jdk/internal/event/SelectionEvent.java line 35: > 33: * {@link #commit(long, long, int)} method > 34: * must be the same as the order of the fields. > 35: */ You should probably define what a "selection operation" is and put a link to `Selector::select`. src/java.base/share/classes/jdk/internal/event/SelectionEvent.java line 38: > 36: public class SelectionEvent extends Event { > 37: > 38: public int count; It could also be interesting to provide the `timeout` that was given to the selection operation. src/java.base/share/classes/sun/nio/ch/SelectorImpl.java line 150: > 148: long duration = SelectionEvent.timestamp() - start; > 149: if (SelectionEvent.shouldCommit(duration)) { > 150: SelectionEvent.commit(start, duration, n); Maybe the value of `timeout` should be provided here? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16710#discussion_r1401948063 PR Review Comment: https://git.openjdk.org/jdk/pull/16710#discussion_r1401950078 PR Review Comment: https://git.openjdk.org/jdk/pull/16710#discussion_r1401951825 From alanb at openjdk.org Wed Nov 22 12:30:06 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 22 Nov 2023 12:30:06 GMT Subject: RFR: 8310994: Add JFR event for selection operations In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 16:22:55 GMT, Tim Prinzing wrote: > Added mirror event with static methods: jdk.internal.event.SelectionEvent that provides the duration of select calls and the count of how many keys are available. > > Emit the event from SelectorImpl::lockAndDoSelect > > Test at jdk.jfr.event.io.TestSelectionEvents src/jdk.jfr/share/classes/jdk/jfr/events/SelectionEvent.java line 43: > 41: > 42: @Label("Selection Count") > 43: @Description("Number of channels selected") I suspect you'll need to rename this event to something like "SelectorSelect" as "Selection" could be anything. We'll to find a better name for the field and its label too. There are two form of selection operations. One form operates on a selected-key set where the select/selectNow methods returns the number of keys aded to the Selector's ready set. The other form performs an action on each selected key. I'll try to come up a suggestions for the names, I suspect a label "number of channels ready for I/O or added to ready set" would be the most accurate. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16710#discussion_r1401970073 From alanb at openjdk.org Wed Nov 22 12:36:06 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 22 Nov 2023 12:36:06 GMT Subject: RFR: 8310994: Add JFR event for selection operations In-Reply-To: References: Message-ID: <1ysPxgEJtBJ-Isq2-punGaxcWpN0fXvTA2sBsE9zJjI=.d3b2aeb0-b7c6-429a-a2e3-a9c8658c5bc4@github.com> On Fri, 17 Nov 2023 16:22:55 GMT, Tim Prinzing wrote: > Added mirror event with static methods: jdk.internal.event.SelectionEvent that provides the duration of select calls and the count of how many keys are available. > > Emit the event from SelectorImpl::lockAndDoSelect > > Test at jdk.jfr.event.io.TestSelectionEvents src/jdk.jfr/share/conf/jfr/default.jfc line 745: > 743: true > 744: true > 745: 20 ms The default threshold is 20ms but that means the default won't emit en event when spinning. If this event is used for performance and troubleshooting purposes then maybe we should emit an event when select returns 0 and not emit an event when selectNow returns 0 (selectNow is used to flush channels from the Selector so returning 0 is normal and not interesting). ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16710#discussion_r1401979076 From rriggs at openjdk.org Wed Nov 22 17:41:12 2023 From: rriggs at openjdk.org (Roger Riggs) Date: Wed, 22 Nov 2023 17:41:12 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions [v2] In-Reply-To: References: Message-ID: <9iN2k63EPOMEylrmmGWjbbGYV7vCdFzGqYAJPTNiEok=.7d8b807e-75eb-46c8-b96d-8945a6d69ed8@github.com> On Mon, 20 Nov 2023 23:30:45 GMT, Brian Burkhalter wrote: >> Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - 8298318: Correct type in path.getExtension spec > - Merge > - 8298318: (fs) APIs for handling filename extensions I thought we were moving away from the addExtension/removeExtension methods to the recommendation for `withExtension` and `withoutExtension`. They have more useful semantics and avoid having to through multiple intermediate Paths to get to the desired result. They could include (some) OS specific semantics while producing the useful and easy to understand result. The APIs should not require the developer to be aware of and code around OS specific details. For an outlier like ".gitconfig" they won't be using any of these functions, that's the full file name, semantically it does not have an extension and certainly one would not consider the leading "." to be the base name without the extension; that would be a directory. Even if the "." were considered to be part of the extension, that would leave the empty string as the base and is not meaningful if its empty. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1823209088 From duke at openjdk.org Thu Nov 23 07:04:09 2023 From: duke at openjdk.org (Anthony Vanelverdinghe) Date: Thu, 23 Nov 2023 07:04:09 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions [v2] In-Reply-To: <9iN2k63EPOMEylrmmGWjbbGYV7vCdFzGqYAJPTNiEok=.7d8b807e-75eb-46c8-b96d-8945a6d69ed8@github.com> References: <9iN2k63EPOMEylrmmGWjbbGYV7vCdFzGqYAJPTNiEok=.7d8b807e-75eb-46c8-b96d-8945a6d69ed8@github.com> Message-ID: On Wed, 22 Nov 2023 17:37:57 GMT, Roger Riggs wrote: > I thought we were moving away from the addExtension/removeExtension methods to the recommendation for `withExtension` and `withoutExtension`. My bad. The point about it being impossible(?) to specify an invariant with these methods still holds though. > The APIs should not require the developer to be aware of and code around OS specific details. For an outlier like ".gitconfig" they won't be using any of these functions, that's the full file name, semantically it does not have an extension and certainly one would not consider the leading "." to be the base name without the extension; that would be a directory. Even if the "." were considered to be part of the extension, that would leave the empty string as the base and is not meaningful if its empty. `.gitconfig` was a bad example. Consider `Path.of("C:\\Music\.mp3").getExtension()` instead. What I'm advocating, is that this should result in an extension of `mp3`, because that's what Windows users expect (& what I would expect if I were to use this API). On Windows, it's perfectly acceptable for the base name to be empty. (In an ideal world, cross-platform applications like Git would adapt to OS-specific conventions and the file would be named `gitconfig` on Windows.) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1823894253 From egahlin at openjdk.org Thu Nov 23 11:21:07 2023 From: egahlin at openjdk.org (Erik Gahlin) Date: Thu, 23 Nov 2023 11:21:07 GMT Subject: RFR: 8310994: Add JFR event for selection operations In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 12:25:45 GMT, Alan Bateman wrote: >> Added mirror event with static methods: jdk.internal.event.SelectionEvent that provides the duration of select calls and the count of how many keys are available. >> >> Emit the event from SelectorImpl::lockAndDoSelect >> >> Test at jdk.jfr.event.io.TestSelectionEvents > > src/jdk.jfr/share/classes/jdk/jfr/events/SelectionEvent.java line 43: > >> 41: >> 42: @Label("Selection Count") >> 43: @Description("Number of channels selected") > > I suspect you'll need to rename this event to something like "SelectorSelect" as "Selection" could be anything. > > We'll to find a better name for the field and the label too. There are two forms of selection operations. One form operates on a selected-key set where the select/selectNow methods returns the number of keys aded to the Selector's ready set. The other form performs an action on each selected key. I'll try to come up a suggestions for the names, I suspect a label "number of channels ready for I/O or added to ready set" would be the most accurate. It would also be good if the name reflect that it is related to channels so it won't clash with other events in the future. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16710#discussion_r1403242271 From duke at openjdk.org Sun Nov 26 22:55:22 2023 From: duke at openjdk.org (duke) Date: Sun, 26 Nov 2023 22:55:22 GMT Subject: Withdrawn: 8308807: MulticastSocket cannot join IPv4 multicast group when IPv6 is enabled (aix) In-Reply-To: <4tkYKvpBoxO7IK4HevAqpEY5KoqxXUK6rs0scqyxgJc=.69de9b21-5a2f-45ef-b05f-1bc8ad398dad@github.com> References: <4tkYKvpBoxO7IK4HevAqpEY5KoqxXUK6rs0scqyxgJc=.69de9b21-5a2f-45ef-b05f-1bc8ad398dad@github.com> Message-ID: On Thu, 25 May 2023 07:14:19 GMT, Deepa Kumari wrote: > DatagramSocket delegates to an inner DatagramSocket object. Irrespective of whether datagramSocket is IPv4 or IPv6, we create an IPv6 datagramChannel as its's delegate. So, This can cause problems with operations like joinGroup. > > On AIX, IPv6 datagramSocket can not join an IPv4 multicast group. > > These failures can be fixed by making sure that the delegate created for a datagram socket has the same protocol family. > > > > > Reported Issue : [JDK-8308807](https://bugs.openjdk.org/browse/JDK-8308807) This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/14142 From rriggs at openjdk.org Mon Nov 27 17:02:13 2023 From: rriggs at openjdk.org (Roger Riggs) Date: Mon, 27 Nov 2023 17:02:13 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions [v2] In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 23:30:45 GMT, Brian Burkhalter wrote: >> Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - 8298318: Correct type in path.getExtension spec > - Merge > - 8298318: (fs) APIs for handling filename extensions Two popular Java libraries for file access and path utilities are Apache Commons and Guava, both do not consider the "." to be part of the extension. They also do not consider a leading "." to be a delimiter of an extension. It may cause confusion for Java developers and lead to bugs to add an API with different conventions. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1828252931 From duke at openjdk.org Mon Nov 27 17:57:07 2023 From: duke at openjdk.org (Anthony Vanelverdinghe) Date: Mon, 27 Nov 2023 17:57:07 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions [v2] In-Reply-To: References: Message-ID: On Mon, 27 Nov 2023 16:59:34 GMT, Roger Riggs wrote: > Two popular Java libraries for file access and path utilities are Apache Commons and Guava, > both do not consider the "." to be part of the extension. Point taken, it would be confusing if the JDK would behave otherwise, though only for Java developers who are actually accustomed to the methods in those libraries. Given how easy it is to just do `filename.substring(filename.lastIndexOf('.'))`, I'd argue that there's at least as many Java developers who would expect the dot to be part of the extension. > They also do not consider a leading "." to be a delimiter of an extension. This is factually false: both [Guava](https://guava.dev/releases/32.1.3-jre/api/docs/com/google/common/io/MoreFiles.html#getFileExtension(java.nio.file.Path)) and [Apache Commons](https://commons.apache.org/proper/commons-io/apidocs/org/apache/commons/io/FilenameUtils.html#getExtension(java.lang.String)) consider a leading dot to be a delimiter of an extension, both in their Javadoc specification and in their implementation: ![image](https://github.com/openjdk/jdk/assets/15714253/dfb92224-d871-4b70-8fec-675e93264de4) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1828339847 From rriggs at openjdk.org Mon Nov 27 23:43:10 2023 From: rriggs at openjdk.org (Roger Riggs) Date: Mon, 27 Nov 2023 23:43:10 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions [v2] In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 23:30:45 GMT, Brian Burkhalter wrote: >> Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - 8298318: Correct type in path.getExtension spec > - Merge > - 8298318: (fs) APIs for handling filename extensions Indeed, I was looking for more from those APIs than is there. They have the simplest definition, but one that overlooks the various ways developers can carelessly cause themselves trouble. That lead to suggestions to avoid pitfalls. Filenames that start with "." introduce a number of special cases. "." and ".." are not just conventions but have strong semantics. Considering either of them to have an extension is simple, but neither is useful. Removing the extension from "..", leaves "." which is not the base of "..", its an entirely different thing. Similarly, removing the extension of "." leaves the empty string, not a valid filename, though sometimes a no-op when concatenating strings. For other files that begin with ".", it is conventional to name the file using the entire string. For example, the directory ".git" is not a empty filename with a "git" extension; it is the full name of the directory. Other dot files such as ".bashrc" or ".qtrc" are not related though both would have the same base (empty) if the leading "." was considered the start of an extension. More nuanced semantics for working with extensions may be more useful to developers and less open to misuse than the simplest literal parsing. A scan of StackOverflow questions related to file extensions shows a wide variety of uses and suggestions for how to remove or extract extensions that have a more robust handling of the edge cases. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1828810099 From michaelm at openjdk.org Tue Nov 28 09:52:22 2023 From: michaelm at openjdk.org (Michael McMahon) Date: Tue, 28 Nov 2023 09:52:22 GMT Subject: RFR: 8319417: (dc) DatagramChannel.connect undocumented behavior Message-ID: Hi, This is a small doc change for DatagramChannel.getLocalAddress() and the equivalent methods on DatagramSocket which documents some existing (but not currently documented) behavior. Thanks, Michael ------------- Commit messages: - Alan round 2 - Alan review - Merge branch 'master' into 8319417-datagram - update - update 1 - spec update Changes: https://git.openjdk.org/jdk/pull/16635/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16635&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319417 Stats: 20 lines in 2 files changed: 19 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16635.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16635/head:pull/16635 PR: https://git.openjdk.org/jdk/pull/16635 From alanb at openjdk.org Tue Nov 28 12:21:11 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 28 Nov 2023 12:21:11 GMT Subject: RFR: 8319417: (dc) DatagramChannel.connect undocumented behavior In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 17:28:29 GMT, Michael McMahon wrote: > Hi, > > This is a small doc change for DatagramChannel.getLocalAddress() and the equivalent methods on DatagramSocket which documents some existing (but not currently documented) behavior. > > Thanks, > Michael Marked as reviewed by alanb (Reviewer). src/java.base/share/classes/java/nio/channels/DatagramChannel.java line 665: > 663: * {@code SocketAddress} representing the loopback address if > 664: * denied by the security manager, or {@code null} if the > 665: * channel's socket is not bound. The period at the end of the return description isn't needed, the rest looks good. ------------- PR Review: https://git.openjdk.org/jdk/pull/16635#pullrequestreview-1752749517 PR Review Comment: https://git.openjdk.org/jdk/pull/16635#discussion_r1407672441 From michaelm at openjdk.org Tue Nov 28 12:31:22 2023 From: michaelm at openjdk.org (Michael McMahon) Date: Tue, 28 Nov 2023 12:31:22 GMT Subject: RFR: 8319417: (dc) DatagramChannel.connect undocumented behavior [v2] In-Reply-To: References: Message-ID: > Hi, > > This is a small doc change for DatagramChannel.getLocalAddress() and the equivalent methods on DatagramSocket which documents some existing (but not currently documented) behavior. > > Thanks, > Michael Michael McMahon has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: - removed stray period - Merge branch 'master' into 8319417-datagram - Alan round 2 - Alan review - Merge branch 'master' into 8319417-datagram - update - update 1 - spec update ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16635/files - new: https://git.openjdk.org/jdk/pull/16635/files/0dfe2bf7..92eb98d0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16635&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16635&range=00-01 Stats: 3885 lines in 110 files changed: 2727 ins; 549 del; 609 mod Patch: https://git.openjdk.org/jdk/pull/16635.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16635/head:pull/16635 PR: https://git.openjdk.org/jdk/pull/16635 From msheppar at openjdk.org Tue Nov 28 13:39:09 2023 From: msheppar at openjdk.org (Mark Sheppard) Date: Tue, 28 Nov 2023 13:39:09 GMT Subject: RFR: 8319417: (dc) DatagramChannel.connect undocumented behavior [v2] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 12:31:22 GMT, Michael McMahon wrote: >> Hi, >> >> This is a small doc change for DatagramChannel.getLocalAddress() and the equivalent methods on DatagramSocket which documents some existing (but not currently documented) behavior. >> >> Thanks, >> Michael > > Michael McMahon has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: > > - removed stray period > - Merge branch 'master' into 8319417-datagram > - Alan round 2 > - Alan review > - Merge branch 'master' into 8319417-datagram > - update > - update 1 > - spec update The current wording says that: "If the channel's socket was initially bound to the wildcard address and is now {@link #isConnected connected}, then the address returned may be the local address selected as the source address for datagrams sent via this channel instead of the wildcard address." The may is a bit ambiguous ... is this OS dependent or is it universal that for a connected DatagramSocket or DatagramChannel then the local address of the intreface for which the send was routed will be returned. If it is OS dependent, should that be stated, also ? Also, the return for the case of an unbound DatagramSocket or DatagramChannel is not specified. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16635#issuecomment-1829859589 From alanb at openjdk.org Tue Nov 28 13:51:49 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 28 Nov 2023 13:51:49 GMT Subject: RFR: 8319417: (dc) DatagramChannel.connect undocumented behavior [v2] In-Reply-To: References: Message-ID: <1BIOnqHASe_6tnf9L4dVkneBmk520pu_XpUTbhEfV2M=.d9c6e525-b914-4dd7-9a2e-04d5f0088f35@github.com> On Tue, 28 Nov 2023 13:35:33 GMT, Mark Sheppard wrote: > Also, the return for the case of an unbound DatagramSocket or DatagramChannel is not specified. DatagramChannel::getLocalAddress is specified to return null when channel's socket is not bound. DatagramSocket::getLocalSocketAddress also specifies that it returns null when not bound. But maybe you mean something else? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16635#issuecomment-1829883200 From michaelm at openjdk.org Tue Nov 28 16:46:07 2023 From: michaelm at openjdk.org (Michael McMahon) Date: Tue, 28 Nov 2023 16:46:07 GMT Subject: RFR: 8319417: (dc) DatagramChannel.connect undocumented behavior [v2] In-Reply-To: References: Message-ID: <2h-LLES9Y3J_PP5vnVcpvXSx76775uiIoZb0e_Y1IT4=.0b6e0f1f-5c7d-440d-bbe0-519682646fe1@github.com> On Tue, 28 Nov 2023 13:35:33 GMT, Mark Sheppard wrote: > The current wording says that: "If the channel's socket was initially bound to the wildcard address and is now {@link #isConnected connected}, then the address returned may be the local address selected as the source address for datagrams sent via this channel instead of the wildcard address." > > The may is a bit ambiguous ... is this OS dependent or is it universal that for a connected DatagramSocket or DatagramChannel then the local address of the intreface for which the send was routed will be returned. If it is OS dependent, should that be stated, also ? > > Also, the return for the case of an unbound DatagramSocket or DatagramChannel is not specified. That is the behavior for the main supported platforms. We had a concern that it might not be the case on other platforms. Hence, the word "may". ------------- PR Comment: https://git.openjdk.org/jdk/pull/16635#issuecomment-1830274066 From msheppar at openjdk.org Tue Nov 28 17:23:22 2023 From: msheppar at openjdk.org (Mark Sheppard) Date: Tue, 28 Nov 2023 17:23:22 GMT Subject: RFR: 8319417: (dc) DatagramChannel.connect undocumented behavior [v2] In-Reply-To: <2h-LLES9Y3J_PP5vnVcpvXSx76775uiIoZb0e_Y1IT4=.0b6e0f1f-5c7d-440d-bbe0-519682646fe1@github.com> References: <2h-LLES9Y3J_PP5vnVcpvXSx76775uiIoZb0e_Y1IT4=.0b6e0f1f-5c7d-440d-bbe0-519682646fe1@github.com> Message-ID: On Tue, 28 Nov 2023 16:43:47 GMT, Michael McMahon wrote: > That is the behavior for the main supported platforms. We had a concern that it might not be the case on other platforms. Hence, the word "may". yes, appreciate that sentiment, but "may" is a bit flippy floppy - a definitely maybe. A stronger emphasis that this OS dependent, such as the inclusion of an perhaps an apiNote to that effect, would be useful and informative ------------- PR Comment: https://git.openjdk.org/jdk/pull/16635#issuecomment-1830335647 From duke at openjdk.org Tue Nov 28 17:56:08 2023 From: duke at openjdk.org (Anthony Vanelverdinghe) Date: Tue, 28 Nov 2023 17:56:08 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions [v2] In-Reply-To: References: Message-ID: On Mon, 27 Nov 2023 23:39:57 GMT, Roger Riggs wrote: > Filenames that start with "." introduce a number of special cases. > "." and ".." are not just conventions but have strong semantics. [...] Surely everyone agrees that neither `.` nor `..` have an extension. > For other files that begin with ".", it is conventional to name the file using the entire string. It is conventional *on OSes where a leading dot marks a file/directory as hidden*. On such OSes, "hidden marker" has precedence over "file extension separator" and so `.mp3` denotes a hidden file with base `mp3` and without extension. However, a leading dot has no special meaning on Windows (and in the relevant APIs in both Guava and Apache Commons IO). So on Windows, `.mp3` denotes an audio file with an empty base and an extension of `.mp3` (or `mp3`, if you will). What I'm advocating, is that `getExtension()` must support both conventions, either implicitly (by returning an OS- and/or file system-specific result) or explicitly (by allowing the developer to specify the desired convention somehow). ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1830388309 From mgronlun at openjdk.org Tue Nov 28 18:24:10 2023 From: mgronlun at openjdk.org (Markus =?UTF-8?B?R3LDtm5sdW5k?=) Date: Tue, 28 Nov 2023 18:24:10 GMT Subject: RFR: 8310994: Add JFR event for selection operations In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 16:22:55 GMT, Tim Prinzing wrote: > Added mirror event with static methods: jdk.internal.event.SelectionEvent that provides the duration of select calls and the count of how many keys are available. > > Emit the event from SelectorImpl::lockAndDoSelect > > Test at jdk.jfr.event.io.TestSelectionEvents It would be good to integrate this quite soon because it would release the reliance on ASM and let us only use the new ClassFile API. Now, we need to load both implementations, leading to startup regression. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16710#issuecomment-1830431950 From msheppar at openjdk.org Tue Nov 28 18:53:06 2023 From: msheppar at openjdk.org (Mark Sheppard) Date: Tue, 28 Nov 2023 18:53:06 GMT Subject: RFR: 8319417: (dc) DatagramChannel.connect undocumented behavior [v2] In-Reply-To: <1BIOnqHASe_6tnf9L4dVkneBmk520pu_XpUTbhEfV2M=.d9c6e525-b914-4dd7-9a2e-04d5f0088f35@github.com> References: <1BIOnqHASe_6tnf9L4dVkneBmk520pu_XpUTbhEfV2M=.d9c6e525-b914-4dd7-9a2e-04d5f0088f35@github.com> Message-ID: <3Au4yF556eRad7ST0hLfQZjNmxTqtz_Hf9g15p4gHPw=.e3fad13b-0e73-4431-b741-b5c9418c090b@github.com> On Tue, 28 Nov 2023 13:48:35 GMT, Alan Bateman wrote: > > Also, the return for the case of an unbound DatagramSocket or DatagramChannel is not specified. > > DatagramChannel::getLocalAddress is specified to return null when channel's socket is not bound. DatagramSocket::getLocalSocketAddress also specifies that it returns null when not bound. But maybe you mean something else? Yes, indeed I did mean something else, I seemed to have lost the tail end of my sentence (the punchline, so to say) It should have read something like: Also, the return for the case of an unbound DatagramSocket or DatagramChannel is not specified consistently, and provide conflicting return specifications. DatagramChannel return spec Returns: The SocketAddress that the socket is bound to, or the SocketAddress representing the loopback address if denied by the security manager, or null if the channel's socket is not bound. DatagramSocket return spec Returns: the local address to which the socket is bound, null if the socket is closed, or an InetAddress representing [wildcard](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/net/InetAddress.html#isAnyLocalAddress()) address if either the socket is not bound, or the security manager checkConnect method does not allow the operation ------------- PR Comment: https://git.openjdk.org/jdk/pull/16635#issuecomment-1830479494 From alanb at openjdk.org Tue Nov 28 19:02:08 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 28 Nov 2023 19:02:08 GMT Subject: RFR: 8319417: (dc) DatagramChannel.connect undocumented behavior [v2] In-Reply-To: <3Au4yF556eRad7ST0hLfQZjNmxTqtz_Hf9g15p4gHPw=.e3fad13b-0e73-4431-b741-b5c9418c090b@github.com> References: <1BIOnqHASe_6tnf9L4dVkneBmk520pu_XpUTbhEfV2M=.d9c6e525-b914-4dd7-9a2e-04d5f0088f35@github.com> <3Au4yF556eRad7ST0hLfQZjNmxTqtz_Hf9g15p4gHPw=.e3fad13b-0e73-4431-b741-b5c9418c090b@github.com> Message-ID: On Tue, 28 Nov 2023 18:49:53 GMT, Mark Sheppard wrote: > Returns: the local address to which the socket is bound, null if the socket is closed, or an InetAddress representing [wildcard](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/net/InetAddress.html#isAnyLocalAddress()) address if either the socket is not bound, or the security manager checkConnect method does not allow the operation DatagramSocket.getLocalAddress dates from JDK 1.1 and its odd behavior couldn't be changed for compatibility reasons. The newer DatagramSocket.getLocalSocketAddress is specified to return null when not bound. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16635#issuecomment-1830492998 From naoto at openjdk.org Tue Nov 28 19:18:13 2023 From: naoto at openjdk.org (Naoto Sato) Date: Tue, 28 Nov 2023 19:18:13 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer Message-ID: It is best practice to zero out the underlying buffer after use. ------------- Commit messages: - copyright year - fill zero to position - initial commit Changes: https://git.openjdk.org/jdk/pull/16861/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16861&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320798 Stats: 31 lines in 2 files changed: 30 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16861.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16861/head:pull/16861 PR: https://git.openjdk.org/jdk/pull/16861 From bpb at openjdk.org Tue Nov 28 19:26:08 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 28 Nov 2023 19:26:08 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 19:12:50 GMT, Naoto Sato wrote: > It is best practice to zero out the underlying buffer after use. src/java.base/share/classes/sun/nio/cs/StreamDecoder.java line 294: > 292: private void lockedFillZeroToPosition() throws IOException { > 293: ensureOpen(); > 294: Arrays.fill(bb.array(), 0, bb.arrayOffset() + bb.position(), (byte)0); Should the `from` parameter be zero or `bb.arrayOffset()`? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16861#discussion_r1408281785 From naoto at openjdk.org Tue Nov 28 19:30:34 2023 From: naoto at openjdk.org (Naoto Sato) Date: Tue, 28 Nov 2023 19:30:34 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v2] In-Reply-To: References: Message-ID: > It is best practice to zero out the underlying buffer after use. Naoto Sato has updated the pull request incrementally with one additional commit since the last revision: corrected fromIndex ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16861/files - new: https://git.openjdk.org/jdk/pull/16861/files/3d2351b5..90c4dece Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16861&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16861&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16861.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16861/head:pull/16861 PR: https://git.openjdk.org/jdk/pull/16861 From naoto at openjdk.org Tue Nov 28 19:30:36 2023 From: naoto at openjdk.org (Naoto Sato) Date: Tue, 28 Nov 2023 19:30:36 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v2] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 19:23:45 GMT, Brian Burkhalter wrote: >> Naoto Sato has updated the pull request incrementally with one additional commit since the last revision: >> >> corrected fromIndex > > src/java.base/share/classes/sun/nio/cs/StreamDecoder.java line 294: > >> 292: private void lockedFillZeroToPosition() throws IOException { >> 293: ensureOpen(); >> 294: Arrays.fill(bb.array(), 0, bb.arrayOffset() + bb.position(), (byte)0); > > Should the `from` parameter be zero or `bb.arrayOffset()`? Right. Fixed. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16861#discussion_r1408285735 From bpb at openjdk.org Tue Nov 28 19:36:06 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 28 Nov 2023 19:36:06 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v2] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 19:30:34 GMT, Naoto Sato wrote: >> It is best practice to zero out the underlying buffer after use. > > Naoto Sato has updated the pull request incrementally with one additional commit since the last revision: > > corrected fromIndex Looks good now. ------------- Marked as reviewed by bpb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16861#pullrequestreview-1753736289 From lancea at openjdk.org Tue Nov 28 19:47:08 2023 From: lancea at openjdk.org (Lance Andersen) Date: Tue, 28 Nov 2023 19:47:08 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v2] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 19:30:34 GMT, Naoto Sato wrote: >> It is best practice to zero out the underlying buffer after use. > > Naoto Sato has updated the pull request incrementally with one additional commit since the last revision: > > corrected fromIndex Marked as reviewed by lancea (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16861#pullrequestreview-1753782100 From msheppar at openjdk.org Tue Nov 28 20:09:07 2023 From: msheppar at openjdk.org (Mark Sheppard) Date: Tue, 28 Nov 2023 20:09:07 GMT Subject: RFR: 8319417: (dc) DatagramChannel.connect undocumented behavior [v2] In-Reply-To: References: <1BIOnqHASe_6tnf9L4dVkneBmk520pu_XpUTbhEfV2M=.d9c6e525-b914-4dd7-9a2e-04d5f0088f35@github.com> <3Au4yF556eRad7ST0hLfQZjNmxTqtz_Hf9g15p4gHPw=.e3fad13b-0e73-4431-b741-b5c9418c090b@github.com> Message-ID: On Tue, 28 Nov 2023 18:59:15 GMT, Alan Bateman wrote: > > Returns: the local address to which the socket is bound, null if the socket is closed, or an InetAddress representing [wildcard](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/net/InetAddress.html#isAnyLocalAddress()) address if either the socket is not bound, or the security manager checkConnect method does not allow the operation > > DatagramSocket.getLocalAddress dates from JDK 1.1 and its odd behavior couldn't be changed for compatibility reasons. The newer DatagramSocket.getLocalSocketAddress is specified to return null when not bound. Hmm! sound like a bit of a fudge, and doesn't really make sense, as DatagramSocket and DatagramChannel essentially delegate to the same underlying implementation, which in turn delgates to the same underlying native implementation for which the OS will have a strong influence on the semantics offered at the higher level. Thus, in the case of an unbounded lower level socket, upon which a send (or connect) has been invoked, a subsequent getsockname would return whatever the OS has implicitly bound to the socket in terms of address and port. Thus, a similar scenario would be expected at the Java API level, and should result in the getLocalAddress method exhibiting the same semantics, returning the same InetAddress for all specified conditions, irrespective of which API is called. If both APIs delegate to the same underlying implementation, then it is reasonable to expect that they both exhibit a consistent samantics in their APIs. This is especially true as DatagramChannel exposes an associated DatagramSocket, which based your statement will allow a duality of behaviour to exist. Regardless of compatibility, this is reasonably conceived as broken behaviour. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16635#issuecomment-1830638977 From lancea at openjdk.org Tue Nov 28 20:09:08 2023 From: lancea at openjdk.org (Lance Andersen) Date: Tue, 28 Nov 2023 20:09:08 GMT Subject: RFR: 8316141: Improve CEN header validation checking In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 20:52:08 GMT, Eirik Bjorsnos wrote: > > Regarding you comment about checking whether or not to check if the combined length of the CEN header + name length + comment length + extra length > 65K bytes, I chose to add this given the strong wording given this is a really old spec. That being said, I do not object to removing the validation if that is the overall preference. > > I can't claim to have a particularly strong opinion on this, the following is mostly me thinking aloud: > > * Given Hyrum's Law, it is conceivable that someone is currently using the extra or comment fields to attach up to 65535+65535 bytes of metadata for entires. The proposed validation will break such schemes. Does Oracle have a ZIP file corpus which could be used to identify files currently exceeding the combined length clause, just to get a sense of how rare or common this is? > * The actual benefits of adding this validation after all these years is not quite clear to me. I don't see how this improves security, robustness, compatibility, maintainability or other ilities (apart from strictly-following-the-spec-ility :-) > * I created a ZIP file with an entry with an extra field of the maximal expressable length of 0xFFFF. Info-ZIP's `unzip` command on MacOS did not output any warning or error when processing this file. Yes we have a corpus search available and have exercised this patch (along with your ZipInputStream patch) without any regressions. Given where we are in the JDK 22 cycle, going to hold off on finalizing the PR until we fork for JDK 23 and look to move this forward early on allowing for additional time to bake ------------- PR Comment: https://git.openjdk.org/jdk/pull/16570#issuecomment-1830639162 From joehw at openjdk.org Tue Nov 28 20:23:06 2023 From: joehw at openjdk.org (Joe Wang) Date: Tue, 28 Nov 2023 20:23:06 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v2] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 19:30:34 GMT, Naoto Sato wrote: >> It is best practice to zero out the underlying buffer after use. > > Naoto Sato has updated the pull request incrementally with one additional commit since the last revision: > > corrected fromIndex Marked as reviewed by joehw (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16861#pullrequestreview-1753858796 From alanb at openjdk.org Tue Nov 28 20:28:09 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 28 Nov 2023 20:28:09 GMT Subject: RFR: 8319417: (dc) DatagramChannel.connect undocumented behavior [v2] In-Reply-To: References: <1BIOnqHASe_6tnf9L4dVkneBmk520pu_XpUTbhEfV2M=.d9c6e525-b914-4dd7-9a2e-04d5f0088f35@github.com> <3Au4yF556eRad7ST0hLfQZjNmxTqtz_Hf9g15p4gHPw=.e3fad13b-0e73-4431-b741-b5c9418c090b@github.com> Message-ID: On Tue, 28 Nov 2023 20:06:15 GMT, Mark Sheppard wrote: > Hmm! sound like a bit of a fudge, and doesn't really make sense, as DatagramSocket and DatagramChannel essentially delegate to the same underlying implementation, which in turn delgates to the same underlying native implementation for which the OS will have a strong influence on the semantics offered at the higher level. Thus, in the case of an unbounded lower level socket, upon which a send (or connect) has been invoked, a subsequent getsockname would return whatever the OS has implicitly bound to the socket in terms of address and port. Thus, a similar scenario would be expected at the Java API level, and should result in the getLocalAddress method exhibiting the same semantics, returning the same InetAddress for all specified conditions, irrespective of which API is called. If both APIs delegate to the same underlying implementation, then it is reasonable to expect that they both exhibit a consistent samantics in their APIs. This is especially true as DatagramChannel exposes an associated DatagramSocket, which based your statement will allow a duality of behaviour to exist. Regardless of compatibility, this is reasonably conceived as broken behaviour. DatagramSocket::getLocalAddress dates from JDK 1.0 and originally specified/implemented to return an InetAddress. The support for unbound sockets was added in JDK 1.4 (2004) and it would have been an incompatible change to change this method to return null, hence the current spec. I agree it's an odd API for this case but changing it now after all this time would be an incompatible change. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16635#issuecomment-1830668059 From alanb at openjdk.org Tue Nov 28 20:33:06 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 28 Nov 2023 20:33:06 GMT Subject: RFR: 8310994: Add JFR event for selection operations In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 18:21:33 GMT, Markus Gr?nlund wrote: > It would be good to integrate this quite soon because it would release the reliance on ASM and let us only use the new ClassFile API. Now, we need to load both implementations, leading to startup regression. This is a new event so there isn't existing instrumentation to remove. I think this will need changes before it can be integrated as there are naming issues and needs the timeout to at least be able to distinguish immediately selection operations from others. There are quite a few use-cases to work through on how it might be used too. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16710#issuecomment-1830675271 From mgronlun at openjdk.org Tue Nov 28 20:39:08 2023 From: mgronlun at openjdk.org (Markus =?UTF-8?B?R3LDtm5sdW5k?=) Date: Tue, 28 Nov 2023 20:39:08 GMT Subject: RFR: 8310994: Add JFR event for selection operations In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 20:30:52 GMT, Alan Bateman wrote: > > It would be good to integrate this quite soon because it would release the reliance on ASM and let us only use the new ClassFile API. Now, we need to load both implementations, leading to startup regression. > > This is a new event so there isn't existing instrumentation to remove. I think this will need changes before it can be integrated as there are naming issues and needs the timeout to at least be able to distinguish immediately selection operations from others. There are quite a few use-cases to work through on how it might be used too. Isn't this a replacement for the existing instrumentation? If these are new events I am sorry. What is keeping us from removing ASM in that case? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16710#issuecomment-1830684595 From alanb at openjdk.org Tue Nov 28 20:41:08 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 28 Nov 2023 20:41:08 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v2] In-Reply-To: References: Message-ID: <8iNMIOZuwP6-W26OQIuvoTWRb6wNp9FBPBZaxGuAIqE=.b36c2264-ab0a-4c40-adec-e9b1c1d6dfc3@github.com> On Tue, 28 Nov 2023 19:30:34 GMT, Naoto Sato wrote: >> It is best practice to zero out the underlying buffer after use. > > Naoto Sato has updated the pull request incrementally with one additional commit since the last revision: > > corrected fromIndex Marked as reviewed by alanb (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16861#pullrequestreview-1753892317 From alanb at openjdk.org Tue Nov 28 20:41:10 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 28 Nov 2023 20:41:10 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v2] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 19:27:45 GMT, Naoto Sato wrote: >> src/java.base/share/classes/sun/nio/cs/StreamDecoder.java line 294: >> >>> 292: private void lockedFillZeroToPosition() throws IOException { >>> 293: ensureOpen(); >>> 294: Arrays.fill(bb.array(), 0, bb.arrayOffset() + bb.position(), (byte)0); >> >> Should the `from` parameter be zero or `bb.arrayOffset()`? > > Right. Fixed. The buffer is allocated in StreamDecoder so I assume the array offset is 0 anyway, so I think they will be the same. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16861#discussion_r1408394249 From alanb at openjdk.org Tue Nov 28 20:44:09 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 28 Nov 2023 20:44:09 GMT Subject: RFR: 8316141: Improve CEN header validation checking In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 20:06:25 GMT, Lance Andersen wrote: > Given where we are in the JDK 22 cycle, going to hold off on finalizing the PR until we fork for JDK 23 and look to move this forward early on allowing for additional time to bake Tightening validation always comes with risk. Doing it early in JDK 23 to allow time for course correction if needed seems a good plan. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16570#issuecomment-1830700871 From alanb at openjdk.org Tue Nov 28 20:46:10 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 28 Nov 2023 20:46:10 GMT Subject: RFR: 8310994: Add JFR event for selection operations In-Reply-To: References: Message-ID: <9uNVx-mdtv6JcAZwi55bpD5gyV-EnPUPus3AIf0Harc=.e97702d3-ee3a-48d9-b2a8-f93c7d7fcf87@github.com> On Tue, 28 Nov 2023 20:34:34 GMT, Markus Gr?nlund wrote: > Isn't this a replacement for the existing instrumentation? If these are new events I am sorry. It's a new event. > What is keeping us from removing ASM in that case? I think file I/O, there is still instrumentation used for FileInputStream, FileOutputStream, RandomAccessFile and nio.FileChannel. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16710#issuecomment-1830703533 From duke at openjdk.org Tue Nov 28 20:55:06 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Tue, 28 Nov 2023 20:55:06 GMT Subject: RFR: 8316141: Improve CEN header validation checking In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 20:41:21 GMT, Alan Bateman wrote: > Doing it early in JDK 23 to allow time for course correction if needed seems a good plan. Another benefit is that if we should decide to validate LOC headers similarly in `ZipInputStream`, delaying until 23 will allow us to introduce these very similar changes in the same release. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16570#issuecomment-1830715882 From tprinzing at openjdk.org Tue Nov 28 22:15:06 2023 From: tprinzing at openjdk.org (Tim Prinzing) Date: Tue, 28 Nov 2023 22:15:06 GMT Subject: RFR: 8310994: Add JFR event for selection operations In-Reply-To: References: Message-ID: <1DFRyaMd3eg3u2dA4DoIMaCUFuL_3jewY9hXusCtE9o=.7da770e9-7d1a-451c-b759-abf9fb2ed5cc@github.com> On Fri, 17 Nov 2023 16:22:55 GMT, Tim Prinzing wrote: > Added mirror event with static methods: jdk.internal.event.SelectionEvent that provides the duration of select calls and the count of how many keys are available. > > Emit the event from SelectorImpl::lockAndDoSelect > > Test at jdk.jfr.event.io.TestSelectionEvents Yes, in JDKEvents this is the list: private static final Class[] instrumentationClasses = new Class[] { FileInputStreamInstrumentor.class, FileOutputStreamInstrumentor.class, RandomAccessFileInstrumentor.class, FileChannelImplInstrumentor.class }; On Tue, Nov 28, 2023 at 2:43?PM Alan Bateman ***@***.***> wrote: > Isn't this a replacement for the existing instrumentation? If these are > new events I am sorry. > > It's a new event. > > What is keeping us from removing ASM in that case? > > I think file I/O, there is still instrumentation used for FileInputStream, > FileOutputStream, RandomAccessFile and nio.FileChannel. > > ? > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > ------------- PR Comment: https://git.openjdk.org/jdk/pull/16710#issuecomment-1830842223 From michaelm at openjdk.org Tue Nov 28 22:22:08 2023 From: michaelm at openjdk.org (Michael McMahon) Date: Tue, 28 Nov 2023 22:22:08 GMT Subject: RFR: 8319417: (dc) DatagramChannel.connect undocumented behavior [v2] In-Reply-To: References: <2h-LLES9Y3J_PP5vnVcpvXSx76775uiIoZb0e_Y1IT4=.0b6e0f1f-5c7d-440d-bbe0-519682646fe1@github.com> Message-ID: On Tue, 28 Nov 2023 17:20:05 GMT, Mark Sheppard wrote: > > That is the behavior for the main supported platforms. We had a concern that it might not be the case on other platforms. Hence, the word "may". > > yes, appreciate that sentiment, but "may" is a bit flippy floppy - a definitely maybe. A stronger emphasis that this is OS dependent, such as the inclusion of an perhaps an apiNote to that effect, would be useful and informative I think it's the same effect. I'd prefer not to complicate the text. We don't actually know the exact conditions where it might be supported and when not. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16635#issuecomment-1830849690 From michaelm at openjdk.org Wed Nov 29 14:06:20 2023 From: michaelm at openjdk.org (Michael McMahon) Date: Wed, 29 Nov 2023 14:06:20 GMT Subject: Integrated: 8319417: (dc) DatagramChannel.connect undocumented behavior In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 17:28:29 GMT, Michael McMahon wrote: > Hi, > > This is a small doc change for DatagramChannel.getLocalAddress() and the equivalent methods on DatagramSocket which documents some existing (but not currently documented) behavior. > > Thanks, > Michael This pull request has now been integrated. Changeset: a2c5f1fc Author: Michael McMahon URL: https://git.openjdk.org/jdk/commit/a2c5f1fc914ef5c28d044b75598f895cf6097138 Stats: 19 lines in 2 files changed: 19 ins; 0 del; 0 mod 8319417: (dc) DatagramChannel.connect undocumented behavior Reviewed-by: alanb ------------- PR: https://git.openjdk.org/jdk/pull/16635 From jvernee at openjdk.org Wed Nov 29 17:33:18 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Wed, 29 Nov 2023 17:33:18 GMT Subject: RFR: 8318966: Some methods make promises about Java array element alignment that are too strong In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 22:46:03 GMT, Jorn Vernee wrote: > See the JBS issue for an extended problem description. > > This patch changes the specification and implementation of `MethodHandles::byteArrayViewVarHandle`, `MethodHandles::byteBufferViewVarHandle`, `ByteBuffer::alignedSlice`, and `ByteBuffer::alignmentOffset` to weaken the guarantees they make about the alignment of Java array elements, in order to bring them in line with the guarantees made by an arbitrary JVM implementation (which makes no guarantees about array element alignment beyond them being aligned to their natural alignment). > > - `MethodHandles::byteArrayViewVarHandle`: we can not guarantee any alignment for the accesses. Which means we can only reliably support plain get and set access modes. The javadoc text explaining which other access modes are supported, or how to compute aligned offsets into the array is dropped, as it is not guaranteed to be correct on all JVM implementations. The implementation of the returned VarHandle is changed to throw an `UnsupportedOperationException` for the unsupported access modes, as mandated by the spec of `VarHandle` [1]. > > - `MethodHandles::byteBufferViewVarHandle`: the implementation/spec is incorrect when accessing a heap buffer (wrapping a byte[]), for the same reasons as `byteArrayViewVarHandle`. The spec is changed to specify that when accessing a _heap buffer_, only plain get and set access modes are supported. The implementation of the returned var handle is changed to throw an `IllegalStateException` when an access is attempted on a heap buffer using an access mode other than plain get or set. Note that we don't throw an outright `UnsupportedOperationException` for this case, since whether the access modes are supported depends on the byte buffer instance being used. > > - `ByteBuffer::alignedSlice` and `ByteBuffer::alignmentOffset`: The former method depends directly on the latter for all its alignment computations. We change the implementation of the latter method to throw an `UnsupportedOperationException` for all unit sizes greater than 1, when the buffer is non-direct. This change is largely covered by the existing specification: > > > * @throws UnsupportedOperationException > * If the native platform does not guarantee stable alignment offset > * values for the given unit size when managing the memory regions > * of buffers of the same kind as this buffer (direct or > * non-direct). For example, if garbage collection would result > * in the mo... Anything else needed to move this forward? I think I've addressed all the review comments made so far. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16681#issuecomment-1828491878 From duke at openjdk.org Thu Nov 30 10:33:12 2023 From: duke at openjdk.org (Anthony Vanelverdinghe) Date: Thu, 30 Nov 2023 10:33:12 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions [v2] In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 23:30:45 GMT, Brian Burkhalter wrote: >> Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - 8298318: Correct type in path.getExtension spec > - Merge > - 8298318: (fs) APIs for handling filename extensions The more I think about it, the more I'm convinced that the concept of a filename is sufficiently complex to warrant its own type. Moreover, I believe extension-specific APIs would be out of place in `Path`, as they apply to a single name, whereas `Path` is all about a sequence of names. While an additional type would increase the API surface, the existing `Path::getFileName` can be repurposed (if I'm correct that the proposed change would be a compatible one). So I'd like to propose: * introducing an interface `Filename` which extends `Path` and `CharSequence` * changing `Path::getFileName` to return an instance of `Filename` where `Filename` would look something like below and would allow code such as: var foo = Path.of(".foo.tar.gz").getFileName(); if(foo.extension().equals(".gz") && foo.base().extension().equals(".tar")) { var name = foo.base().base(); assert "foo".contentEquals(name); } /** * Represents a filename. * * Invariant (pseudocode): hidden() + base() + extension() = toString() */ interface Filename extends Path, CharSequence { /** * An enum that allows overriding how the filename must be interpreted. */ enum DotOption { HIDDEN_MARKER_AS_LEADING_DOT, NO_HIDDEN_MARKER } /** * {@return the hidden marker} * * If this instance represents a Filename where a leading dot serves as a "hidden" marker * or DotOption.HIDDEN_MARKER_AS_LEADING_DOT is specified, * returns `Optional.of(...)` * If this instance represents a Filename where a leading dot has no special meaning * or DotOption.NO_HIDDEN_MARKER is specified, * returns `Optional.empty()` */ Optional hidden(DotOption... options); /** * {@return the filename without the hidden marker and the file extension} */ Filename base(); /** * {@return the file extension including the dot} * * The filenames `.` and `..` don't have an extension. * * For filenames with a single, leading dot: * If this instance represents a Filename where a leading dot serves as a "hidden" marker * or DotOption.HIDDEN_MARKER_AS_LEADING_DOT is specified, * returns the empty String * If this instance represents a Filename where a leading dot has no special meaning * or DotOption.NO_HIDDEN_MARKER is specified, * returns the whole filename * * For any other filenames, the extension is the substring starting at the last dot. */ String extension(DotOption... options); // TODO: consider additional methods such as withExtension, withoutExtension, ... } ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1833489866 From jpai at openjdk.org Thu Nov 30 11:01:07 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Thu, 30 Nov 2023 11:01:07 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v2] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 19:30:34 GMT, Naoto Sato wrote: >> It is best practice to zero out the underlying buffer after use. > > Naoto Sato has updated the pull request incrementally with one additional commit since the last revision: > > corrected fromIndex src/java.base/share/classes/sun/nio/cs/StreamDecoder.java line 293: > 291: > 292: private void lockedFillZeroToPosition() throws IOException { > 293: ensureOpen(); Hello Naoto, is this open check needed? Given that we are just zeroing out already read content, would it thus be OK to clear that content to zero even after this `Reader` instance has been closed? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16861#discussion_r1410504144 From jpai at openjdk.org Thu Nov 30 11:13:10 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Thu, 30 Nov 2023 11:13:10 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v2] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 19:30:34 GMT, Naoto Sato wrote: >> It is best practice to zero out the underlying buffer after use. > > Naoto Sato has updated the pull request incrementally with one additional commit since the last revision: > > corrected fromIndex Given that the default `JdkConsole` provider implementation is the one that uses jline (in `jdk.internal.le` module), should we also do something to zero out this data in its implementation too. A brief glance suggests that it too uses an internal implementation class called `jdk.internal.org.jline.reader.impl.BufferImpl`. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16861#issuecomment-1833550541 From alanb at openjdk.org Thu Nov 30 11:59:05 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 30 Nov 2023 11:59:05 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v2] In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 11:10:44 GMT, Jaikiran Pai wrote: > Given that the default `JdkConsole` provider implementation is the one that uses jline (in `jdk.internal.le` module), should we also do something to zero out this data in its implementation too. A brief glance suggests that it too uses an internal implementation class called `jdk.internal.org.jline.reader.impl.BufferImpl`. This PR is the console implementation in java.base, there will be other changes for jline. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16861#issuecomment-1833622797 From naoto at openjdk.org Thu Nov 30 17:39:30 2023 From: naoto at openjdk.org (Naoto Sato) Date: Thu, 30 Nov 2023 17:39:30 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v3] In-Reply-To: References: Message-ID: <8mADwh5tk-5DhucpbLxUAiPDOgxSkJru210ndcjXWaU=.f1dbb46e-974a-4975-8b08-cb8d34bf92cc@github.com> > It is best practice to zero out the underlying buffer after use. Naoto Sato has updated the pull request incrementally with one additional commit since the last revision: remove ensureOpen() ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16861/files - new: https://git.openjdk.org/jdk/pull/16861/files/90c4dece..6d2d6368 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16861&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16861&range=01-02 Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16861.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16861/head:pull/16861 PR: https://git.openjdk.org/jdk/pull/16861 From naoto at openjdk.org Thu Nov 30 17:39:33 2023 From: naoto at openjdk.org (Naoto Sato) Date: Thu, 30 Nov 2023 17:39:33 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v2] In-Reply-To: References: Message-ID: <1fhvC7QU7wM7B6osoSaUQHFV1m4TcfTi884lMOjf_nc=.684114bf-09f1-4285-88b5-2b6fb5a2335f@github.com> On Thu, 30 Nov 2023 10:58:13 GMT, Jaikiran Pai wrote: >> Naoto Sato has updated the pull request incrementally with one additional commit since the last revision: >> >> corrected fromIndex > > src/java.base/share/classes/sun/nio/cs/StreamDecoder.java line 293: > >> 291: >> 292: private void lockedFillZeroToPosition() throws IOException { >> 293: ensureOpen(); > > Hello Naoto, is this open check needed? Given that we are just zeroing out already read content, would it thus be OK to clear that content to zero even after this `Reader` instance has been closed? Thanks, Jai. Removed it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16861#discussion_r1411051514 From naoto at openjdk.org Thu Nov 30 17:42:06 2023 From: naoto at openjdk.org (Naoto Sato) Date: Thu, 30 Nov 2023 17:42:06 GMT Subject: RFR: 8320798: Console read line with zero out should zero out underlying buffer [v2] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 19:30:34 GMT, Naoto Sato wrote: >> It is best practice to zero out the underlying buffer after use. > > Naoto Sato has updated the pull request incrementally with one additional commit since the last revision: > > corrected fromIndex Created an equivalent issue for addressing JLine: https://bugs.openjdk.org/browse/JDK-8321131 ------------- PR Comment: https://git.openjdk.org/jdk/pull/16861#issuecomment-1834254962 From rriggs at openjdk.org Thu Nov 30 20:32:13 2023 From: rriggs at openjdk.org (Roger Riggs) Date: Thu, 30 Nov 2023 20:32:13 GMT Subject: RFR: 8298318: (fs) APIs for handling filename extensions [v2] In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 23:30:45 GMT, Brian Burkhalter wrote: >> Add to `java.nio.file.Path` a method `getExtension` to retrieve the `Path`'s extension, and companion methods `removeExtension` and `addExtension`. > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - 8298318: Correct type in path.getExtension spec > - Merge > - 8298318: (fs) APIs for handling filename extensions I think you're making things too complicated. The more I think of it, I think it needs to be simpler. Developers that are working with dot-files, know it and won't be looking for file extensions at all. They also know whether the files they are working with have extensions and know how to do it themselves but like a simple function to replace or remove the extension. The API will be practical and easy to use even if it ignores the leading dots (as do Guava and Apache Commons). It will also be consistent with other APIs in OpenJDK that look at extensions including Files.probeContentType. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16226#issuecomment-1834513172