From bpb at openjdk.org Wed Mar 1 00:35:38 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 1 Mar 2023 00:35:38 GMT Subject: RFR: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService Message-ID: Modify `sun.nio.fs.PollingWatchService` to ignore any polling sensitivity modifiers as done by the other `WatchService`s. ------------- Commit messages: - 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService Changes: https://git.openjdk.org/jdk/pull/12795/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12795&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8303413 Stats: 22 lines in 1 file changed: 1 ins; 8 del; 13 mod Patch: https://git.openjdk.org/jdk/pull/12795.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12795/head:pull/12795 PR: https://git.openjdk.org/jdk/pull/12795 From bpb at openjdk.org Wed Mar 1 01:40:15 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 1 Mar 2023 01:40:15 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v18] In-Reply-To: References: Message-ID: On Sat, 25 Feb 2023 13:29:52 GMT, Markus KARG wrote: >> This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > updated copyright In the test classes `TransferTo` and `TransferTo2`, the methods `testNullPointerException` and `testStreamContents` are identical aside from expecting different `DataProvider`s. The contents of the methods could be abstracted out to common methods called by different `Test`s which expect different `DataProvider`s. The test classes `TransferTo_2GB_transferFrom` and `TransferTo_2GB_transferTo` comprise roughly 120 lines of code each, but a diff of their contents reveals little in the way of substantive differences aside from jtreg tags, comments, and names: 96,99c95,97 < // performing actual transfer, effectively by multiple invocations of < // FileChannel.transferFrom(ReadableByteChannel) < try (InputStream inputStream = Channels.newInputStream(Channels.newChannel( < new BufferedInputStream(Files.newInputStream(sourceFile)))); --- > // perform actual transfer, effectively by multiple invocations > // of FileChannel.transferTo(WritableByteChannel) > try (InputStream inputStream = Channels.newInputStream(FileChannel.open(sourceFile)); Perhaps something along these lines might work: * rename `TransferToBase` to for example to `TransferBase` or `TransferFromToBase`; * rename `TransferTo2` to `TransferFrom`; * rename `TransferTo_2GB_transferX` classes to `TransferX_2GB`; * refactor common code to `TransferBase` along the lines described above; * refactor child classes as needed to use the common code. Assuming the foregoing makes sense, there would remain a total of four tests. It might be possible also to merge `TransferFrom` and `TransferTo` into a single test, but I think the two tests `*_2GB*` should remain separate as they likely each have a sufficiently long run time. ------------- PR: https://git.openjdk.org/jdk/pull/6711 From bpb at openjdk.org Wed Mar 1 01:42:42 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 1 Mar 2023 01:42:42 GMT Subject: RFR: 8303260: (fc) FileChannel::transferFrom should support position > size() Message-ID: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> Enhance `FileChannel::transferFrom` to extend the channel's file if `position > size()`. This is consistent with `FileChannel::write(ByteBuffer, long)`. ------------- Commit messages: - 8303260: (fc) FileChannel::transferFrom should support position > size() Changes: https://git.openjdk.org/jdk/pull/12797/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12797&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8303260 Stats: 150 lines in 3 files changed: 138 ins; 2 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/12797.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12797/head:pull/12797 PR: https://git.openjdk.org/jdk/pull/12797 From alanb at openjdk.org Wed Mar 1 08:13:03 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 1 Mar 2023 08:13:03 GMT Subject: RFR: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService In-Reply-To: References: Message-ID: <25cfs6RLymXR4076xcEJ8uvaY7tA2XtXLM38kQC8Mgk=.74fc676d-6a0b-4c0a-b207-de31b5ba674c@github.com> On Wed, 1 Mar 2023 00:27:46 GMT, Brian Burkhalter wrote: > Modify `sun.nio.fs.PollingWatchService` to ignore any polling sensitivity modifiers as done by the other `WatchService`s. Test change look okay but it will need a CSR and RN to track the behavior change. Also test/jdk/java/nio/file/WatchService/SensitivityModifier.java will need attention - maybe it should just be replaced with a sanity test that ensures that the register method does not throw when called with one of these modifiers. ------------- PR: https://git.openjdk.org/jdk/pull/12795 From alanb at openjdk.org Wed Mar 1 15:18:29 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 1 Mar 2023 15:18:29 GMT Subject: RFR: 8303260: (fc) FileChannel::transferFrom should support position > size() In-Reply-To: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> References: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> Message-ID: On Wed, 1 Mar 2023 01:34:33 GMT, Brian Burkhalter wrote: > Enhance `FileChannel::transferFrom` to extend the channel's file if `position > size()`. This is consistent with `FileChannel::write(ByteBuffer, long)`. The update to the API docs looks okay, the implementation change looks fine. I just wondering if there is anything that we might be missing to explain the restriction in the original API. ------------- PR: https://git.openjdk.org/jdk/pull/12797 From bpb at openjdk.org Wed Mar 1 16:34:04 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 1 Mar 2023 16:34:04 GMT Subject: RFR: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService In-Reply-To: <25cfs6RLymXR4076xcEJ8uvaY7tA2XtXLM38kQC8Mgk=.74fc676d-6a0b-4c0a-b207-de31b5ba674c@github.com> References: <25cfs6RLymXR4076xcEJ8uvaY7tA2XtXLM38kQC8Mgk=.74fc676d-6a0b-4c0a-b207-de31b5ba674c@github.com> Message-ID: <84b0ZEODps1G8na6xS8ksSZ60njgYdWEDYHkyTfF30E=.5a7291d5-0b2e-433a-b0f7-7552369c0670@github.com> On Wed, 1 Mar 2023 08:09:52 GMT, Alan Bateman wrote: > Also test/jdk/java/nio/file/WatchService/SensitivityModifier.java will need attention I ran this test already and it still succeeds. ------------- PR: https://git.openjdk.org/jdk/pull/12795 From alanb at openjdk.org Wed Mar 1 16:44:04 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 1 Mar 2023 16:44:04 GMT Subject: RFR: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService In-Reply-To: <84b0ZEODps1G8na6xS8ksSZ60njgYdWEDYHkyTfF30E=.5a7291d5-0b2e-433a-b0f7-7552369c0670@github.com> References: <25cfs6RLymXR4076xcEJ8uvaY7tA2XtXLM38kQC8Mgk=.74fc676d-6a0b-4c0a-b207-de31b5ba674c@github.com> <84b0ZEODps1G8na6xS8ksSZ60njgYdWEDYHkyTfF30E=.5a7291d5-0b2e-433a-b0f7-7552369c0670@github.com> Message-ID: <91LKg6-SSUOPqU1cxte24ifHcYDJluIiv5pAE_Gw-lI=.e5cc4f54-2647-43a7-b870-2dca80ac8e97@github.com> On Wed, 1 Mar 2023 16:30:53 GMT, Brian Burkhalter wrote: > I ran this test already and it still succeeds. I'm sure it does, but shouldn't it be replaced with some simple to just make sure that the modifiers are ignored? It is currently one of the slowest tests in the jdk_nio tree and with the changes here it overlaps with the existing WatchService tests. ------------- PR: https://git.openjdk.org/jdk/pull/12795 From bpb at openjdk.org Wed Mar 1 17:54:35 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 1 Mar 2023 17:54:35 GMT Subject: RFR: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService [v2] In-Reply-To: References: Message-ID: > Modify `sun.nio.fs.PollingWatchService` to ignore any polling sensitivity modifiers as done by the other `WatchService`s. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8303413: Reduce SensitivityModifier test to ensuring no exception on register ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12795/files - new: https://git.openjdk.org/jdk/pull/12795/files/2ab63574..be2c1b4c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12795&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12795&range=00-01 Stats: 48 lines in 1 file changed: 0 ins; 47 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/12795.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12795/head:pull/12795 PR: https://git.openjdk.org/jdk/pull/12795 From alanb at openjdk.org Thu Mar 2 08:35:06 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 2 Mar 2023 08:35:06 GMT Subject: RFR: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService [v2] In-Reply-To: References: Message-ID: On Wed, 1 Mar 2023 17:54:35 GMT, Brian Burkhalter wrote: >> Modify `sun.nio.fs.PollingWatchService` to ignore any polling sensitivity modifiers as done by the other `WatchService`s. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8303413: Reduce SensitivityModifier test to ensuring no exception on register test/jdk/java/nio/file/WatchService/SensitivityModifier.java line 96: > 94: // register the directories (random sensitivity) > 95: register(dirs, watcher); > 96: I assume there is a lot more that is no longer needed in this test, e.g most of doTest is not needed, POLL_TIMEOUT_SECONDS can be removed, ... I suspect the resulting test will be very simple - just test that the temp dir can be registered with each of the sensitivity levels. ------------- PR: https://git.openjdk.org/jdk/pull/12795 From michaelo at apache.org Thu Mar 2 10:51:55 2023 From: michaelo at apache.org (Michael Osipov) Date: Thu, 2 Mar 2023 11:51:55 +0100 Subject: ATOMIC_MOVE fails under high frequency conditions on Windows In-Reply-To: References: <0f99feeb-3508-8853-730c-f9870ebe62af@apache.org> Message-ID: Am 2023-02-27 um 17:56 schrieb Robert Muir: > On Sun, Feb 26, 2023 at 3:37?PM Michael Osipov wrote: >> The MoveFileEx does not properly block until the operation completes and >> some Windows-internal locking fails the operation. Even the >> While investigating the problem, it seems to be very common on Windows >> with Jenkins, Kafka, etc. but also Python folks discussed the issue >> (https://github.com/python/cpython/issues/53074). >> Now, if I apply the following patch: >>> - Files.move( tempFile, file, StandardCopyOption.ATOMIC_MOVE ); >>> + Kernel32.INSTANCE.MoveFileEx( tempFile.toAbsolutePath().toString(), file.toAbsolutePath().toString(), >>> + new DWORD( WinBase.MOVEFILE_REPLACE_EXISTING | WinBase.MOVEFILE_WRITE_THROUGH ) ); >> >> It just works. MOVEFILE_WRITE_THROUGH guarantees the operaton to block >> until it is completed. > > Hi, we rely on this as well in apache lucene, and have some gnarly > tests around it, but we haven't seen this specific issue. > Maybe we dodge it because we always fsync() files before renaming > them? I don't see any synchronization to disk in your sample code. No, we haven't used fsync() so far, this collocated file stuff is pretty new. We simply didn't expect this and never had such issues. Can you guide me to the code spots in Lucene for the optimal behavior? M From rcmuir at gmail.com Thu Mar 2 13:07:24 2023 From: rcmuir at gmail.com (Robert Muir) Date: Thu, 2 Mar 2023 08:07:24 -0500 Subject: ATOMIC_MOVE fails under high frequency conditions on Windows In-Reply-To: References: <0f99feeb-3508-8853-730c-f9870ebe62af@apache.org> Message-ID: On Thu, Mar 2, 2023 at 5:52?AM Michael Osipov wrote: > > Can you guide me to the code spots in Lucene for the optimal behavior? > You just want to look into FileChannel.force(). Here is the helper method we use: https://github.com/apache/lucene/blob/70c152bf321f20c589f1e002fa01ccbfc74aa398/lucene/core/src/java/org/apache/lucene/util/IOUtils.java#L408-L444 From bpb at openjdk.org Thu Mar 2 16:51:38 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 2 Mar 2023 16:51:38 GMT Subject: RFR: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService [v3] In-Reply-To: References: Message-ID: > Modify `sun.nio.fs.PollingWatchService` to ignore any polling sensitivity modifiers as done by the other `WatchService`s. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8303413: Simplify SensitivityModifier test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12795/files - new: https://git.openjdk.org/jdk/pull/12795/files/be2c1b4c..2e9867bd Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12795&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12795&range=01-02 Stats: 41 lines in 1 file changed: 3 ins; 25 del; 13 mod Patch: https://git.openjdk.org/jdk/pull/12795.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12795/head:pull/12795 PR: https://git.openjdk.org/jdk/pull/12795 From bpb at openjdk.org Thu Mar 2 16:54:27 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 2 Mar 2023 16:54:27 GMT Subject: RFR: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService [v3] In-Reply-To: References: Message-ID: On Thu, 2 Mar 2023 08:32:45 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8303413: Simplify SensitivityModifier test > > test/jdk/java/nio/file/WatchService/SensitivityModifier.java line 96: > >> 94: // register the directories (random sensitivity) >> 95: register(dirs, watcher); >> 96: > > I assume there is a lot more that is no longer needed in this test, e.g most of doTest is not needed, POLL_TIMEOUT_SECONDS can be removed, ... I suspect the resulting test will be very simple - just test that the temp dir can be registered with each of the sensitivity levels. Fixed (enough?) in 2e9867bd45cfbad727699a1254514227b995dd7a. ------------- PR: https://git.openjdk.org/jdk/pull/12795 From alanb at openjdk.org Thu Mar 2 19:39:24 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 2 Mar 2023 19:39:24 GMT Subject: RFR: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService [v3] In-Reply-To: References: Message-ID: On Thu, 2 Mar 2023 16:51:38 GMT, Brian Burkhalter wrote: >> Modify `sun.nio.fs.PollingWatchService` to ignore any polling sensitivity modifiers as done by the other `WatchService`s. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8303413: Simplify SensitivityModifier test Updated version looks fine. In this case, I'd probably update the @bug line to add 8303413, only because the test is very different to the original test. ------------- Marked as reviewed by alanb (Reviewer). PR: https://git.openjdk.org/jdk/pull/12795 From bpb at openjdk.org Thu Mar 2 20:07:58 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 2 Mar 2023 20:07:58 GMT Subject: RFR: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService [v4] In-Reply-To: References: Message-ID: > Modify `sun.nio.fs.PollingWatchService` to ignore any polling sensitivity modifiers as done by the other `WatchService`s. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8303413: Add 8303413 to @bug tag in SensitivityModifier test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12795/files - new: https://git.openjdk.org/jdk/pull/12795/files/2e9867bd..a2216a72 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12795&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12795&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/12795.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12795/head:pull/12795 PR: https://git.openjdk.org/jdk/pull/12795 From bpb at openjdk.org Thu Mar 2 20:08:01 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 2 Mar 2023 20:08:01 GMT Subject: RFR: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService [v3] In-Reply-To: References: Message-ID: <8O0oZyFuL2SQ_XLv9AWcaR9P7hWPzTSvSbG4Asu-Hq0=.aaf546bd-57ba-48e5-a6c0-b986aeb6957b@github.com> On Thu, 2 Mar 2023 19:36:21 GMT, Alan Bateman wrote: > I'd probably update the @bug line to add 8303413 Done in a2216a728b642a649c4c6261cd8ed1b1bc18136a. ------------- PR: https://git.openjdk.org/jdk/pull/12795 From alanb at openjdk.org Fri Mar 3 07:11:17 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 3 Mar 2023 07:11:17 GMT Subject: RFR: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService [v4] In-Reply-To: References: Message-ID: On Thu, 2 Mar 2023 20:07:58 GMT, Brian Burkhalter wrote: >> Modify `sun.nio.fs.PollingWatchService` to ignore any polling sensitivity modifiers as done by the other `WatchService`s. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8303413: Add 8303413 to @bug tag in SensitivityModifier test Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/12795 From bpb at openjdk.org Fri Mar 3 18:37:27 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 3 Mar 2023 18:37:27 GMT Subject: Integrated: 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService In-Reply-To: References: Message-ID: On Wed, 1 Mar 2023 00:27:46 GMT, Brian Burkhalter wrote: > Modify `sun.nio.fs.PollingWatchService` to ignore any polling sensitivity modifiers as done by the other `WatchService`s. This pull request has now been integrated. Changeset: a04b1049 Author: Brian Burkhalter URL: https://git.openjdk.org/jdk/commit/a04b1049ffc29c2ab2ec1b39e70b72288a39b371 Stats: 110 lines in 2 files changed: 3 ins; 79 del; 28 mod 8303413: (fs) Ignore polling interval sensitivity modifiers in PollingWatchService Reviewed-by: alanb ------------- PR: https://git.openjdk.org/jdk/pull/12795 From alanb at openjdk.org Sun Mar 5 19:24:02 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 5 Mar 2023 19:24:02 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled Message-ID: This is a small update to the underlying implementation of Socket.setTrafficClass so that it attempts to set the IP_TOS socket option when the socket is IPv6 and the operating system allows this option to be set on IPv6 sockets. The change is deliberately limited to the SocketImpl and SocketChannel implementation to avoid impacting DatagramChannel. Socket.setTrafficClass is specified as a hint. For TCP connections it is possible on some platforms to set IP_TOS on an unbound socket to the ToS value to use in the SYN packet when establishing a connection. Setting it after a connection is established is possible on some platforms too. The JDK uses IPv6 sockets since JDK 1.4 so it gets a bit more complicated due to patchy support for IP_TOS. - Linux allows IP_TOS to be set on an IPv6 socket at any time. If set on an unbound socket then the ToS will have the expected value when establishing a connection and it can be changed any time after connecting. - macOS doesn't allow IP_TOS to be set on an IPv6 socket and it doesn't use the IPV6_TCLASS value as the IPv4 ToS. So on macOS, you have to use an IPv4 socket and set IP_TOS before connecting the socket. - Windows hasn't allowed applications change the ToS for a long time. It allows the IP_TOS to be set on an unbound IPv6 socket but the value is not used. The bug report report is Socket.setTrafficClass hasn't worked on Linux since JDK 13 when the SocketImpl was replaced. The workaround is to run with-Djava.net.preferIPv4=true. The proposed change is to just attempt to set both IPV6_TCLASS and IP_TOS when the socket is IPv6 and the operating system allows IPPROTO level sockets to be setup on IPv6 sockets. At some point I'd like to replace the underlying socket options implementation as it no longer needs to be in libnet, and that would be the limit to re-visit the equivalent in the channel implementations. I had to use tcpdump and strace to test the changes, I don't think it's possible to create an automated test for this change. ------------- Commit messages: - Extend to SocketChannel socket adaptor - Initial commit Changes: https://git.openjdk.org/jdk/pull/12872/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12872&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8303509 Stats: 27 lines in 3 files changed: 17 ins; 1 del; 9 mod Patch: https://git.openjdk.org/jdk/pull/12872.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12872/head:pull/12872 PR: https://git.openjdk.org/jdk/pull/12872 From duke at openjdk.org Sun Mar 5 21:09:11 2023 From: duke at openjdk.org (altrisi) Date: Sun, 5 Mar 2023 21:09:11 GMT Subject: RFR: 8303260: (fc) FileChannel::transferFrom should support position > size() In-Reply-To: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> References: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> Message-ID: On Wed, 1 Mar 2023 01:34:33 GMT, Brian Burkhalter wrote: > Enhance `FileChannel::transferFrom` to extend the channel's file if `position > size()`. This is consistent with `FileChannel::write(ByteBuffer, long)`. Unrelated to the content of this PR, but the bot seems to be obsessed with editing the main comment description, at the time of writing it has made 4852 edits, seems to make around one per minute. Thought I'd mention it given it's easy to miss so I figured this way it's more likely to get resolved. ------------- PR: https://git.openjdk.org/jdk/pull/12797 From djelinski at openjdk.org Mon Mar 6 09:14:12 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 6 Mar 2023 09:14:12 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled In-Reply-To: References: Message-ID: On Sun, 5 Mar 2023 09:33:34 GMT, Alan Bateman wrote: > This is a small update to the underlying implementation of Socket.setTrafficClass so that it attempts to set the IP_TOS socket option when the socket is IPv6 and the operating system allows this option to be set on IPv6 sockets. The change is deliberately limited to the SocketImpl and SocketChannel implementation to avoid impacting DatagramChannel. > > Socket.setTrafficClass is specified as a hint. For TCP connections it is possible on some platforms to set IP_TOS on an unbound socket to the ToS value to use in the SYN packet when establishing a connection. Setting it after a connection is established is possible on some platforms too. The JDK uses IPv6 sockets since JDK 1.4 so it gets a bit more complicated due to patchy support for IP_TOS. > > - Linux allows IP_TOS to be set on an IPv6 socket at any time. If set on an unbound socket then the ToS will have the expected value when establishing a connection and it can be changed any time after connecting. > - macOS doesn't allow IP_TOS to be set on an IPv6 socket and it doesn't use the IPV6_TCLASS value as the IPv4 ToS. So on macOS, you have to use an IPv4 socket and set IP_TOS before connecting the socket. > - Windows hasn't allowed applications change the ToS for a long time. It allows the IP_TOS to be set on an unbound IPv6 socket but the value is not used. > > The bug report report is Socket.setTrafficClass hasn't worked on Linux since JDK 13 when the SocketImpl was replaced. The workaround is to run with-Djava.net.preferIPv4=true. > > The proposed change is to just attempt to set both IPV6_TCLASS and IP_TOS when the socket is IPv6 and the operating system allows IPPROTO level sockets to be setup on IPv6 sockets. At some point I'd like to replace the underlying socket options implementation as it no longer needs to be in libnet, and that would be the limit to re-visit the equivalent in the channel implementations. > > I had to use tcpdump and strace to test the changes, I don't think it's possible to create an automated test for this change. LGTM. Have you considered merging the new `setIpSocketOption` with the existing `DatagramChannelImpl.setOption`? ------------- Marked as reviewed by djelinski (Committer). PR: https://git.openjdk.org/jdk/pull/12872 From alanb at openjdk.org Mon Mar 6 09:53:14 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 6 Mar 2023 09:53:14 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled In-Reply-To: References: Message-ID: On Mon, 6 Mar 2023 09:11:04 GMT, Daniel Jeli?ski wrote: > Have you considered merging the new `setIpSocketOption` with the existing `DatagramChannelImpl.setOption`? Yes, but not for this PR because there is a larger piece of work required here to get rid of a lot of cruft related to socket options. ------------- PR: https://git.openjdk.org/jdk/pull/12872 From duke at openjdk.org Mon Mar 6 15:26:15 2023 From: duke at openjdk.org (JackyWoo) Date: Mon, 6 Mar 2023 15:26:15 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled Message-ID: This pr try to fix [JDK-8303509](https://bugs.openjdk.org/browse/JDK-8303509): Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled. ### Change Log When setting traffic class with IPv6 enabled, also set to IPv4 family. ------------- Commit messages: - add some comments - 8303509:fix setTrafficClass not work when IPv6 enabled Changes: https://git.openjdk.org/jdk/pull/12852/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12852&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8303509 Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/12852.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12852/head:pull/12852 PR: https://git.openjdk.org/jdk/pull/12852 From kcr at openjdk.org Mon Mar 6 15:26:15 2023 From: kcr at openjdk.org (Kevin Rushforth) Date: Mon, 6 Mar 2023 15:26:15 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled In-Reply-To: References: Message-ID: On Fri, 3 Mar 2023 10:36:45 GMT, JackyWoo wrote: > This pr try to fix [JDK-8303509](https://bugs.openjdk.org/browse/JDK-8303509): Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled. > > ### Change Log > When setting traffic class with IPv6 enabled, also set to IPv4 family. You will need to change the title of your PR, which you might as well do while waiting for your OCA to be confirmed. First, add a space after the "8303509:". Second, change the rest of it to _exactly_ match the title of the bug, meaning do not use the word "fix" in the title, nor use back-ticks for code style. Putting that all together, the title should be exactly the following: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled ------------- PR: https://git.openjdk.org/jdk/pull/12852 From alanb at openjdk.org Mon Mar 6 15:52:37 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 6 Mar 2023 15:52:37 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled In-Reply-To: References: Message-ID: <7XHeHM7NpxwkYtFsB-m1x-T4_X2VF1k56h9kLzaeKkI=.0b2143f4-74df-47ca-810c-164e46298158@github.com> On Fri, 3 Mar 2023 10:36:45 GMT, JackyWoo wrote: > This pr try to fix [JDK-8303509](https://bugs.openjdk.org/browse/JDK-8303509): Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled. > > ### Change Log > When setting traffic class with IPv6 enabled, also set to IPv4 family. [PR 12872](https://github.com/openjdk/jdk/pull/12872) is already under review for this issue. ------------- PR: https://git.openjdk.org/jdk/pull/12852 From michaelm at openjdk.org Mon Mar 6 16:08:23 2023 From: michaelm at openjdk.org (Michael McMahon) Date: Mon, 6 Mar 2023 16:08:23 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled In-Reply-To: References: Message-ID: On Sun, 5 Mar 2023 09:33:34 GMT, Alan Bateman wrote: > This is a small update to the underlying implementation of Socket.setTrafficClass so that it attempts to set the IP_TOS socket option when the socket is IPv6 and the operating system allows this option to be set on IPv6 sockets. The change is deliberately limited to the SocketImpl and SocketChannel implementation to avoid impacting DatagramChannel. > > Socket.setTrafficClass is specified as a hint. For TCP connections it is possible on some platforms to set IP_TOS on an unbound socket to the ToS value to use in the SYN packet when establishing a connection. Setting it after a connection is established is possible on some platforms too. The JDK uses IPv6 sockets since JDK 1.4 so it gets a bit more complicated due to patchy support for IP_TOS. > > - Linux allows IP_TOS to be set on an IPv6 socket at any time. If set on an unbound socket then the ToS will have the expected value when establishing a connection and it can be changed any time after connecting. > - macOS doesn't allow IP_TOS to be set on an IPv6 socket and it doesn't use the IPV6_TCLASS value as the IPv4 ToS. So on macOS, you have to use an IPv4 socket and set IP_TOS before connecting the socket. > - Windows hasn't allowed applications change the ToS for a long time. It allows the IP_TOS to be set on an unbound IPv6 socket but the value is not used. > > The bug report report is Socket.setTrafficClass hasn't worked on Linux since JDK 13 when the SocketImpl was replaced. The workaround is to run with-Djava.net.preferIPv4=true. > > The proposed change is to just attempt to set both IPV6_TCLASS and IP_TOS when the socket is IPv6 and the operating system allows IPPROTO level sockets to be setup on IPv6 sockets. At some point I'd like to replace the underlying socket options implementation as it no longer needs to be in libnet, and that would be the limit to re-visit the equivalent in the channel implementations. > > I had to use tcpdump and strace to test the changes, I don't think it's possible to create an automated test for this change. src/java.base/share/classes/sun/nio/ch/Net.java line 104: > 102: * need to be set so that the settings are effective for IPv4 connections > 103: * and datagrams. > 104: */ There is a similar reference to multicast in the native implementations (windows and unix) of shouldSetBothIPv4AndIPv6Options0(). Maybe, the same comment change should be there too. ------------- PR: https://git.openjdk.org/jdk/pull/12872 From michaelm at openjdk.org Mon Mar 6 16:33:07 2023 From: michaelm at openjdk.org (Michael McMahon) Date: Mon, 6 Mar 2023 16:33:07 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled In-Reply-To: References: Message-ID: On Sun, 5 Mar 2023 09:33:34 GMT, Alan Bateman wrote: > This is a small update to the underlying implementation of Socket.setTrafficClass so that it attempts to set the IP_TOS socket option when the socket is IPv6 and the operating system allows this option to be set on IPv6 sockets. The change is deliberately limited to the SocketImpl and SocketChannel implementation to avoid impacting DatagramChannel. > > Socket.setTrafficClass is specified as a hint. For TCP connections it is possible on some platforms to set IP_TOS on an unbound socket to the ToS value to use in the SYN packet when establishing a connection. Setting it after a connection is established is possible on some platforms too. The JDK uses IPv6 sockets since JDK 1.4 so it gets a bit more complicated due to patchy support for IP_TOS. > > - Linux allows IP_TOS to be set on an IPv6 socket at any time. If set on an unbound socket then the ToS will have the expected value when establishing a connection and it can be changed any time after connecting. > - macOS doesn't allow IP_TOS to be set on an IPv6 socket and it doesn't use the IPV6_TCLASS value as the IPv4 ToS. So on macOS, you have to use an IPv4 socket and set IP_TOS before connecting the socket. > - Windows hasn't allowed applications change the ToS for a long time. It allows the IP_TOS to be set on an unbound IPv6 socket but the value is not used. > > The bug report report is Socket.setTrafficClass hasn't worked on Linux since JDK 13 when the SocketImpl was replaced. The workaround is to run with-Djava.net.preferIPv4=true. > > The proposed change is to just attempt to set both IPV6_TCLASS and IP_TOS when the socket is IPv6 and the operating system allows IPPROTO level sockets to be setup on IPv6 sockets. At some point I'd like to replace the underlying socket options implementation as it no longer needs to be in libnet, and that would be the limit to re-visit the equivalent in the channel implementations. > > I had to use tcpdump and strace to test the changes, I don't think it's possible to create an automated test for this change. The change looks fine (with or without comment suggestion) ------------- Marked as reviewed by michaelm (Reviewer). PR: https://git.openjdk.org/jdk/pull/12872 From alanb at openjdk.org Mon Mar 6 17:17:47 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 6 Mar 2023 17:17:47 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled [v2] In-Reply-To: References: Message-ID: The message from this sender included one or more files which could not be scanned for virus detection; do not open these files unless you are certain of the sender's intent. ---------------------------------------------------------------------- > This is a small update to the underlying implementation of Socket.setTrafficClass so that it attempts to set the IP_TOS socket option when the socket is IPv6 and the operating system allows this option to be set on IPv6 sockets. The change is deliberately limited to the SocketImpl and SocketChannel implementation to avoid impacting DatagramChannel. > > Socket.setTrafficClass is specified as a hint. For TCP connections it is possible on some platforms to set IP_TOS on an unbound socket to the ToS value to use in the SYN packet when establishing a connection. Setting it after a connection is established is possible on some platforms too. The JDK uses IPv6 sockets since JDK 1.4 so it gets a bit more complicated due to patchy support for IP_TOS. > > - Linux allows IP_TOS to be set on an IPv6 socket at any time. If set on an unbound socket then the ToS will have the expected value when establishing a connection and it can be changed any time after connecting. > - macOS doesn't allow IP_TOS to be set on an IPv6 socket and it doesn't use the IPV6_TCLASS value as the IPv4 ToS. So on macOS, you have to use an IPv4 socket and set IP_TOS before connecting the socket. > - Windows hasn't allowed applications change the ToS for a long time. It allows the IP_TOS to be set on an unbound IPv6 socket but the value is not used. > > The bug report report is Socket.setTrafficClass hasn't worked on Linux since JDK 13 when the SocketImpl was replaced. The workaround is to run with-Djava.net.preferIPv4=true. > > The proposed change is to just attempt to set both IPV6_TCLASS and IP_TOS when the socket is IPv6 and the operating system allows IPPROTO level sockets to be setup on IPv6 sockets. At some point I'd like to replace the underlying socket options implementation as it no longer needs to be in libnet, and that would be the limit to re-visit the equivalent in the channel implementations. > > I had to use tcpdump and strace to test the changes, I don't think it's possible to create an automated test for this change. Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: - Update Net.shouldSetBothIPv4AndIPv6Options0 comments - Merge - Extend to SocketChannel socket adaptor - Initial commit ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12872/files - new: https://git.openjdk.org/jdk/pull/12872/files/c6766a4d..fd89b8f0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12872&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12872&range=00-01 Stats: 3935 lines in 82 files changed: 2177 ins; 1099 del; 659 mod Patch: https://git.openjdk.org/jdk/pull/12872.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12872/head:pull/12872 PR: https://git.openjdk.org/jdk/pull/12872 From alanb at openjdk.org Mon Mar 6 17:17:50 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 6 Mar 2023 17:17:50 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled [v2] In-Reply-To: References: Message-ID: On Mon, 6 Mar 2023 16:05:07 GMT, Michael McMahon wrote: >> Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: >> >> - Update Net.shouldSetBothIPv4AndIPv6Options0 comments >> - Merge >> - Extend to SocketChannel socket adaptor >> - Initial commit > > src/java.base/share/classes/sun/nio/ch/Net.java line 104: > >> 102: * need to be set so that the settings are effective for IPv4 connections >> 103: * and datagrams. >> 104: */ > > There is a similar reference to multicast in the native implementations (windows and unix) of shouldSetBothIPv4AndIPv6Options0(). Maybe, the same comment change should be there too. Good point, the comments in the native code should be aligned with the method description. ------------- PR: https://git.openjdk.org/jdk/pull/12872 From duke at openjdk.org Wed Mar 8 06:40:14 2023 From: duke at openjdk.org (JackyWoo) Date: Wed, 8 Mar 2023 06:40:14 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled In-Reply-To: <7XHeHM7NpxwkYtFsB-m1x-T4_X2VF1k56h9kLzaeKkI=.0b2143f4-74df-47ca-810c-164e46298158@github.com> References: <7XHeHM7NpxwkYtFsB-m1x-T4_X2VF1k56h9kLzaeKkI=.0b2143f4-74df-47ca-810c-164e46298158@github.com> Message-ID: On Mon, 6 Mar 2023 15:49:16 GMT, Alan Bateman wrote: >> This pr try to fix [JDK-8303509](https://bugs.openjdk.org/browse/JDK-8303509): Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled. >> >> ### Change Log >> When setting traffic class with IPv6 enabled, also set to IPv4 family. > > [PR 12872](https://github.com/openjdk/jdk/pull/12872) is already under review for this issue. @AlanBateman I encountered the problem in my production environment, so I reported the bug. Thanks for your fix. And I will close my PR. ------------- PR: https://git.openjdk.org/jdk/pull/12852 From duke at openjdk.org Wed Mar 8 06:40:16 2023 From: duke at openjdk.org (JackyWoo) Date: Wed, 8 Mar 2023 06:40:16 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled In-Reply-To: References: Message-ID: On Fri, 3 Mar 2023 11:56:20 GMT, Kevin Rushforth wrote: >> This pr try to fix [JDK-8303509](https://bugs.openjdk.org/browse/JDK-8303509): Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled. >> >> ### Change Log >> When setting traffic class with IPv6 enabled, also set to IPv4 family. > > You will need to change the title of your PR, which you might as well do while waiting for your OCA to be confirmed. First, add a space after the "8303509:". Second, change the rest of it to _exactly_ match the title of the bug, meaning do not use the word "fix" in the title, nor use back-ticks for code style. Putting that all together, the title should be exactly the following: > > 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled @kevinrushforth Thanks for you help to my first open JDK PR. ------------- PR: https://git.openjdk.org/jdk/pull/12852 From michaelm at openjdk.org Wed Mar 8 10:53:09 2023 From: michaelm at openjdk.org (Michael McMahon) Date: Wed, 8 Mar 2023 10:53:09 GMT Subject: RFR: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled [v2] In-Reply-To: References: Message-ID: On Mon, 6 Mar 2023 17:17:47 GMT, Alan Bateman wrote: >> This is a small update to the underlying implementation of Socket.setTrafficClass so that it attempts to set the IP_TOS socket option when the socket is IPv6 and the operating system allows this option to be set on IPv6 sockets. The change is deliberately limited to the SocketImpl and SocketChannel implementation to avoid impacting DatagramChannel. >> >> Socket.setTrafficClass is specified as a hint. For TCP connections it is possible on some platforms to set IP_TOS on an unbound socket to the ToS value to use in the SYN packet when establishing a connection. Setting it after a connection is established is possible on some platforms too. The JDK uses IPv6 sockets since JDK 1.4 so it gets a bit more complicated due to patchy support for IP_TOS. >> >> - Linux allows IP_TOS to be set on an IPv6 socket at any time. If set on an unbound socket then the ToS will have the expected value when establishing a connection and it can be changed any time after connecting. >> - macOS doesn't allow IP_TOS to be set on an IPv6 socket and it doesn't use the IPV6_TCLASS value as the IPv4 ToS. So on macOS, you have to use an IPv4 socket and set IP_TOS before connecting the socket. >> - Windows hasn't allowed applications change the ToS for a long time. It allows the IP_TOS to be set on an unbound IPv6 socket but the value is not used. >> >> The bug report report is Socket.setTrafficClass hasn't worked on Linux since JDK 13 when the SocketImpl was replaced. The workaround is to run with-Djava.net.preferIPv4=true. >> >> The proposed change is to just attempt to set both IPV6_TCLASS and IP_TOS when the socket is IPv6 and the operating system allows IPPROTO level sockets to be setup on IPv6 sockets. At some point I'd like to replace the underlying socket options implementation as it no longer needs to be in libnet, and that would be the limit to re-visit the equivalent in the channel implementations. >> >> I had to use tcpdump and strace to test the changes, I don't think it's possible to create an automated test for this change. > > Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: > > - Update Net.shouldSetBothIPv4AndIPv6Options0 comments > - Merge > - Extend to SocketChannel socket adaptor > - Initial commit Comment change looks good. ------------- Marked as reviewed by michaelm (Reviewer). PR: https://git.openjdk.org/jdk/pull/12872 From mike at hydraulic.software Wed Mar 8 10:14:36 2023 From: mike at hydraulic.software (Mike Hearn) Date: Wed, 8 Mar 2023 11:14:36 +0100 Subject: UNIX domain sockets bug on Windows In-Reply-To: <247b5106-9d23-fc84-e602-ec0e75a5e2ca@oracle.com> References: <57459407-d234-1be3-94e0-8b52826f1e86@oracle.com> <247b5106-9d23-fc84-e602-ec0e75a5e2ca@oracle.com> Message-ID: Unfortunately I need to resurrect this thread. Although in November we worked around the issue by patching the JVM we ship with our applications, now the apps being shipped by one of our customers are hitting the exact same thing. Obviously telling them to make their own JVM builds with patches is not a workable solution here, nor really for anyone else. Here's what we know: 1. The problem occurs only on some machines but not others (but the machines affected are affected reliably). 2. For unclear reasons on those machines the problem only occurs if the app is installed using an MSIX package (the type used by the Windows Store). If the app is run from an ordinary zip, network connections function correctly. This is probably why you find it hard to reproduce. I can supply such an app if it would help with testing. 3. We don't think it's security software related because the affected machines don't use any beyond the built-in Windows Defender (or put another way, if it is related, then that's the same as it being a Windows bug). Here's my guess as to what might be happening. Apps that are installed via the Windows Store or MSIX outside the store run using a lightweight form of file system virtualization. This is how Windows links files created in %APPDATA% to the app that created them, ensuring they get cleaned up correctly at uninstall time. https://learn.microsoft.com/en-us/windows/msix/desktop/desktop-to-uwp-behind-the-scenes The implementation involves a kernel driver called bindflt. In theory this virtualization doesn't affect temporary directories, but the combination of UNIX domain sockets+bindflt is probably not very common at the moment, and it may be the case that the Windows team have shipped a regression in some versions. Now if that theory is correct, it doesn't really explain why some machines are affected and others aren't when they all appear to be running the same OS version. That is mysterious and I can't explain it, which counts against the bindflt theory. An alternative possibility is that it's another race in winsock; you previously had to do rollbacks due to the discovery of such races which implies there might be more. But then we face the question of why the packaging type appears to change the outcome on the machines that do experience the bug. Regardless of the cause the result is the same: new Javas can't be used for apps distributed in this way. The best workaround would be to detect that this codepath is failing with the stack trace in the first email in this thread and fall back to the old TCP codepath. Is there any chance you'd consider such a change? Also if anyone at Oracle has contacts on the Windows team, it'd be great to ask them to investigate the issue. I don't have any such contacts :( thanks, -mike From michael.x.mcmahon at oracle.com Wed Mar 8 11:35:44 2023 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Wed, 8 Mar 2023 11:35:44 +0000 Subject: UNIX domain sockets bug on Windows In-Reply-To: References: <57459407-d234-1be3-94e0-8b52826f1e86@oracle.com> <247b5106-9d23-fc84-e602-ec0e75a5e2ca@oracle.com> Message-ID: If this filter is doing filename remapping, then one thing worth trying is to run with the system property jdk.net.unixdomain.tmpdir set to some alternate location. This specifies the location of the directory where we create the socket files for "automatically" bound sockets for ServerSocketChannel (ie when binding with a null address). This is what the NIO PipeImpl does when creating its loopback socket. Alternatively, you could try asking your customer to run a simple test of UNIX domain sockets on an affected machine and see what happens. Eg. the following snippet: var server =ServerSocketChannel.open(StandardProtocolFamily.UNIX); var name =Path.of("foo.sock"); Files.deleteIfExists(name); server.bind(UnixDomainSocketAddress.of(name)); var client =SocketChannel.open(server.getLocalAddress()); var s1 =server.accept(); System.out.printf("client: %s, server connection: %s\n",client.toString(),s1.toString()); - Michael On 08/03/2023 10:14, Mike Hearn wrote: > Unfortunately I need to resurrect this thread. Although in November we > worked around the issue by patching the JVM we ship with our > applications, now the apps being shipped by one of our customers are > hitting the exact same thing. Obviously telling them to make their own > JVM builds with patches is not a workable solution here, nor really > for anyone else. > > Here's what we know: > > 1. The problem occurs only on some machines but not others (but the > machines affected are affected reliably). > > 2. For unclear reasons on those machines the problem only occurs if > the app is installed using an MSIX package (the type used by the > Windows Store). If the app is run from an ordinary zip, network > connections function correctly. This is probably why you find it hard > to reproduce. I can supply such an app if it would help with testing. > > 3. We don't think it's security software related because the affected > machines don't use any beyond the built-in Windows Defender (or put > another way, if it is related, then that's the same as it being a > Windows bug). > > Here's my guess as to what might be happening. Apps that are installed > via the Windows Store or MSIX outside the store run using a > lightweight form of file system virtualization. This is how Windows > links files created in %APPDATA% to the app that created them, > ensuring they get cleaned up correctly at uninstall time. > > https://urldefense.com/v3/__https://learn.microsoft.com/en-us/windows/msix/desktop/desktop-to-uwp-behind-the-scenes__;!!ACWV5N9M2RV99hQ!LjA2A-_P6j5L4VE77_w_iMpceuHJIhdeVkj2dv2mTJgFEc_y3FPOOTfOTLAa8LkAODbJlqd5nHhfcAiJm_29sB6uMb8B$ > > The implementation involves a kernel driver called bindflt. In theory > this virtualization doesn't affect temporary directories, but the > combination of UNIX domain sockets+bindflt is probably not very common > at the moment, and it may be the case that the Windows team have > shipped a regression in some versions. Now if that theory is correct, > it doesn't really explain why some machines are affected and others > aren't when they all appear to be running the same OS version. That is > mysterious and I can't explain it, which counts against the bindflt > theory. An alternative possibility is that it's another race in > winsock; you previously had to do rollbacks due to the discovery of > such races which implies there might be more. But then we face the > question of why the packaging type appears to change the outcome on > the machines that do experience the bug. > > Regardless of the cause the result is the same: new Javas can't be > used for apps distributed in this way. The best workaround would be to > detect that this codepath is failing with the stack trace in the first > email in this thread and fall back to the old TCP codepath. Is there > any chance you'd consider such a change? Also if anyone at Oracle has > contacts on the Windows team, it'd be great to ask them to investigate > the issue. I don't have any such contacts :( > > thanks, > -mike -------------- next part -------------- An HTML attachment was scrubbed... URL: From djelinski at openjdk.org Wed Mar 8 11:59:09 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Wed, 8 Mar 2023 11:59:09 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions Message-ID: This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. Other changes include: - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. - `getLastErrorString` is no longer exported by libjava. Tier1-3 tests continue to pass. No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. ------------- Commit messages: - Update copyrights - Remove LastError - Change getLastErrorString to return a jstring Changes: https://git.openjdk.org/jdk/pull/12922/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12922&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8303814 Stats: 138 lines in 8 files changed: 19 ins; 44 del; 75 mod Patch: https://git.openjdk.org/jdk/pull/12922.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12922/head:pull/12922 PR: https://git.openjdk.org/jdk/pull/12922 From jpai at openjdk.org Wed Mar 8 12:15:13 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Wed, 8 Mar 2023 12:15:13 GMT Subject: RFR: 8303260: (fc) FileChannel::transferFrom should support position > size() In-Reply-To: References: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> Message-ID: On Sun, 5 Mar 2023 21:06:18 GMT, altrisi wrote: > Unrelated to the content of this PR, but the bot seems to be obsessed with editing the main comment description, at the time of writing it has made 4852 edits, seems to make around one per minute. Noted. Thank you @altrisi. The team that deals with skara bots has been made aware of this issue https://bugs.openjdk.org/browse/SKARA-1837 ------------- PR: https://git.openjdk.org/jdk/pull/12797 From mike at hydraulic.software Wed Mar 8 13:01:12 2023 From: mike at hydraulic.software (Mike Hearn) Date: Wed, 8 Mar 2023 14:01:12 +0100 Subject: UNIX domain sockets bug on Windows In-Reply-To: References: <57459407-d234-1be3-94e0-8b52826f1e86@oracle.com> <247b5106-9d23-fc84-e602-ec0e75a5e2ca@oracle.com> Message-ID: Thanks Michael, that's been a very useful suggestion. After I mailed, we discovered a VM that could repro the problem. This let us check the theory that it's FS virt related and indeed it is; disabling it fixes the issue. It is still worth working around it in the JVM because not everyone will see this thread and apps distributed via the MS Store using their standard instructions will hit it. To readers from the future! You have two ways to fix this if you're using a JVM/Windows build with the issue. One way is to add the following to the appropriate sections of your appx manifest: $(KnownFolder:LocalAppData)\Temp\JavaSockets and then on Windows set the system property jdk.net.unixdomain.tmpdir = "%LOCALAPPDATA%/Temp/JavaSockets" Or a better way, head over to https://hydraulic.software/ and download Conveyor which works around this bug and (sadly) several other Windows bugs for you. On Wed, 8 Mar 2023 at 12:35, Michael McMahon wrote: > > If this filter is doing filename remapping, then one thing worth trying is to run with the system property jdk.net.unixdomain.tmpdir set to some alternate location. This specifies the location of the directory where we create the socket files for "automatically" bound sockets for ServerSocketChannel (ie when binding with a null address). This is what the NIO PipeImpl does when creating its loopback socket. > > Alternatively, you could try asking your customer to run a simple test of UNIX domain sockets on an affected machine and see what happens. Eg. the following snippet: > > var server = ServerSocketChannel.open(StandardProtocolFamily.UNIX); > var name = Path.of("foo.sock"); > Files.deleteIfExists(name); > server.bind(UnixDomainSocketAddress.of(name)); > var client = SocketChannel.open(server.getLocalAddress()); > var s1 = server.accept(); > System.out.printf("client: %s, server connection: %s\n", client.toString(), s1.toString()); > > - Michael > > On 08/03/2023 10:14, Mike Hearn wrote: > > Unfortunately I need to resurrect this thread. Although in November we > worked around the issue by patching the JVM we ship with our > applications, now the apps being shipped by one of our customers are > hitting the exact same thing. Obviously telling them to make their own > JVM builds with patches is not a workable solution here, nor really > for anyone else. > > Here's what we know: > > 1. The problem occurs only on some machines but not others (but the > machines affected are affected reliably). > > 2. For unclear reasons on those machines the problem only occurs if > the app is installed using an MSIX package (the type used by the > Windows Store). If the app is run from an ordinary zip, network > connections function correctly. This is probably why you find it hard > to reproduce. I can supply such an app if it would help with testing. > > 3. We don't think it's security software related because the affected > machines don't use any beyond the built-in Windows Defender (or put > another way, if it is related, then that's the same as it being a > Windows bug). > > Here's my guess as to what might be happening. Apps that are installed > via the Windows Store or MSIX outside the store run using a > lightweight form of file system virtualization. This is how Windows > links files created in %APPDATA% to the app that created them, > ensuring they get cleaned up correctly at uninstall time. > > https://urldefense.com/v3/__https://learn.microsoft.com/en-us/windows/msix/desktop/desktop-to-uwp-behind-the-scenes__;!!ACWV5N9M2RV99hQ!LjA2A-_P6j5L4VE77_w_iMpceuHJIhdeVkj2dv2mTJgFEc_y3FPOOTfOTLAa8LkAODbJlqd5nHhfcAiJm_29sB6uMb8B$ > > The implementation involves a kernel driver called bindflt. In theory > this virtualization doesn't affect temporary directories, but the > combination of UNIX domain sockets+bindflt is probably not very common > at the moment, and it may be the case that the Windows team have > shipped a regression in some versions. Now if that theory is correct, > it doesn't really explain why some machines are affected and others > aren't when they all appear to be running the same OS version. That is > mysterious and I can't explain it, which counts against the bindflt > theory. An alternative possibility is that it's another race in > winsock; you previously had to do rollbacks due to the discovery of > such races which implies there might be more. But then we face the > question of why the packaging type appears to change the outcome on > the machines that do experience the bug. > > Regardless of the cause the result is the same: new Javas can't be > used for apps distributed in this way. The best workaround would be to > detect that this codepath is failing with the stack trace in the first > email in this thread and fall back to the old TCP codepath. Is there > any chance you'd consider such a change? Also if anyone at Oracle has > contacts on the Windows team, it'd be great to ask them to investigate > the issue. I don't have any such contacts :( > > thanks, > -mike From naoto at openjdk.org Wed Mar 8 20:05:16 2023 From: naoto at openjdk.org (Naoto Sato) Date: Wed, 8 Mar 2023 20:05:16 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions In-Reply-To: References: Message-ID: On Wed, 8 Mar 2023 11:30:27 GMT, Daniel Jeli?ski wrote: > This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. > > Other changes include: > - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. > - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. > - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. > - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. > - `getLastErrorString` is no longer exported by libjava. > > Tier1-3 tests continue to pass. > > No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. > Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. Hi Daniel, I like the idea of using `jstring` directly to avoid platform string. My concern is that is it OK to make it not export from libjava? Would it cause any concern on other platforms? Also, now it is not consistent with the other sibling `getErrorString()` call. ------------- PR: https://git.openjdk.org/jdk/pull/12922 From djelinski at openjdk.org Wed Mar 8 22:03:14 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Wed, 8 Mar 2023 22:03:14 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions In-Reply-To: References: Message-ID: On Wed, 8 Mar 2023 11:30:27 GMT, Daniel Jeli?ski wrote: > This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. > > Other changes include: > - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. > - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. > - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. > - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. > - `getLastErrorString` is no longer exported by libjava. > > Tier1-3 tests continue to pass. > > No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. > Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. I think it's fine to remove `getLastErrorString` from the list of libjava exports. After my changes it's no longer used outside libjava, and it was never a part of the JDK's public interface. Regarding consistency with `getErrorString`, I'm not too concerned about that; the functions have similar names but serve different purposes. ------------- PR: https://git.openjdk.org/jdk/pull/12922 From cjplummer at openjdk.org Wed Mar 8 23:26:13 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 8 Mar 2023 23:26:13 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions In-Reply-To: References: Message-ID: The message from this sender included one or more files which could not be scanned for virus detection; do not open these files unless you are certain of the sender's intent. ---------------------------------------------------------------------- On Wed, 8 Mar 2023 11:30:27 GMT, Daniel Jeli?ski wrote: > This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. > > Other changes include: > - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. > - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. > - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. > - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. > - `getLastErrorString` is no longer exported by libjava. > > Tier1-3 tests continue to pass. > > No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. > Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. We don't have a test for the SA changes you made. The best way to test it is with clhsdb. Run the following against a JVM pid: `$ jhsdb clhsdb --pid ` Use "jstack -v" to get a native pc from a frame, and then try `disassemble` on the address. It most likely will produce an exception since it can't find hsdis, which is actually what we want to be testing in this case: hsdb> disassemble 0x00007f38b371fca0 Error: sun.jvm.hotspot.debugger.DebuggerException: hsdis-amd64.so: cannot open shared object file: No such file or directory You'll need to test separately on Windows and any unix platform. ------------- PR: https://git.openjdk.org/jdk/pull/12922 From alanb at openjdk.org Thu Mar 9 08:17:41 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 9 Mar 2023 08:17:41 GMT Subject: Integrated: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled In-Reply-To: References: Message-ID: On Sun, 5 Mar 2023 09:33:34 GMT, Alan Bateman wrote: > This is a small update to the underlying implementation of Socket.setTrafficClass so that it attempts to set the IP_TOS socket option when the socket is IPv6 and the operating system allows this option to be set on IPv6 sockets. The change is deliberately limited to the SocketImpl and SocketChannel implementation to avoid impacting DatagramChannel. > > Socket.setTrafficClass is specified as a hint. For TCP connections it is possible on some platforms to set IP_TOS on an unbound socket to the ToS value to use in the SYN packet when establishing a connection. Setting it after a connection is established is possible on some platforms too. The JDK uses IPv6 sockets since JDK 1.4 so it gets a bit more complicated due to patchy support for IP_TOS. > > - Linux allows IP_TOS to be set on an IPv6 socket at any time. If set on an unbound socket then the ToS will have the expected value when establishing a connection and it can be changed any time after connecting. > - macOS doesn't allow IP_TOS to be set on an IPv6 socket and it doesn't use the IPV6_TCLASS value as the IPv4 ToS. So on macOS, you have to use an IPv4 socket and set IP_TOS before connecting the socket. > - Windows hasn't allowed applications change the ToS for a long time. It allows the IP_TOS to be set on an unbound IPv6 socket but the value is not used. > > The bug report report is Socket.setTrafficClass hasn't worked on Linux since JDK 13 when the SocketImpl was replaced. The workaround is to run with-Djava.net.preferIPv4=true. > > The proposed change is to just attempt to set both IPV6_TCLASS and IP_TOS when the socket is IPv6 and the operating system allows IPPROTO level sockets to be setup on IPv6 sockets. At some point I'd like to replace the underlying socket options implementation as it no longer needs to be in libnet, and that would be the limit to re-visit the equivalent in the channel implementations. > > I had to use tcpdump and strace to test the changes, I don't think it's possible to create an automated test for this change. This pull request has now been integrated. Changeset: dd794108 Author: Alan Bateman URL: https://git.openjdk.org/jdk/commit/dd79410824fa57c7fb1ce56c643bb52540f9a206 Stats: 32 lines in 5 files changed: 17 ins; 1 del; 14 mod 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled Reviewed-by: djelinski, michaelm ------------- PR: https://git.openjdk.org/jdk/pull/12872 From djelinski at openjdk.org Thu Mar 9 08:54:06 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Thu, 9 Mar 2023 08:54:06 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions In-Reply-To: References: Message-ID: On Wed, 8 Mar 2023 23:23:33 GMT, Chris Plummer wrote: >> This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. >> >> Other changes include: >> - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. >> - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. >> - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. >> - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. >> - `getLastErrorString` is no longer exported by libjava. >> >> Tier1-3 tests continue to pass. >> >> No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. >> Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. > > We don't have a test for the SA changes you made. The best way to test it is with clhsdb. Run the following against a JVM pid: > > `$ jhsdb clhsdb --pid ` > > Use "jstack -v" to get a native pc from a frame, and then try `disassemble` on the address. It most likely will produce an exception since it can't find hsdis, which is actually what we want to be testing in this case: > > > hsdb> disassemble 0x00007f38b371fca0 > Error: sun.jvm.hotspot.debugger.DebuggerException: hsdis-amd64.so: cannot open shared object file: No such file or directory > > > You'll need to test separately on Windows and any unix platform. Thanks @plummercj for the instructions. Here's the results: Linux, with this change applied: hsdb> disassemble 0x00007f3484558da0 Error: sun.jvm.hotspot.debugger.DebuggerException: hsdis-amd64.so: cannot open shared object file: No such file or directory Windows, EN, with the change: hsdb> disassemble 0x00000107d4dae0c6 Error: sun.jvm.hotspot.debugger.DebuggerException: The specified module could not be found Windows, misconfigured CN, without the change: hsdb> disassemble 0x00000200d60de0b4 Error: sun.jvm.hotspot.debugger.DebuggerException: ????????? Windows, misconfigured CN, with the change: hsdb> disassemble 0x000001fab996e0b4 Error: sun.jvm.hotspot.debugger.DebuggerException: ????????? (note: I had to run `chcp 65001` in the console, otherwise the exception would still display incorrectly) ------------- PR: https://git.openjdk.org/jdk/pull/12922 From naoto at openjdk.org Thu Mar 9 18:15:41 2023 From: naoto at openjdk.org (Naoto Sato) Date: Thu, 9 Mar 2023 18:15:41 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions In-Reply-To: References: Message-ID: On Wed, 8 Mar 2023 11:30:27 GMT, Daniel Jeli?ski wrote: > This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. > > Other changes include: > - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. > - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. > - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. > - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. > - `getLastErrorString` is no longer exported by libjava. > > Tier1-3 tests continue to pass. > > No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. > Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. Looks good (w/ some minor comments) src/java.base/share/native/libzip/zip_util.c line 767: > 765: * or NULL if an error occurred. If a zip error occurred then *pmsg will > 766: * be set to the error message text if pmsg != 0. Otherwise, *pmsg will be > 767: * set to NULL. Caller doesn't need to free the error message. I'd put some more context here why the caller does not need to free. (as it is a static text) src/java.base/windows/native/libjava/jni_util_md.c line 80: > 78: 0, > 79: buf, > 80: sizeof(buf) / sizeof(WCHAR), Maybe better to #define the size 256 so that this division is not needed. src/java.base/windows/native/libnio/ch/FileDispatcherImpl.c line 208: > 206: > 207: if (result == 0) { > 208: JNU_ThrowIOExceptionWithLastError(env, "Write failed"); Could be replaced with `JNU_ThrowIOException`? src/java.base/windows/native/libnio/ch/FileDispatcherImpl.c line 260: > 258: > 259: if (result == 0) { > 260: JNU_ThrowIOExceptionWithLastError(env, "Write failed"); Same as above src/java.base/windows/native/libnio/ch/FileDispatcherImpl.c line 299: > 297: > 298: if (result == 0) { > 299: JNU_ThrowIOExceptionWithLastError(env, "Write failed"); Same here ------------- PR: https://git.openjdk.org/jdk/pull/12922 From bpb at openjdk.org Fri Mar 10 00:43:04 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 10 Mar 2023 00:43:04 GMT Subject: RFR: 8303934: (fc) Use splice in FileChannel::transferFrom when the source is a pipe (Linux) Message-ID: On Linux, perform `transferFrom()` via the `splice(2)` system call when the source `ReadableByteChannel` is a `Pipe.SourceChannel` (`sun.nio.ch.SourceChannelImpl`). ------------- Commit messages: - 8303934: (fc) Use splice in FileChannel::transferFrom when the source is a pipe (Linux) Changes: https://git.openjdk.org/jdk/pull/12965/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12965&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8303934 Stats: 87 lines in 4 files changed: 49 ins; 6 del; 32 mod Patch: https://git.openjdk.org/jdk/pull/12965.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12965/head:pull/12965 PR: https://git.openjdk.org/jdk/pull/12965 From bpb at openjdk.org Fri Mar 10 00:43:04 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 10 Mar 2023 00:43:04 GMT Subject: RFR: 8303934: (fc) Use splice in FileChannel::transferFrom when the source is a pipe (Linux) In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 00:35:38 GMT, Brian Burkhalter wrote: > On Linux, perform `transferFrom()` via the `splice(2)` system call when the source `ReadableByteChannel` is a `Pipe.SourceChannel` (`sun.nio.ch.SourceChannelImpl`). The Linux `splice()` system call copies data in kernel address space between two file descriptors without copying the data to user address space. At present the execution pathway for transferring from a pipe source is the arbitrary channel pathway which is the slowest. Using `splice()` for this case increases throughput by more than 30%. This is the increase measured using this simple benchmark (excerpt) for a buffer of capacity 8192: ByteBuffer buf; Pipe pipe; FileChannel fc; @Benchmark public long transferFromPipe() throws IOException { pipe.sink().write(buf); buf.rewind(); return fc.transferFrom(pipe.source(), 0, SIZE); } As the benchmark measurement includes writing to the pipe sink, the increase in throughput should actually be larger than that measured. Test tiers 1-3 pass on all platforms. The modifications to the AIX and Windows source files are format cleanup only. ------------- PR: https://git.openjdk.org/jdk/pull/12965 From bpb at openjdk.org Fri Mar 10 01:26:10 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 10 Mar 2023 01:26:10 GMT Subject: RFR: 8303934: (fc) Use splice in FileChannel::transferFrom when the source is a pipe (Linux) In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 00:35:38 GMT, Brian Burkhalter wrote: > On Linux, perform `transferFrom()` via the `splice(2)` system call when the source `ReadableByteChannel` is a `Pipe.SourceChannel` (`sun.nio.ch.SourceChannelImpl`). The Linux system call `copy_file_range(2)` does not work with pipes. In the case of `FileChannel::transferTo` writing to a pipe, the native code path falls back to `sendfile(2)` which does handle a destination pipe so `splice(2)` is not needed there. The `offset` parameter of `sendfile()` however refers to the input so that call is not usable in `FileChannel::transferFrom` which requires the position in the output, i.e., the channel?s file. ------------- PR: https://git.openjdk.org/jdk/pull/12965 From cjplummer at openjdk.org Fri Mar 10 05:50:03 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 10 Mar 2023 05:50:03 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions In-Reply-To: References: Message-ID: On Wed, 8 Mar 2023 11:30:27 GMT, Daniel Jeli?ski wrote: > This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. > > Other changes include: > - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. > - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. > - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. > - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. > - `getLastErrorString` is no longer exported by libjava. > > Tier1-3 tests continue to pass. > > No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. > Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. I'm approving the SA changes. Thanks for the testing. ------------- Marked as reviewed by cjplummer (Reviewer). PR: https://git.openjdk.org/jdk/pull/12922 From djelinski at openjdk.org Fri Mar 10 08:06:16 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Fri, 10 Mar 2023 08:06:16 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions In-Reply-To: References: Message-ID: On Thu, 9 Mar 2023 00:17:42 GMT, Naoto Sato wrote: >> This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. >> >> Other changes include: >> - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. >> - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. >> - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. >> - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. >> - `getLastErrorString` is no longer exported by libjava. >> >> Tier1-3 tests continue to pass. >> >> No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. >> Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. > > src/java.base/windows/native/libnio/ch/FileDispatcherImpl.c line 208: > >> 206: >> 207: if (result == 0) { >> 208: JNU_ThrowIOExceptionWithLastError(env, "Write failed"); > > Could be replaced with `JNU_ThrowIOException`? If we got here, `WriteFile` just failed and `GetLastError` contains interesting information. `JNU_ThrowIOExceptionWithLastError` will generate specific error message in user's language, `JNU_ThrowIOException` would just throw `Write failed`. I don't think we want to change this. ------------- PR: https://git.openjdk.org/jdk/pull/12922 From alanb at openjdk.org Fri Mar 10 10:43:14 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 10 Mar 2023 10:43:14 GMT Subject: RFR: 8303260: (fc) FileChannel::transferFrom should support position > size() In-Reply-To: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> References: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> Message-ID: On Wed, 1 Mar 2023 01:34:33 GMT, Brian Burkhalter wrote: > Enhance `FileChannel::transferFrom` to extend the channel's file if `position > size()`. This is consistent with `FileChannel::write(ByteBuffer, long)`. The spec + impl changes looks fine. Is this test copied/edited from an older test? Configuration name FAST_TRANSFER_xxx looks a bit odd and maybe we just replace it with a simpler JUnit style test? ------------- PR: https://git.openjdk.org/jdk/pull/12797 From duke at openjdk.org Fri Mar 10 15:39:05 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 10 Mar 2023 15:39:05 GMT Subject: RFR: 8303972: TestLocOffsetFromZip64EF could use a sparse file Message-ID: Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. By using a sparse ZIP file instead of calling out to the `zip` command line, we can speed up the test (from 50s to 3s on my Macbook Pro), save 4GB disk space during builds and remove a dependency on the `zip` command in OS/distros. To get an idea on how the new test performs on file systems which do not support sparse files, I also ran the using a disk image in MacOS with a non-spase file system. The test completed in 14 seconds. Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. ------------- Commit messages: - Remove accidentally introduced blank line - Merge branch 'master' into loc-offset-zip64-sparse - Rewrite TestLocOffsetFromZip64EF to create ZIP file using sparse ZipOutputStream instead of calling out to the ZIP command line Changes: https://git.openjdk.org/jdk/pull/12979/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12979&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8303972 Stats: 120 lines in 2 files changed: 20 ins; 56 del; 44 mod Patch: https://git.openjdk.org/jdk/pull/12979.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12979/head:pull/12979 PR: https://git.openjdk.org/jdk/pull/12979 From brian.burkhalter at oracle.com Fri Mar 10 15:39:08 2023 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 10 Mar 2023 15:39:08 +0000 Subject: RFR: 8303260: (fc) FileChannel::transferFrom should support position > size() In-Reply-To: References: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> Message-ID: <49AC3CAF-1512-47A5-B502-AB6A68A66E5F@oracle.com> On Mar 10, 2023, at 2:43 AM, Alan Bateman > wrote: On Wed, 1 Mar 2023 01:34:33 GMT, Brian Burkhalter > wrote: Enhance `FileChannel::transferFrom` to extend the channel's file if `position > size()`. This is consistent with `FileChannel::write(ByteBuffer, long)`. The spec + impl changes looks fine. Is this test copied/edited from an older test? No, I don?t recall doing that. Configuration name FAST_TRANSFER_xxx looks a bit odd and maybe we just replace it with a simpler JUnit style test? Will investigate. -------------- next part -------------- An HTML attachment was scrubbed... URL: From duke at openjdk.org Fri Mar 10 20:21:15 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 10 Mar 2023 20:21:15 GMT Subject: RFR: 8303972: TestLocOffsetFromZip64EF could use a sparse file [v2] In-Reply-To: References: Message-ID: > Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. > > By using a sparse ZIP file instead of calling out to the `zip` command line, we can speed up the test (from 50s to 3s on my Macbook Pro), save 4GB disk space during builds and remove a dependency on the `zip` command in OS/distros. > > To get an idea on how the new test performs on file systems which do not support sparse files, I also ran the using a disk image in MacOS with a non-spase file system. The test completed in 14 seconds. > > Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. Eirik Bjorsnos has updated the pull request incrementally with one additional commit since the last revision: Instead of making a big file to force Zip64, make a small file with a Zip64-formatted CEN header. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12979/files - new: https://git.openjdk.org/jdk/pull/12979/files/ee8adc0c..2efd7275 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12979&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12979&range=00-01 Stats: 107 lines in 1 file changed: 49 ins; 23 del; 35 mod Patch: https://git.openjdk.org/jdk/pull/12979.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12979/head:pull/12979 PR: https://git.openjdk.org/jdk/pull/12979 From duke at openjdk.org Fri Mar 10 20:50:05 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 10 Mar 2023 20:50:05 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v3] In-Reply-To: References: Message-ID: > Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. > > Instead of calling out to the `zip` command line, we produce a small-sized Zip64 file in the test itself. This file has the features required to reproduce the ZipFileSystem issue, namely that the 'INFO-ZIP extended timestamp' extended field must come before the 'Zip64 extended information' field. (This would casue ZipFileSystem to read the LOC at position 0xFFFFFFFF). > > This speed up the test (from 50s to 3s on my Macbook Pro), saves 4GB disk space during builds removes a dependency on the `zip` command in OS/distros. > > Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. Eirik Bjorsnos has updated the pull request incrementally with one additional commit since the last revision: The Zip64 fields are all zero, we don't need to update them ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12979/files - new: https://git.openjdk.org/jdk/pull/12979/files/2efd7275..2a68bd89 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12979&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12979&range=01-02 Stats: 13 lines in 1 file changed: 2 ins; 6 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/12979.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12979/head:pull/12979 PR: https://git.openjdk.org/jdk/pull/12979 From lancea at openjdk.org Fri Mar 10 21:18:06 2023 From: lancea at openjdk.org (Lance Andersen) Date: Fri, 10 Mar 2023 21:18:06 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v3] In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 20:50:05 GMT, Eirik Bjorsnos wrote: >> Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. >> >> Instead of calling out to the `zip` command line, we produce a small-sized Zip64 file in the test itself. This file has the features required to reproduce the ZipFileSystem issue, namely that the 'INFO-ZIP extended timestamp' extended field must come before the 'Zip64 extended information' field. (This would casue ZipFileSystem to read the LOC at position 0xFFFFFFFF). >> >> This speed up the test (from 50s to 3s on my Macbook Pro), saves 4GB disk space during builds removes a dependency on the `zip` command in OS/distros. >> >> Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. > > Eirik Bjorsnos has updated the pull request incrementally with one additional commit since the last revision: > > The Zip64 fields are all zero, we don't need to update them This test was leveraging a file created by info-zip intentionally so care needs to be taken to ensure that the zip that is being created is similar LOC and CEN output ------------- PR: https://git.openjdk.org/jdk/pull/12979 From duke at openjdk.org Fri Mar 10 21:18:08 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 10 Mar 2023 21:18:08 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 15:10:29 GMT, Eirik Bjorsnos wrote: > Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. > > Instead of calling out to the `zip` command line, we produce a small-sized Zip64 file in the test itself. This file has the features required to reproduce the ZipFileSystem issue, namely that the 'INFO-ZIP extended timestamp' extended field must come before the 'Zip64 extended information' field. (This would casue ZipFileSystem to read the LOC at position 0xFFFFFFFF). > > This speed up the test (from 50s to 3s on my Macbook Pro), saves 4GB disk space during builds removes a dependency on the `zip` command in OS/distros. > > Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. Converting to draft while I investigate why the updated test does not actually catch the regression ------------- PR: https://git.openjdk.org/jdk/pull/12979 From duke at openjdk.org Fri Mar 10 21:18:09 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 10 Mar 2023 21:18:09 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v3] In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 16:56:55 GMT, Lance Andersen wrote: > This test was leveraging a file created by info-zip intentionally so care needs to be taken to ensure that the zip that is being created is similar LOC and CEN output Yes, I just noticed that and currently looking into it. I think in particular it is sensitive to the ordering of the extended time stamp and the Zip64 entries, which is not the same as ZipOutputStream. ------------- PR: https://git.openjdk.org/jdk/pull/12979 From duke at openjdk.org Fri Mar 10 21:18:11 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 10 Mar 2023 21:18:11 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v2] In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 20:21:15 GMT, Eirik Bjorsnos wrote: >> Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. >> >> Instead of calling out to the `zip` command line, we produce a small-sized Zip64 file in the test itself. This file has the features required to reproduce the ZipFileSystem issue, namely that the 'INFO-ZIP extended timestamp' extended field must come before the 'Zip64 extended information' field. (This would casue ZipFileSystem to read the LOC at position 0xFFFFFFFF). >> >> This speed up the test (from 50s to 3s on my Macbook Pro), saves 4GB disk space during builds removes a dependency on the `zip` command in OS/distros. >> >> Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. > > Eirik Bjorsnos has updated the pull request incrementally with one additional commit since the last revision: > > Instead of making a big file to force Zip64, make a small file with a Zip64-formatted CEN header. While it is possible to use a sparse file here, there is a problem that when ZipFileSystem reads inside a 'hole', it will skip ahead to the next data which turned out to be a LOC header. We could carefully select an entry size such that 0XFFFFFFFF does not not fall into in a hole, but that feels a bit too magic. Instead, I opted for creating a small ZIP file with a Zip64-formatted CEN. Since this issue is sensitive to ordering of extra fields, the 'extended timestamp' needs to come before the 'Zip64 extended information field'. The way I solve this is to add an opaque extra field, right sized for 'Zip64'. This uses a tag ID unknown to ZipEntry.setExtra, such that it will not be parsed as Zip64 but simply passed through. ZipOutputStream will output this after the extended timestamp. After the ZIP is produced, the Zip64 entry is updated with the correct tag. I have verified that this causes ZipFileSystem (before JDK-8255380) to attempt to read the LOC from 0xFFFFFFFF. With a small file size, this causes a slightly different error message because of the attempt to read beyond the end of the file. ----------System.out:(160/10450)---------- config TestLocOffsetFromZip64EF.setUp(): success test TestLocOffsetFromZip64EF.walkZipFSTest({}): failure java.util.zip.ZipException: loc: reading failed at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.readExtra(ZipFileSystem.java:3127) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.readCEN(ZipFileSystem.java:2831) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.(ZipFileSystem.java:2767) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.getFileAttributes(ZipFileSystem.java:550) ``` While the symptom is slightly different, the cause is the same: ZipFileSystem tries to read the LOC at offset 0xFFFFFFFF because it has not read the Zip64 entry yet. ------------- PR: https://git.openjdk.org/jdk/pull/12979 From alanb at openjdk.org Fri Mar 10 21:18:13 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 10 Mar 2023 21:18:13 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v3] In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 20:50:05 GMT, Eirik Bjorsnos wrote: >> Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. >> >> Instead of calling out to the `zip` command line, we produce a small-sized Zip64 file in the test itself. This file has the features required to reproduce the ZipFileSystem issue, namely that the 'INFO-ZIP extended timestamp' extended field must come before the 'Zip64 extended information' field. (This would casue ZipFileSystem to read the LOC at position 0xFFFFFFFF). >> >> This speed up the test (from 50s to 3s on my Macbook Pro), saves 4GB disk space during builds removes a dependency on the `zip` command in OS/distros. >> >> Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. > > Eirik Bjorsnos has updated the pull request incrementally with one additional commit since the last revision: > > The Zip64 fields are all zero, we don't need to update them test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java line 136: > 134: // Make a ZIP with two entries > 135: try (FileOutputStream fileOutputStream = new FileOutputStream(new File(ZIP_FILE_NAME)); > 136: ZipOutputStream zo = new ZipOutputStream(new SparseOutputStream(fileOutputStream))) { Can you use FileChannel.open and specify SPARSE in the set of open options, I think that would make it clearer that the file is sparse and remove the need for SparseOutputStream. ------------- PR: https://git.openjdk.org/jdk/pull/12979 From duke at openjdk.org Fri Mar 10 21:18:14 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 10 Mar 2023 21:18:14 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v3] In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 15:48:40 GMT, Alan Bateman wrote: >> Eirik Bjorsnos has updated the pull request incrementally with one additional commit since the last revision: >> >> The Zip64 fields are all zero, we don't need to update them > > test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java line 136: > >> 134: // Make a ZIP with two entries >> 135: try (FileOutputStream fileOutputStream = new FileOutputStream(new File(ZIP_FILE_NAME)); >> 136: ZipOutputStream zo = new ZipOutputStream(new SparseOutputStream(fileOutputStream))) { > > Can you use FileChannel.open and specify SPARSE in the set of open options, I think that would make it clearer that the file is sparse and remove the need for SparseOutputStream. Not sure I understand. I could use FileChannel.open with SPARSE (which btw is ignored by the implementation:), but that doesn't give me an OutputStream I can pass to ZipOutputStream? I could use Channels.newOutputStream, but that would not create holes in the sparse files? My current thinking is that we need the SparseOutputStream to detect that ZIpOutputStream is writing empty bytes and replace that with `channel.position(channel.position() + len)` ------------- PR: https://git.openjdk.org/jdk/pull/12979 From alanb at openjdk.org Fri Mar 10 21:18:14 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 10 Mar 2023 21:18:14 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v3] In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 15:54:12 GMT, Eirik Bjorsnos wrote: > Not sure I understand. I could use FileChannel.open with SPARSE (which btw is ignored by the implementation:) That is true on platforms where files can be sparse by default. On Windows you'll see this translates to create the file as sparse, it needs to be opt-in when creating the file. ------------- PR: https://git.openjdk.org/jdk/pull/12979 From duke at openjdk.org Fri Mar 10 21:18:14 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 10 Mar 2023 21:18:14 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v3] In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 16:58:38 GMT, Alan Bateman wrote: >> Not sure I understand. I could use FileChannel.open with SPARSE (which btw is ignored by the implementation:), but that doesn't give me an OutputStream I can pass to ZipOutputStream? >> >> I could use Channels.newOutputStream, but that would not create holes in the sparse files? >> >> My current thinking is that we need the SparseOutputStream to detect that ZIpOutputStream is writing empty bytes and replace that with `channel.position(channel.position() + len)` > >> Not sure I understand. I could use FileChannel.open with SPARSE (which btw is ignored by the implementation:) > > That is true on platforms where files can be sparse by default. On Windows you'll see this translates to create the file as sparse, it needs to be opt-in when creating the file. I opted to create a small-sized Zip64 file instead (see below). This file does not need to be sparse. ------------- PR: https://git.openjdk.org/jdk/pull/12979 From duke at openjdk.org Fri Mar 10 21:18:14 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 10 Mar 2023 21:18:14 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v3] In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 20:34:56 GMT, Eirik Bjorsnos wrote: >>> Not sure I understand. I could use FileChannel.open with SPARSE (which btw is ignored by the implementation:) >> >> That is true on platforms where files can be sparse by default. On Windows you'll see this translates to create the file as sparse, it needs to be opt-in when creating the file. > > I opted to create a small-sized Zip64 file instead (see below). This file does not need to be sparse. > That is true on platforms where files can be sparse by default. On Windows you'll see this translates to create the file as sparse, it needs to be opt-in when creating the file. Thanks for this info, Alan! While this PR no longer makes use of sparse files, I have applied your comments about FileChannel.open for NTFS in another PR (which does make use of sparse files): https://github.com/openjdk/jdk/pull/12948 ------------- PR: https://git.openjdk.org/jdk/pull/12979 From rriggs at openjdk.org Fri Mar 10 22:05:16 2023 From: rriggs at openjdk.org (Roger Riggs) Date: Fri, 10 Mar 2023 22:05:16 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions In-Reply-To: References: Message-ID: On Wed, 8 Mar 2023 11:30:27 GMT, Daniel Jeli?ski wrote: > This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. > > Other changes include: > - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. > - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. > - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. > - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. > - `getLastErrorString` is no longer exported by libjava. > > Tier1-3 tests continue to pass. > > No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. > Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. src/java.base/share/native/libjava/jni_util.c line 133: > 131: if (s != NULL) { > 132: jobject x = NULL; > 133: if (messagelen) { Avoid implicit compare with 0; use `messagelen > 0` or similar. src/java.base/share/native/libjava/jni_util.c line 137: > 135: size_t messageextlen = messagelen + 4; > 136: char *str1 = (char *)malloc((messageextlen) * sizeof(char)); > 137: if (str1 == 0) { Use NULL when comparing to a pointer. src/java.base/unix/native/libjava/jni_util_md.c line 69: > 67: if (errno == 0) return NULL; > 68: getErrorString(errno, buf, sizeof(buf)); > 69: return (*buf) ? JNU_NewStringPlatform(env, buf) : NULL; I would have used `buf[0] != 0` since its declared as a local buffer; but this file doesn't have much of a style to follow. ------------- PR: https://git.openjdk.org/jdk/pull/12922 From duke at openjdk.org Sat Mar 11 07:28:14 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Sat, 11 Mar 2023 07:28:14 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v4] In-Reply-To: References: Message-ID: > Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. > > Instead of calling out to the `zip` command line, we produce a small-sized Zip64 file in the test itself. This file has the features required to reproduce the ZipFileSystem issue, namely that the 'INFO-ZIP extended timestamp' field must come before the 'Zip64 extended information' field. (This would casue ZipFileSystem to read the LOC at position 0xFFFFFFFF). > > This speed up the test (from 50s to 3s on my Macbook Pro), saves 4GB disk space during builds removes a dependency on the `zip` command in OS/distros. > > Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. Eirik Bjorsnos has updated the pull request incrementally with one additional commit since the last revision: APPNOTE.txt actually specifies an 'unknown' tag, let's use that instead of 0x42 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12979/files - new: https://git.openjdk.org/jdk/pull/12979/files/2a68bd89..22eba5c3 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12979&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12979&range=02-03 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/12979.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12979/head:pull/12979 PR: https://git.openjdk.org/jdk/pull/12979 From Brian.Stafford at microsoft.com Fri Mar 10 00:20:36 2023 From: Brian.Stafford at microsoft.com (Brian Stafford) Date: Fri, 10 Mar 2023 00:20:36 +0000 Subject: UNIX domain sockets bug on Windows Message-ID: Hi Mike, We'll follow up on this bug report with the Windows team. Thanks, Brian Stafford he/him Principal Software Engineering Manager Java Engineering Group brstaffo at microsoft.com Microsoft > Thanks Michael, that's been a very useful suggestion. After I mailed, > we discovered a VM that could repro the problem. This let us check the > theory that it's FS virt related and indeed it is; disabling it fixes > the issue. It is still worth working around it in the JVM because not > everyone will see this thread and apps distributed via the MS Store > using their standard instructions will hit it. > > To readers from the future! You have two ways to fix this if you're > using a JVM/Windows build with the issue. One way is to add the > following to the appropriate sections of your appx manifest: > > > > > $(KnownFolder:LocalAppData)\Temp\JavaSockets > > > > > > > > and then on Windows set the system property jdk.net.unixdomain.tmpdir > = "%LOCALAPPDATA%/Temp/JavaSockets" > > Or a better way, head over to https://hydraulic.software/ and download > Conveyor which works around this bug and (sadly) several other Windows > bugs for you. From duke at openjdk.org Sun Mar 12 16:31:39 2023 From: duke at openjdk.org (Markus KARG) Date: Sun, 12 Mar 2023 16:31:39 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v19] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: - updated copyright - Implementing 8278268 Signed-off-by: Markus Karg ------------- Changes: https://git.openjdk.org/jdk/pull/6711/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=6711&range=18 Stats: 806 lines in 6 files changed: 574 ins; 199 del; 33 mod Patch: https://git.openjdk.org/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.org/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.org/jdk/pull/6711 From duke at openjdk.org Sun Mar 12 16:31:40 2023 From: duke at openjdk.org (Markus KARG) Date: Sun, 12 Mar 2023 16:31:40 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v18] In-Reply-To: References: Message-ID: On Tue, 28 Feb 2023 23:49:59 GMT, Brian Burkhalter wrote: > Before anything else, you might consider merging the current master into your branch. Aside from the copyright year change of the most recent commit, the branch is over two and a half months out of date. I have rebased this PR branch ontop of the current master branch, so it is up-to-date-now. ------------- PR: https://git.openjdk.org/jdk/pull/6711 From duke at openjdk.org Sun Mar 12 17:04:28 2023 From: duke at openjdk.org (Markus KARG) Date: Sun, 12 Mar 2023 17:04:28 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v18] In-Reply-To: References: Message-ID: On Wed, 1 Mar 2023 01:37:42 GMT, Brian Burkhalter wrote: >> Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. Incremental views are not available. > > In the test classes `TransferTo` and `TransferTo2`, the methods `testNullPointerException` and `testStreamContents` are identical aside from expecting different `DataProvider`s. The contents of the methods could be abstracted out to common methods called by different `Test`s which expect different `DataProvider`s. > > The test classes `TransferTo_2GB_transferFrom` and `TransferTo_2GB_transferTo` comprise roughly 120 lines of code each, but a diff of their contents reveals little in the way of substantive differences aside from jtreg tags, comments, and names: > > 96,99c95,97 > < // performing actual transfer, effectively by multiple invocations of > < // FileChannel.transferFrom(ReadableByteChannel) > < try (InputStream inputStream = Channels.newInputStream(Channels.newChannel( > < new BufferedInputStream(Files.newInputStream(sourceFile)))); > --- >> // perform actual transfer, effectively by multiple invocations >> // of FileChannel.transferTo(WritableByteChannel) >> try (InputStream inputStream = Channels.newInputStream(FileChannel.open(sourceFile)); > > Perhaps something along these lines might work: > * rename `TransferToBase` to for example to `TransferBase` or `TransferFromToBase`; > * rename `TransferTo2` to `TransferFrom`; > * rename `TransferTo_2GB_transferX` classes to `TransferX_2GB`; > * refactor common code to `TransferBase` along the lines described above; > * refactor child classes as needed to use the common code. > > Assuming the foregoing makes sense, there would remain a total of four tests. It might be possible also to merge `TransferFrom` and `TransferTo` into a single test, but I think the two tests `*_2GB*` should remain separate as they likely each have a sufficiently long run time. @bplb Before I stark to refactor anything, it makes sense to briefly discuss your recent proposal: > In the test classes `TransferTo` and `TransferTo2`, the methods `testNullPointerException` and `testStreamContents` are identical aside from expecting different `DataProvider`s. The contents of the methods could be abstracted out to common methods called by different `Test`s which expect different `DataProvider`s. This is correct and I have no objections to that, but on the other hand, there is no need to keep the separate `TransferTo2` class separated, as the reason for the existence of `TransferTo2` was a bug somewhere else in OpenJDK that does not exist anymore (for reference, the bug was in the Pipe implementation on Windows). Do that fact, Lance proposed to *merge* these files: > I haven't gone through this in detail, but we want to merge The remaining TransferTo2 tests into TransferTo. So can you please briefly **confirm** that I *shall not* merge `TransferTo2` into `TransferTo` as *Lance* proposed (hence `TransferTo2` will not be a *separate* class anymore) but **instead** I *only shall* abstract `testNullPointerException` and `testStreamContents` as *you* proposed (hence keep `TransferTo2` as a *separate* class)? Thanks. >...I think the two tests `*_2GB*` should remain separate as they likely each have a sufficiently long run time. I confirm that we had intentionally kept these two tests in separate classes due to the long run time of each, so I agree to keep them *as separate* classes, but I will reduce *duplicate* code. I need to mention that the **file names** *should not* be changed according to you proposed, as `*_transferFrom` is actually *not* testing the method `TransferFrom`, but it is testing the method `TransferTo`'s implementation which *internally uses* `transferFrom`. Hence it would be wrong w.r.t to the general test file naming convention to rename the test to `TransferFrom`. So can you please briefly **confirm** that I am correct with this objection and the 2GB files *should not* be renamed? ------------- PR: https://git.openjdk.org/jdk/pull/6711 From djelinski at openjdk.org Mon Mar 13 09:48:19 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 13 Mar 2023 09:48:19 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions [v2] In-Reply-To: References: Message-ID: > This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. > > Other changes include: > - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. > - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. > - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. > - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. > - `getLastErrorString` is no longer exported by libjava. > > Tier1-3 tests continue to pass. > > No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. > Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. Daniel Jeli?ski has updated the pull request incrementally with three additional commits since the last revision: - Address review comments - Mention that the returned text is static and thread safe - Define buffer size ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12922/files - new: https://git.openjdk.org/jdk/pull/12922/files/8ab8d729..ea91b651 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12922&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12922&range=00-01 Stats: 8 lines in 4 files changed: 2 ins; 0 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/12922.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12922/head:pull/12922 PR: https://git.openjdk.org/jdk/pull/12922 From djelinski at openjdk.org Mon Mar 13 09:48:22 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 13 Mar 2023 09:48:22 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions [v2] In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 21:47:45 GMT, Roger Riggs wrote: >> Daniel Jeli?ski has updated the pull request incrementally with three additional commits since the last revision: >> >> - Address review comments >> - Mention that the returned text is static and thread safe >> - Define buffer size > > src/java.base/share/native/libjava/jni_util.c line 133: > >> 131: if (s != NULL) { >> 132: jobject x = NULL; >> 133: if (messagelen) { > > Avoid implicit compare with 0; use `messagelen > 0` or similar. preexisting, but fixed. ------------- PR: https://git.openjdk.org/jdk/pull/12922 From djelinski at openjdk.org Mon Mar 13 09:48:27 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 13 Mar 2023 09:48:27 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions [v2] In-Reply-To: References: Message-ID: On Thu, 9 Mar 2023 18:08:32 GMT, Naoto Sato wrote: >> Daniel Jeli?ski has updated the pull request incrementally with three additional commits since the last revision: >> >> - Address review comments >> - Mention that the returned text is static and thread safe >> - Define buffer size > > src/java.base/share/native/libzip/zip_util.c line 767: > >> 765: * or NULL if an error occurred. If a zip error occurred then *pmsg will >> 766: * be set to the error message text if pmsg != 0. Otherwise, *pmsg will be >> 767: * set to NULL. Caller doesn't need to free the error message. > > I'd put some more context here why the caller does not need to free. (as it is a static text) Added; also mentioned that we want the buffer to be thread-safe. Let me know if that's what you had in mind. > src/java.base/windows/native/libjava/jni_util_md.c line 80: > >> 78: 0, >> 79: buf, >> 80: sizeof(buf) / sizeof(WCHAR), > > Maybe better to #define the size 256 so that this division is not needed. done ------------- PR: https://git.openjdk.org/jdk/pull/12922 From alanb at openjdk.org Mon Mar 13 12:39:21 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 13 Mar 2023 12:39:21 GMT Subject: RFR: 8303934: (fc) Use splice in FileChannel::transferFrom when the source is a pipe (Linux) In-Reply-To: References: Message-ID: On Fri, 10 Mar 2023 00:35:38 GMT, Brian Burkhalter wrote: > On Linux, perform `transferFrom()` via the `splice(2)` system call when the source `ReadableByteChannel` is a `Pipe.SourceChannel` (`sun.nio.ch.SourceChannelImpl`). A concern with optimizations like this is that they don't coordinate with other methods on the source channel, in this case read and close. This goes for the optimizations in the transferTo implementation too. ------------- PR: https://git.openjdk.org/jdk/pull/12965 From rriggs at openjdk.org Mon Mar 13 15:10:31 2023 From: rriggs at openjdk.org (Roger Riggs) Date: Mon, 13 Mar 2023 15:10:31 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions [v2] In-Reply-To: References: Message-ID: <88rPs699hRVHL_w1dIfbGgp5fhsoP9a4aTXqtuXZD8g=.1a441cba-0807-4de1-9174-92e2eeff3dad@github.com> On Mon, 13 Mar 2023 09:48:19 GMT, Daniel Jeli?ski wrote: >> This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. >> >> Other changes include: >> - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. >> - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. >> - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. >> - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. >> - `getLastErrorString` is no longer exported by libjava. >> >> Tier1-3 tests continue to pass. >> >> No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. >> Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. > > Daniel Jeli?ski has updated the pull request incrementally with three additional commits since the last revision: > > - Address review comments > - Mention that the returned text is static and thread safe > - Define buffer size LGTM for some pre-existing code style inconsistencies. src/java.base/share/native/libzip/zip_util.c line 812: > 810: > 811: if (strlen(name) >= PATH_MAX) { > 812: if (pmsg) { Some pre-existing references to pmsg still use implicit comparison, though not material to this PR, can you update them to compare with NULL. ------------- Marked as reviewed by rriggs (Reviewer). PR: https://git.openjdk.org/jdk/pull/12922 From djelinski at openjdk.org Mon Mar 13 15:55:27 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 13 Mar 2023 15:55:27 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions [v3] In-Reply-To: References: Message-ID: > This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. > > Other changes include: > - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. > - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. > - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. > - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. > - `getLastErrorString` is no longer exported by libjava. > > Tier1-3 tests continue to pass. > > No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. > Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. Daniel Jeli?ski has updated the pull request incrementally with one additional commit since the last revision: Use NULL where appropriate ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12922/files - new: https://git.openjdk.org/jdk/pull/12922/files/ea91b651..efd72a1d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12922&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12922&range=01-02 Stats: 12 lines in 1 file changed: 0 ins; 0 del; 12 mod Patch: https://git.openjdk.org/jdk/pull/12922.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12922/head:pull/12922 PR: https://git.openjdk.org/jdk/pull/12922 From djelinski at openjdk.org Mon Mar 13 16:01:05 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 13 Mar 2023 16:01:05 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions [v2] In-Reply-To: <88rPs699hRVHL_w1dIfbGgp5fhsoP9a4aTXqtuXZD8g=.1a441cba-0807-4de1-9174-92e2eeff3dad@github.com> References: <88rPs699hRVHL_w1dIfbGgp5fhsoP9a4aTXqtuXZD8g=.1a441cba-0807-4de1-9174-92e2eeff3dad@github.com> Message-ID: On Mon, 13 Mar 2023 15:05:04 GMT, Roger Riggs wrote: >> Daniel Jeli?ski has updated the pull request incrementally with three additional commits since the last revision: >> >> - Address review comments >> - Mention that the returned text is static and thread safe >> - Define buffer size > > src/java.base/share/native/libzip/zip_util.c line 812: > >> 810: >> 811: if (strlen(name) >= PATH_MAX) { >> 812: if (pmsg) { > > Some pre-existing references to pmsg still use implicit comparison, though not material to this PR, can you update them to compare with NULL. done. ------------- PR: https://git.openjdk.org/jdk/pull/12922 From naoto at openjdk.org Mon Mar 13 16:47:52 2023 From: naoto at openjdk.org (Naoto Sato) Date: Mon, 13 Mar 2023 16:47:52 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions [v3] In-Reply-To: References: Message-ID: On Mon, 13 Mar 2023 15:55:27 GMT, Daniel Jeli?ski wrote: >> This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. >> >> Other changes include: >> - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. >> - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. >> - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. >> - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. >> - `getLastErrorString` is no longer exported by libjava. >> >> Tier1-3 tests continue to pass. >> >> No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. >> Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. > > Daniel Jeli?ski has updated the pull request incrementally with one additional commit since the last revision: > > Use NULL where appropriate Marked as reviewed by naoto (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/12922 From brian.burkhalter at oracle.com Mon Mar 13 23:16:09 2023 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 13 Mar 2023 23:16:09 +0000 Subject: String encoding to ByteBuffer In-Reply-To: <1127337400.2205617.1677454797231@email.ionos.com> References: <1127337400.2205617.1677454797231@email.ionos.com> Message-ID: <82795151-C155-4D44-B875-E27F7E81E70C@oracle.com> Redirecting to nio-dev which is the more appropriate forum for this topic. > On Feb 26, 2023, at 3:39 PM, Carl M wrote: > > I'm looking into adding a fast path case for encoding Strings into ByteBuffers, and wanted to get feedback on a possible approach. My use case is taking mostly-ASCII, UTF-8 Strings and writing them to the disk/network. To do this today, there are two approaches which both have drawbacks: > > 1. Use String.getBytes(StandardCharsets.UTF_8), and call ByteBuffer.put(). The downside of this approach is that I need to make a copy of the String's byte[] value. The upside of this approach is that ByteBuffer uses the intrinsic copy methods, which are fast. > > 2. Wrap the String in a CharBuffer, and call CharsetEncoder.encode(CharBuffer, ByteBuffer). This avoids copying the String value. However, when using the UTF_8 encoder, there is no fastpath for writing to direct ByteBuffers. sun.nio.cs.UTF_8.encodeLoop() only has fast paths for when the destination is array based. This allocates less memory, but is overall slower in my JMH benchmark. > > To fix this, I looked at adding an overload to CharsetEncoder to accept a String (or a CharSequence), and a ByteBuffer as a destination. However, this is not easily doable, since it's hard to call it in a loop. In the case that the String overflows the BB, the caller needs to be able to provide a new BB and resume from where they left off. The CharBuffer approach works here because it keeps the position last read, and can resume from there. > > To encode a String, we need to know that the character index written to resume with a larger buffer. However, the return type on CharsetEncoder's encode method is a CoderResult. The length() method on this can't be called for underflow cases. This means that there isn't a usable return type here (neither int nor CoderResult can be used). > > Another, almost-possible solution I was considering adding a special case to UTF_8 for direct buffer destinations, and a corresponding JLA.encodeASCII overload that accepts a ByteBuffer. The challenge here is that a wrapped CharBuffer doesn't have an array, and so doesn't get the fast path copying. > > The reason I am reaching out here is that I am looking for feedback on my analysis of the existing API. I am wondering what API compromises could be made to fast path writing Strings to direct buffers, which I feel is probably a common operation. The only reasonable way I can see to implement is a new return type, which also seems undesirable as well. > > Carl From djelinski at openjdk.org Tue Mar 14 17:22:18 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Tue, 14 Mar 2023 17:22:18 GMT Subject: Integrated: 8303814: getLastErrorString should avoid charset conversions In-Reply-To: References: Message-ID: On Wed, 8 Mar 2023 11:30:27 GMT, Daniel Jeli?ski wrote: > This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. > > Other changes include: > - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. > - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. > - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. > - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. > - `getLastErrorString` is no longer exported by libjava. > > Tier1-3 tests continue to pass. > > No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. > Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. This pull request has now been integrated. Changeset: baf11e73 Author: Daniel Jeli?ski URL: https://git.openjdk.org/jdk/commit/baf11e734f7b5308490edc74f3168744c0857b24 Stats: 151 lines in 8 files changed: 21 ins; 44 del; 86 mod 8303814: getLastErrorString should avoid charset conversions Reviewed-by: naoto, cjplummer, rriggs ------------- PR: https://git.openjdk.org/jdk/pull/12922 From jwaters at openjdk.org Tue Mar 14 17:50:01 2023 From: jwaters at openjdk.org (Julian Waters) Date: Tue, 14 Mar 2023 17:50:01 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions [v3] In-Reply-To: References: Message-ID: On Mon, 13 Mar 2023 15:55:27 GMT, Daniel Jeli?ski wrote: >> This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. >> >> Other changes include: >> - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. >> - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. >> - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. >> - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. >> - `getLastErrorString` is no longer exported by libjava. >> >> Tier1-3 tests continue to pass. >> >> No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. >> Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. > > Daniel Jeli?ski has updated the pull request incrementally with one additional commit since the last revision: > > Use NULL where appropriate The change in Windows behaviour seems like a worrying gotcha that someone using the method might be trapped by (C library errors and system errors are reported for Unix while only WINAPI is returned on Windows). Although this has already been pushed I'd still like to mention that me and Thomas did have quite a bit of discussion regarding the conundrum on Windows about whether to report both when the error routines are called or have separate methods/mechanism to select either, see [8292016](https://bugs.openjdk.org/browse/JDK-8292016?filter=-1) ------------- PR: https://git.openjdk.org/jdk/pull/12922 From djelinski at openjdk.org Tue Mar 14 18:20:17 2023 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Tue, 14 Mar 2023 18:20:17 GMT Subject: RFR: 8303814: getLastErrorString should avoid charset conversions [v3] In-Reply-To: References: Message-ID: On Mon, 13 Mar 2023 15:55:27 GMT, Daniel Jeli?ski wrote: >> This patch modifies the `getLastErrorString` method to return a `jstring`. Thanks to that we can avoid unnecessary back and forth conversions between Unicode and other charsets on Windows. >> >> Other changes include: >> - the Windows implementation of `getLastErrorString` no longer checks `errno`. I verified all uses of the method and confirmed that `errno` is not used anywhere. >> - While at it, I found and fixed a few calls to `JNU_ThrowIOExceptionWithLastError` that were done in context where `LastError` was not set. >> - jdk.hotspot.agent was modified to use `JNU_ThrowByNameWithLastError` and `JNU_ThrowByName` instead of `getLastErrorString`; the code is expected to have identical behavior. >> - zip_util was modified to return static messages instead of generated ones. The generated messages were not observed anywhere, because they were replaced by a static message in ZIP_Open, which is the only method used by other native code. >> - `getLastErrorString` is no longer exported by libjava. >> >> Tier1-3 tests continue to pass. >> >> No new automated regression test; testing this requires installing a language pack that cannot be displayed in the current code page. >> Tested this manually by installing Chinese language pack on English Windows 11, selecting Chinese language, then checking if the message on exception thrown by `InetAddress.getByName("nonexistent.local");` starts with `"?????????"` (or `"\u4e0d\u77e5\u9053\u8fd9\u6837\u7684\u4e3b\u673a\u3002"`). Without the change, the exception message started with a row of question marks. > > Daniel Jeli?ski has updated the pull request incrementally with one additional commit since the last revision: > > Use NULL where appropriate we never use any of the JNU_XXX functions to report errno on Windows as far as I could tell. And even if we did, we'd need to call SetLastError(0) before JNU_Throw to make it work, which we never did. I think we're ok here. ------------- PR: https://git.openjdk.org/jdk/pull/12922 From bpb at openjdk.org Wed Mar 15 21:49:15 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 15 Mar 2023 21:49:15 GMT Subject: RFR: 8303260: (fc) FileChannel::transferFrom should support position > size() [v2] In-Reply-To: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> References: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> Message-ID: > Enhance `FileChannel::transferFrom` to extend the channel's file if `position > size()`. This is consistent with `FileChannel::write(ByteBuffer, long)`. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8303260: Convert test to JUnit 5 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12797/files - new: https://git.openjdk.org/jdk/pull/12797/files/82afccce..0639c239 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12797&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12797&range=00-01 Stats: 91 lines in 1 file changed: 43 ins; 21 del; 27 mod Patch: https://git.openjdk.org/jdk/pull/12797.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12797/head:pull/12797 PR: https://git.openjdk.org/jdk/pull/12797 From bpb at openjdk.org Wed Mar 15 21:49:34 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 15 Mar 2023 21:49:34 GMT Subject: RFR: 8303260: (fc) FileChannel::transferFrom should support position > size() In-Reply-To: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> References: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> Message-ID: On Wed, 1 Mar 2023 01:34:33 GMT, Brian Burkhalter wrote: > Enhance `FileChannel::transferFrom` to extend the channel's file if `position > size()`. This is consistent with `FileChannel::write(ByteBuffer, long)`. > [M]aybe we just replace it (the test) with a simpler JUnit style test? Addressed by 0639c2390e228f572c71a1642801033e7fc45222. ------------- PR: https://git.openjdk.org/jdk/pull/12797 From duke at openjdk.org Thu Mar 16 02:10:30 2023 From: duke at openjdk.org (JackyWoo) Date: Thu, 16 Mar 2023 02:10:30 GMT Subject: Withdrawn: 8303509: Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled In-Reply-To: References: Message-ID: On Fri, 3 Mar 2023 10:36:45 GMT, JackyWoo wrote: > This pr try to fix [JDK-8303509](https://bugs.openjdk.org/browse/JDK-8303509): Socket setTrafficClass does not work for IPv4 connections when IPv6 enabled. > > ### Change Log > When setting traffic class with IPv6 enabled, also set to IPv4 family. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/12852 From lancea at openjdk.org Fri Mar 17 16:48:28 2023 From: lancea at openjdk.org (Lance Andersen) Date: Fri, 17 Mar 2023 16:48:28 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v4] In-Reply-To: References: Message-ID: <--LXw0fy4u2KX4YqPmS4S4VlmXml9ocFFDBEM3pevNU=.f1879919-38d4-436a-a98d-b7f8dc9269dc@github.com> On Sat, 11 Mar 2023 07:28:14 GMT, Eirik Bjorsnos wrote: >> Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. >> >> Instead of calling out to the `zip` command line, we produce a small-sized Zip64 file in the test itself. This file has the features required to reproduce the ZipFileSystem issue, namely that the 'INFO-ZIP extended timestamp' field must come before the 'Zip64 extended information' field. (This would casue ZipFileSystem to read the LOC at position 0xFFFFFFFF). >> >> This speed up the test (from 50s to 3s on my Macbook Pro), saves 4GB disk space during builds removes a dependency on the `zip` command in OS/distros. >> >> Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. > > Eirik Bjorsnos has updated the pull request incrementally with one additional commit since the last revision: > > APPNOTE.txt actually specifies an 'unknown' tag, let's use that instead of 0x42 Thank you for your work here Eirik, I think we are close. I think I would include an additional entry in the zip just as an extra sanity check that we can navigate an additional entry It is also worth noting that the LOC will still contain an extra field with the unknown tag of 0x9902 while the CEN is being changed to use Zip64, 0x1 A few more minor comments below. I ran the current version of the test and it runs quickly on all platforms via mach5 Best Lance test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java line 173: > 171: ByteBuffer buffer = ByteBuffer.wrap(zip64).order(ByteOrder.LITTLE_ENDIAN); > 172: buffer.putShort(UNKNOWN_TAG); // Opaque tag makes ZipEntry.setExtra ignore it > 173: buffer.putShort((short) (zip64.length - 2 * Short.BYTES)); // Data size The length of the zip64 extra field is going to be 28. I would suggest making this a bit clearer and adding a field/constant which is set to 28 for clarity to make it easier for those who might look at this code later test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java line 218: > 216: > 217: // Update the Zip64 field with the real tag > 218: buffer.putShort((short) 0x1); // Tag for Zip64 I would probably make 0x1 a constant like the unknown tag. It would be good to beef up the comment as the byte array written by ZipEntry::setExtra is set to the unknown tag for the Tag and the length is set to 28 which includes all of the Zip64 extra fields ------------- PR: https://git.openjdk.org/jdk/pull/12979 From bpb at openjdk.org Fri Mar 17 17:01:02 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 17 Mar 2023 17:01:02 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v19] In-Reply-To: References: Message-ID: On Sun, 12 Mar 2023 16:31:39 GMT, Markus KARG wrote: >> This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. > > Markus KARG has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: > > - updated copyright > - Implementing 8278268 > > Signed-off-by: Markus Karg > So can you please briefly **confirm** that I _shall not_ merge `TransferTo2` into `TransferTo` as _Lance_ proposed (hence `TransferTo2` will not be a _separate_ class anymore) but **instead** I _only shall_ abstract `testNullPointerException` and `testStreamContents` as _you_ proposed (hence keep `TransferTo2` as a _separate_ class)? Thanks. That is fine for now. It will be possible to revisit merging at a later time if it appears reasonable. > > ...I think the two tests `*_2GB*` should remain separate as they likely each have a sufficiently long run time. > > I confirm that we had intentionally kept these two tests in separate classes due to the long run time of each, so I agree to keep them _as separate_ classes, but I will reduce _duplicate_ code. Sounds good. > So can you please briefly **confirm** that I am correct with this objection and the 2GB files _should not_ be renamed? That is fine. ------------- PR: https://git.openjdk.org/jdk/pull/6711 From duke at openjdk.org Fri Mar 17 18:36:02 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 17 Mar 2023 18:36:02 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v5] In-Reply-To: References: Message-ID: > Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. > > Instead of calling out to the `zip` command line, we produce a small-sized Zip64 file in the test itself. This file has the features required to reproduce the ZipFileSystem issue, namely that the 'INFO-ZIP extended timestamp' field must come before the 'Zip64 extended information' field. (This would casue ZipFileSystem to read the LOC at position 0xFFFFFFFF). > > This speed up the test (from 50s to 3s on my Macbook Pro), saves 4GB disk space during builds removes a dependency on the `zip` command in OS/distros. > > Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. Eirik Bjorsnos has updated the pull request incrementally with one additional commit since the last revision: Add extra entry as a sanity check. Extract some constants for sizes and extract method makeOpaqueExtraField to allow some more in-depth commenting. Move extra field processing until after the LOC headers are written, such that this information only occurs in the CEN ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12979/files - new: https://git.openjdk.org/jdk/pull/12979/files/22eba5c3..2fc841df Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12979&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12979&range=03-04 Stats: 72 lines in 1 file changed: 48 ins; 7 del; 17 mod Patch: https://git.openjdk.org/jdk/pull/12979.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12979/head:pull/12979 PR: https://git.openjdk.org/jdk/pull/12979 From duke at openjdk.org Fri Mar 17 19:26:44 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 17 Mar 2023 19:26:44 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v5] In-Reply-To: References: Message-ID: On Fri, 17 Mar 2023 18:36:02 GMT, Eirik Bjorsnos wrote: >> Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. >> >> Instead of calling out to the `zip` command line, we produce a small-sized Zip64 file in the test itself. This file has the features required to reproduce the ZipFileSystem issue, namely that the 'INFO-ZIP extended timestamp' field must come before the 'Zip64 extended information' field. (This would casue ZipFileSystem to read the LOC at position 0xFFFFFFFF). >> >> This speed up the test (from 50s to 3s on my Macbook Pro), saves 4GB disk space during builds removes a dependency on the `zip` command in OS/distros. >> >> Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. > > Eirik Bjorsnos has updated the pull request incrementally with one additional commit since the last revision: > > Add extra entry as a sanity check. Extract some constants for sizes and extract method makeOpaqueExtraField to allow some more in-depth commenting. Move extra field processing until after the LOC headers are written, such that this information only occurs in the CEN I ran the latest version of the test against a reverted ZipFileSystem to confirmed that it still catches the regression it was made for. ------------- PR: https://git.openjdk.org/jdk/pull/12979 From duke at openjdk.org Fri Mar 17 19:26:42 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 17 Mar 2023 19:26:42 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v4] In-Reply-To: <--LXw0fy4u2KX4YqPmS4S4VlmXml9ocFFDBEM3pevNU=.f1879919-38d4-436a-a98d-b7f8dc9269dc@github.com> References: <--LXw0fy4u2KX4YqPmS4S4VlmXml9ocFFDBEM3pevNU=.f1879919-38d4-436a-a98d-b7f8dc9269dc@github.com> Message-ID: On Fri, 17 Mar 2023 16:45:40 GMT, Lance Andersen wrote: > I think I would include an additional entry in the zip just as an extra sanity check that we can navigate an additional entry Added an additional entry as suggested and checked that is was navigated. > It is also worth noting that the LOC will still contain an extra field with the unknown tag of 0x9902 while the CEN is being changed to use Zip64, 0x1 This stray opaque extra field should not be any issue, but just in case, I moved the ZipFIle.setExtra until after the LOC is written, so now only the CEN will have the Zip64 extra field. > The length of the zip64 extra field is going to be 28. I would suggest making this a bit clearer and adding a field/constant which is set to 28 for clarity to make it easier for those who might look at this code later I believe this should be well-described now between the constants and the `makeOpaqueExtraField` method. > I would probably make 0x1 a constant like the unknown tag. Extracted a constant with a comment. > It would be good to beef up the comment as the byte array written by ZipEntry::setExtra is set to the unknown tag for the Tag and the length is set to 28 which includes all of the Zip64 extra fields Opted to extract a method?to create the opaque extra field since that allows for more comments and declutters the calling method. ------------- PR: https://git.openjdk.org/jdk/pull/12979 From duke at openjdk.org Fri Mar 17 19:32:58 2023 From: duke at openjdk.org (Eirik Bjorsnos) Date: Fri, 17 Mar 2023 19:32:58 GMT Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v5] In-Reply-To: References: Message-ID: On Fri, 17 Mar 2023 18:36:02 GMT, Eirik Bjorsnos wrote: >> Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. >> >> Instead of calling out to the `zip` command line, we produce a small-sized Zip64 file in the test itself. This file has the features required to reproduce the ZipFileSystem issue, namely that the 'INFO-ZIP extended timestamp' field must come before the 'Zip64 extended information' field. (This would casue ZipFileSystem to read the LOC at position 0xFFFFFFFF). >> >> This speed up the test (from 50s to 3s on my Macbook Pro), saves 4GB disk space during builds removes a dependency on the `zip` command in OS/distros. >> >> Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. > > Eirik Bjorsnos has updated the pull request incrementally with one additional commit since the last revision: > > Add extra entry as a sanity check. Extract some constants for sizes and extract method makeOpaqueExtraField to allow some more in-depth commenting. Move extra field processing until after the LOC headers are written, such that this information only occurs in the CEN Something worth mentioning: This test currently does not check that the last modified time stamp is actually the one read from the LOC and not from the CEN. And there is a discrepancey here, `ZipFileSystem` reads it from the LOC, while `ZipFile` reads it from the CEN. We could test this by setting different last modified timestamps in the LOC and CEN and use assertEquals to verify that the expected time stamp is parsed. @LanceAndersen Do you think this is worth pursuing in this PR? I have a sketch ready, but I fear will clutter the test with too many concerns. Perhaps better to test in a separate test which could be independent from Zip64 and local header offset concerns? ------------- PR: https://git.openjdk.org/jdk/pull/12979 From lance.andersen at oracle.com Fri Mar 17 19:59:57 2023 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 17 Mar 2023 19:59:57 +0000 Subject: RFR: 8303972: Make test/jdk/jdk/nio/zipfs/TestLocOffsetFromZip64EF.java independent of the zip command line [v5] In-Reply-To: References: Message-ID: <47649A02-DE4F-4C19-83B9-9D42123502E8@oracle.com> On Mar 17, 2023, at 3:32 PM, Eirik Bjorsnos > wrote: On Fri, 17 Mar 2023 18:36:02 GMT, Eirik Bjorsnos > wrote: Please review this PR which brings the currenly problem-listed test TestLocOffsetFromZip64EF back to life. Instead of calling out to the `zip` command line, we produce a small-sized Zip64 file in the test itself. This file has the features required to reproduce the ZipFileSystem issue, namely that the 'INFO-ZIP extended timestamp' field must come before the 'Zip64 extended information' field. (This would casue ZipFileSystem to read the LOC at position 0xFFFFFFFF). This speed up the test (from 50s to 3s on my Macbook Pro), saves 4GB disk space during builds removes a dependency on the `zip` command in OS/distros. Seee [JDK-8301183](https://bugs.openjdk.org/browse/JDK-8301183) for details on the problem-listing. Eirik Bjorsnos has updated the pull request incrementally with one additional commit since the last revision: Add extra entry as a sanity check. Extract some constants for sizes and extract method makeOpaqueExtraField to allow some more in-depth commenting. Move extra field processing until after the LOC headers are written, such that this information only occurs in the CEN Something worth mentioning: This test currently does not check that the last modified time stamp is actually the one read from the LOC and not from the CEN. And there is a discrepancey here, `ZipFileSystem` reads it from the LOC, while `ZipFile` reads it from the CEN. We could test this by setting different last modified timestamps in the LOC and CEN and use assertEquals to verify that the expected time stamp is parsed. @LanceAndersen Do you think this is worth pursuing in this PR? I have a sketch ready, but I fear will clutter the test with too many concerns. Perhaps better to test in a separate test which could be independent from Zip64 and local header offset concerns? Hi Eirik, Please treat the timestamp issue above separately so we are not morphing issues. Best Lance ------------- PR: https://git.openjdk.org/jdk/pull/12979 [cid:E1C4E2F0-ECD0-4C9D-ADB4-B16CA7BCB7FC at home] Lance Andersen | Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: oracle_sig_logo.gif URL: From duke at openjdk.org Sat Mar 18 15:03:29 2023 From: duke at openjdk.org (Markus KARG) Date: Sat, 18 Mar 2023 15:03:29 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v20] In-Reply-To: References: Message-ID: <6DitcTXBQwZGlxvPz0NXWo7CEFkrNcM0ysh54-dEPLA=.3f0a6692-d0d1-4854-9bc8-bc918af70f2c@github.com> > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with two additional commits since the last revision: - Abstracted assertStreamContents - Abstracted assertNullPointerException ------------- Changes: - all: https://git.openjdk.org/jdk/pull/6711/files - new: https://git.openjdk.org/jdk/pull/6711/files/50146a9d..6d3e1835 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=6711&range=19 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=6711&range=18-19 Stats: 114 lines in 3 files changed: 48 ins; 62 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.org/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.org/jdk/pull/6711 From duke at openjdk.org Sat Mar 18 16:35:15 2023 From: duke at openjdk.org (Markus KARG) Date: Sat, 18 Mar 2023 16:35:15 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v21] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with one additional commit since the last revision: Abstracted testMoreThanTwoGB ------------- Changes: - all: https://git.openjdk.org/jdk/pull/6711/files - new: https://git.openjdk.org/jdk/pull/6711/files/6d3e1835..fb198503 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=6711&range=20 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=6711&range=19-20 Stats: 187 lines in 3 files changed: 68 ins; 77 del; 42 mod Patch: https://git.openjdk.org/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.org/jdk pull/6711/head:pull/6711 PR: https://git.openjdk.org/jdk/pull/6711 From duke at openjdk.org Sat Mar 18 16:35:17 2023 From: duke at openjdk.org (Markus KARG) Date: Sat, 18 Mar 2023 16:35:17 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v19] In-Reply-To: References: Message-ID: <8fn-fpHeShp4CQ0O1KNDaNiUsmCivncoOunvwVL7Y2o=.5c564bf5-8207-4e9a-bd14-b620ceb298b8@github.com> On Fri, 17 Mar 2023 16:58:00 GMT, Brian Burkhalter wrote: >> Markus KARG has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - updated copyright >> - Implementing 8278268 >> >> Signed-off-by: Markus Karg > >> So can you please briefly **confirm** that I _shall not_ merge `TransferTo2` into `TransferTo` as _Lance_ proposed (hence `TransferTo2` will not be a _separate_ class anymore) but **instead** I _only shall_ abstract `testNullPointerException` and `testStreamContents` as _you_ proposed (hence keep `TransferTo2` as a _separate_ class)? Thanks. > > That is fine for now. It will be possible to revisit merging at a later time if it appears reasonable. > >> > ...I think the two tests `*_2GB*` should remain separate as they likely each have a sufficiently long run time. >> >> I confirm that we had intentionally kept these two tests in separate classes due to the long run time of each, so I agree to keep them _as separate_ classes, but I will reduce _duplicate_ code. > > Sounds good. > >> So can you please briefly **confirm** that I am correct with this objection and the 2GB files _should not_ be renamed? > > That is fine. @bplb I am done with the requested changes. There is no duplicated code anymore. :-) ------------- PR: https://git.openjdk.org/jdk/pull/6711 From alanb at openjdk.org Sun Mar 19 19:04:22 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 19 Mar 2023 19:04:22 GMT Subject: RFR: 8303260: (fc) FileChannel::transferFrom should support position > size() [v2] In-Reply-To: References: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> Message-ID: On Wed, 15 Mar 2023 21:49:15 GMT, Brian Burkhalter wrote: >> Enhance `FileChannel::transferFrom` to extend the channel's file if `position > size()`. This is consistent with `FileChannel::write(ByteBuffer, long)`. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8303260: Convert test to JUnit 5 test/jdk/java/nio/channels/FileChannel/TransferFromExtend.java line 98: > 96: @MethodSource("fastParamProvider") > 97: void fromFast(long initialPosition, int bufSize, long offset) > 98: throws IOException { Aligning the "throws IOException {" with the method body makes it hard to read, can you move this to the previous line? test/jdk/java/nio/channels/FileChannel/TransferFromExtend.java line 104: > 102: byte[] bytes = new byte[bufSize]; > 103: RND.nextBytes(bytes); > 104: src.write(ByteBuffer.wrap(bytes), 0); Return value from write probably needs to be checked as a short write is possible in some cases. test/jdk/java/nio/channels/FileChannel/TransferFromExtend.java line 115: > 113: @ParameterizedTest > 114: @MethodSource("slowParamProvider") > 115: void fromSlow(long initialPosition, int bufSize, long offset) A suggestion here is to rename this to fromReadingByteChannel to make a bit clearer. test/jdk/java/nio/channels/FileChannel/TransferFromExtend.java line 135: > 133: * > 134: * @throws RuntimeException if an unexpected number of bytes is transferred > 135: * or the transferred values are not as expected This is a JUnit tests so I assume it should use the assertXXX methods rather than throwing RuntimeException. test/jdk/java/nio/channels/FileChannel/TransferFromExtend.java line 144: > 142: READ, WRITE)) { > 143: target.position(initialPos); > 144: target.write(ByteBuffer.wrap(new byte[] {(byte)42})); I assume you should assertEquals that write return 1 here. test/jdk/java/nio/channels/FileChannel/TransferFromExtend.java line 149: > 147: long transferred = target.transferFrom(src, position, count); > 148: if (transferred != count) > 149: throw new RuntimeException(transferred + " != " + count); transferFrom is allowed to return a result < count. There are cases where this can happen in the JDK implementation so I think you just want to check that transferred > 0 and that compare that portion of the file, right? ------------- PR: https://git.openjdk.org/jdk/pull/12797 From bpb at openjdk.org Mon Mar 20 16:48:17 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 20 Mar 2023 16:48:17 GMT Subject: RFR: 8303260: (fc) FileChannel::transferFrom should support position > size() [v2] In-Reply-To: References: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> Message-ID: On Sun, 19 Mar 2023 07:30:09 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8303260: Convert test to JUnit 5 > > test/jdk/java/nio/channels/FileChannel/TransferFromExtend.java line 98: > >> 96: @MethodSource("fastParamProvider") >> 97: void fromFast(long initialPosition, int bufSize, long offset) >> 98: throws IOException { > > Aligning the "throws IOException {" with the method body makes it hard to read, can you move this to the previous line? Changed in 085746fa9e7446c380ed08c951e52796915c1f53. > test/jdk/java/nio/channels/FileChannel/TransferFromExtend.java line 104: > >> 102: byte[] bytes = new byte[bufSize]; >> 103: RND.nextBytes(bytes); >> 104: src.write(ByteBuffer.wrap(bytes), 0); > > Return value from write probably needs to be checked as a short write is possible in some cases. Changed in 085746fa9e7446c380ed08c951e52796915c1f53. > test/jdk/java/nio/channels/FileChannel/TransferFromExtend.java line 115: > >> 113: @ParameterizedTest >> 114: @MethodSource("slowParamProvider") >> 115: void fromSlow(long initialPosition, int bufSize, long offset) > > A suggestion here is to rename this to fromReadingByteChannel to make a bit clearer. Changed in 085746fa9e7446c380ed08c951e52796915c1f53. > test/jdk/java/nio/channels/FileChannel/TransferFromExtend.java line 135: > >> 133: * >> 134: * @throws RuntimeException if an unexpected number of bytes is transferred >> 135: * or the transferred values are not as expected > > This is a JUnit tests so I assume it should use the assertXXX methods rather than throwing RuntimeException. Changed in 085746fa9e7446c380ed08c951e52796915c1f53. > test/jdk/java/nio/channels/FileChannel/TransferFromExtend.java line 144: > >> 142: READ, WRITE)) { >> 143: target.position(initialPos); >> 144: target.write(ByteBuffer.wrap(new byte[] {(byte)42})); > > I assume you should assertEquals that write return 1 here. So changed in 085746fa9e7446c380ed08c951e52796915c1f53. > test/jdk/java/nio/channels/FileChannel/TransferFromExtend.java line 149: > >> 147: long transferred = target.transferFrom(src, position, count); >> 148: if (transferred != count) >> 149: throw new RuntimeException(transferred + " != " + count); > > transferFrom is allowed to return a result < count. There are cases where this can happen in the JDK implementation so I think you just want to check that transferred > 0 and that compare that portion of the file, right? Changed in 085746fa9e7446c380ed08c951e52796915c1f53. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/12797#discussion_r1142409486 PR Review Comment: https://git.openjdk.org/jdk/pull/12797#discussion_r1142409943 PR Review Comment: https://git.openjdk.org/jdk/pull/12797#discussion_r1142409776 PR Review Comment: https://git.openjdk.org/jdk/pull/12797#discussion_r1142408864 PR Review Comment: https://git.openjdk.org/jdk/pull/12797#discussion_r1142408599 PR Review Comment: https://git.openjdk.org/jdk/pull/12797#discussion_r1142409228 From bpb at openjdk.org Mon Mar 20 16:47:06 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 20 Mar 2023 16:47:06 GMT Subject: RFR: 8303260: (fc) FileChannel::transferFrom should support position > size() [v3] In-Reply-To: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> References: <3K-UVcqh0FsqxNOG5w6mFgyr56_fnD-id2_nRUbOnI0=.f3608ba3-057f-4726-83a9-0a4e509e7844@github.com> Message-ID: > Enhance `FileChannel::transferFrom` to extend the channel's file if `position > size()`. This is consistent with `FileChannel::write(ByteBuffer, long)`. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8303260: Address reviewer comments on test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12797/files - new: https://git.openjdk.org/jdk/pull/12797/files/0639c239..085746fa Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12797&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12797&range=01-02 Stats: 23 lines in 1 file changed: 8 ins; 4 del; 11 mod Patch: https://git.openjdk.org/jdk/pull/12797.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/12797/head:pull/12797 PR: https://git.openjdk.org/jdk/pull/12797 From pminborg at openjdk.org Mon Mar 20 18:12:39 2023 From: pminborg at openjdk.org (Per Minborg) Date: Mon, 20 Mar 2023 18:12:39 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) Message-ID: API changes for the FFM API (third preview) Specdiff: https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html Javadoc: https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html ------------- Commit messages: - Update after first round of comments - Remove extra line - Remove Panama-specific content - Generate delta Changes: https://git.openjdk.org/jdk/pull/13079/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8304265 Stats: 13601 lines in 269 files changed: 5644 ins; 5999 del; 1958 mod Patch: https://git.openjdk.org/jdk/pull/13079.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13079/head:pull/13079 PR: https://git.openjdk.org/jdk/pull/13079 From mcimadamore at openjdk.org Mon Mar 20 18:12:41 2023 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Mon, 20 Mar 2023 18:12:41 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) In-Reply-To: References: Message-ID: On Fri, 17 Mar 2023 15:42:56 GMT, Per Minborg wrote: > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html Here are the main API changes introduced in this round: * `SegmentScope` has been simplified into a pure lifetime abstraction and moved into a nested class `MemorySegment.Scope`. All segments have a scope (e.g. the segment lifetime), which is usually the scope of some `Arena`. All the factory methods in `SegmentScope` to create non-closeable lifetimes have been moved to `Arena` (e.g. `Arena.ofAuto` and `Arena.global`). This leads to a simplified API, which still allows to build custom arenas via simple delegation, but, at the same time, allows clients to use arenas with minimal indirections (e.g. `arena.scope()` is no longer needed in many places). Some factory names in `Arena` were also updated (e.g. from `openConfined` to `ofConfined`). * `ValueLayout::OfAddress` has been moved to a toplevel class `AddressLayout`. Also, the method to obtain an address layout of unbounded size (`OfAddress::asUnbounded`) has been changed, so that it now takes the layout of the target region of memory pointed to by the address (`AddressLayout::withTargetLayout`). * A new *layout path* is provided to dereference an address layout. This allows memory segment var handle to deal with complex dereference expressions like `*(a[10].x).y`. * A new linker implementation, namely the *fallback linker* has been added. This linker is based on `libffi` and provides a very easy way to add support for `Linker` API, even in platforms that have limited functionalities (such as [zero](https://openjdk.org/projects/zero/)). * The `VaList` interface has been dropped. Unfortunately, the behavior of `va_list` is hopelessly platform specific, and we could also never make full use of it in the `jextract` tool, given that parsing support `va_list` is very limited in `libclang`. * The API for unsafely attaching spatial/temporal bounds to an unsafe memory segment has been improved and streamlined. The `MemorySegment::ofAddress` method is now a single, unrestricted method which turns a long address into a native memory segment whose base address is the provided address. The returned segment has a scope that is always alive, and has zero-length. To resize, or add new temporal bounds to an existing segments, clients can use the new `MemorySegment::reinterpret` methods. The logic for attaching a cleanup action to a memory segment has also been updated: now the cleanup action will take as input a shallow copy of the memory segment, in a scope that is always alive, so that clients can pass that copy to other functions in order to perform custom cleanup. * We have made some changes and simplfications to the way in which runtime values such as `errno` are preserved, The `CapturedCallState` interface has been removed. Instead, there is a way to obtain a group layout of all the values that can be saved, given the platform in which the linker runs. Clients can query the layout, e.g. obtaining names for the values to be saved, and then create a linker option which lists all the name of the values to be saved. * We have added support for *trivial* (or *leaf*) calls - that is native calls whose execution completes very quickly. This option might be useful when calling functions whose total execution time is comparable to that of the overhead of the change of the thread state from Java to native (in JNI, such calls are handled using *critical JNI*). ------------- PR Comment: https://git.openjdk.org/jdk/pull/13079#issuecomment-1476648707 From duke at openjdk.org Mon Mar 20 18:12:45 2023 From: duke at openjdk.org (ExE Boss) Date: Mon, 20 Mar 2023 18:12:45 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) In-Reply-To: References: Message-ID: On Fri, 17 Mar 2023 15:42:56 GMT, Per Minborg wrote: > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html src/java.base/share/classes/java/lang/foreign/AddressLayout.java line 59: > 57: */ > 58: @PreviewFeature(feature = PreviewFeature.Feature.FOREIGN) > 59: sealed public interface AddressLayout extends ValueLayout permits ValueLayouts.OfAddressImpl { This should?match other sealed?interfaces: Suggestion: public sealed interface AddressLayout extends ValueLayout permits ValueLayouts.OfAddressImpl { src/java.base/share/classes/java/lang/foreign/Linker.java line 578: > 576: * Execution state is captured by a downcall method handle on invocation, by writing it > 577: * to a native segment provided by the user to the downcall method handle. > 578: * For this purpose, a downcall method handle linked with the this Suggestion: * For this purpose, a downcall method handle linked with this ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1140677384 PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1140679387 From erikj at openjdk.org Mon Mar 20 19:32:31 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 20 Mar 2023 19:32:31 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) In-Reply-To: References: Message-ID: On Fri, 17 Mar 2023 15:42:56 GMT, Per Minborg wrote: > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html Build changes look ok. ------------- Marked as reviewed by erikj (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/13079#pullrequestreview-1349265135 From bpb at openjdk.org Mon Mar 20 21:03:56 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 20 Mar 2023 21:03:56 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v21] In-Reply-To: References: Message-ID: <4J5-MPn4DiG3ZB4rDGy9725Qb7VrqQJIjb-mW7cI520=.c75c7030-834c-4ed4-93b0-03bb18231785@github.com> On Sat, 18 Mar 2023 16:35:15 GMT, Markus KARG wrote: >> This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. > > Markus KARG has updated the pull request incrementally with one additional commit since the last revision: > > Abstracted testMoreThanTwoGB test/jdk/java/nio/channels/Channels/TransferToBase.java line 222: > 220: byte[] rndBytes = createRandomBytes(BYTES_PER_WRITE, 0); > 221: ByteBuffer src = ByteBuffer.wrap(rndBytes); > 222: fc.write(src); Should this code perhaps allow for a short write, i.e., `FileChannel::write` returning a value less than `rndBytes.length`? For example, something like this: long pos = initPos; while (pos < BYTES_WRITTEN) { int len = Math.toIntExact(BYTES_WRITTEN - pos); if (len > BYTES_PER_WRITE) len = BYTES_PER_WRITE; byte[] rndBytes = createRandomBytes(len, 0); ByteBuffer src = ByteBuffer.wrap(rndBytes); pos += fc.write(src); } ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/6711#discussion_r1142672553 From psandoz at openjdk.org Mon Mar 20 23:33:41 2023 From: psandoz at openjdk.org (Paul Sandoz) Date: Mon, 20 Mar 2023 23:33:41 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) In-Reply-To: References: Message-ID: On Fri, 17 Mar 2023 15:42:56 GMT, Per Minborg wrote: > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html src/java.base/share/classes/java/lang/foreign/AddressLayout.java line 93: > 91: * @apiNote > 92: * This method can also be used to create an address layout which, when used, creates native memory > 93: * segments with maximal size (e.g. {@linkplain Long#MAX_VALUE}. This can be done by using a target sequence Suggestion: * segments with maximal size (e.g. {@linkplain Long#MAX_VALUE}). This can be done by using a target sequence ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1142769618 From psandoz at openjdk.org Tue Mar 21 00:38:49 2023 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 21 Mar 2023 00:38:49 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) In-Reply-To: References: Message-ID: On Fri, 17 Mar 2023 15:42:56 GMT, Per Minborg wrote: > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html src/java.base/share/classes/java/lang/foreign/Linker.java line 479: > 477: * Otherwise, the invocation throws {@link WrongThreadException}; and > 478: *
  • {@code A} is kept alive during the invocation. For instance, if {@code A} has been obtained using a > 479: * {@linkplain Arena#ofConfined() confined arena}, any attempt to {@linkplain Arena#close() close} Is that correct? Do you mean a shared arena? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1142798464 From psandoz at openjdk.org Tue Mar 21 00:57:17 2023 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 21 Mar 2023 00:57:17 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) In-Reply-To: References: Message-ID: On Fri, 17 Mar 2023 15:42:56 GMT, Per Minborg wrote: > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html src/java.base/share/classes/java/lang/foreign/Linker.java line 609: > 607: * @see #captureStateLayout() > 608: */ > 609: static Option captureCallState(String... capturedState) { What if a name is not recognized? Throw IAE? src/java.base/share/classes/java/lang/foreign/Linker.java line 621: > 619: * to a downcall handle linked with {@link #captureCallState(String...)}} > 620: * > 621: * @see #captureCallState(String...) How does a caller know what the structure may contain? Should we document the platform specific structures? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1142804110 PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1142804836 From psandoz at openjdk.org Tue Mar 21 01:17:42 2023 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 21 Mar 2023 01:17:42 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) In-Reply-To: References: Message-ID: On Fri, 17 Mar 2023 15:42:56 GMT, Per Minborg wrote: > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html src/java.base/share/classes/java/lang/foreign/SymbolLookup.java line 56: > 54: *
  • It can be passed to an existing {@linkplain Linker#downcallHandle(FunctionDescriptor, Linker.Option...) downcall method handle}, as an argument to the underlying foreign function.
  • > 55: *
  • It can be {@linkplain MemorySegment#set(AddressLayout, long, MemorySegment) stored} inside another memory segment.
  • > 56: *
  • It can be used to access the region of memory backing a global variable (this might require Suggestion: *
  • It can be used to access the region of memory backing a global variable (this requires ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1142812300 From pminborg at openjdk.org Tue Mar 21 07:50:22 2023 From: pminborg at openjdk.org (Per Minborg) Date: Tue, 21 Mar 2023 07:50:22 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v2] In-Reply-To: References: Message-ID: > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html Per Minborg has updated the pull request incrementally with two additional commits since the last revision: - Update src/java.base/share/classes/java/lang/foreign/SymbolLookup.java Co-authored-by: Paul Sandoz - Update src/java.base/share/classes/java/lang/foreign/AddressLayout.java Co-authored-by: Paul Sandoz ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13079/files - new: https://git.openjdk.org/jdk/pull/13079/files/bb2f4438..7a78948a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=00-01 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/13079.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13079/head:pull/13079 PR: https://git.openjdk.org/jdk/pull/13079 From pminborg at openjdk.org Tue Mar 21 08:04:25 2023 From: pminborg at openjdk.org (Per Minborg) Date: Tue, 21 Mar 2023 08:04:25 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v3] In-Reply-To: References: Message-ID: > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html Per Minborg has updated the pull request incrementally with one additional commit since the last revision: Remove MemoryInspection classes ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13079/files - new: https://git.openjdk.org/jdk/pull/13079/files/7a78948a..a71518c4 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=01-02 Stats: 684 lines in 3 files changed: 0 ins; 684 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/13079.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13079/head:pull/13079 PR: https://git.openjdk.org/jdk/pull/13079 From pminborg at openjdk.org Tue Mar 21 08:33:26 2023 From: pminborg at openjdk.org (Per Minborg) Date: Tue, 21 Mar 2023 08:33:26 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v4] In-Reply-To: References: Message-ID: <_HwSyb1bsHNIJpGKdznMZR_HfWIU9VeAl1etn4miKGQ=.8397676f-1e24-4a2e-bcfa-ed44fb8a515b@github.com> > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html Per Minborg has updated the pull request incrementally with one additional commit since the last revision: Improve Linker javadocs ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13079/files - new: https://git.openjdk.org/jdk/pull/13079/files/a71518c4..4626a54e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=02-03 Stats: 4 lines in 1 file changed: 2 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/13079.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13079/head:pull/13079 PR: https://git.openjdk.org/jdk/pull/13079 From pminborg at openjdk.org Tue Mar 21 08:33:29 2023 From: pminborg at openjdk.org (Per Minborg) Date: Tue, 21 Mar 2023 08:33:29 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v4] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 00:52:01 GMT, Paul Sandoz wrote: >> Per Minborg has updated the pull request incrementally with one additional commit since the last revision: >> >> Improve Linker javadocs > > src/java.base/share/classes/java/lang/foreign/Linker.java line 609: > >> 607: * @see #captureStateLayout() >> 608: */ >> 609: static Option captureCallState(String... capturedState) { > > What if a name is not recognized? Throw IAE? Added ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1143024389 From pminborg at openjdk.org Tue Mar 21 09:02:29 2023 From: pminborg at openjdk.org (Per Minborg) Date: Tue, 21 Mar 2023 09:02:29 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html Per Minborg has updated the pull request incrementally with one additional commit since the last revision: Add example for Option::captureStateLayout ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13079/files - new: https://git.openjdk.org/jdk/pull/13079/files/4626a54e..21ef0607 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=03-04 Stats: 12 lines in 1 file changed: 12 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/13079.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13079/head:pull/13079 PR: https://git.openjdk.org/jdk/pull/13079 From pminborg at openjdk.org Tue Mar 21 09:02:33 2023 From: pminborg at openjdk.org (Per Minborg) Date: Tue, 21 Mar 2023 09:02:33 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 00:54:10 GMT, Paul Sandoz wrote: >> Per Minborg has updated the pull request incrementally with one additional commit since the last revision: >> >> Add example for Option::captureStateLayout > > src/java.base/share/classes/java/lang/foreign/Linker.java line 621: > >> 619: * to a downcall handle linked with {@link #captureCallState(String...)}} >> 620: * >> 621: * @see #captureCallState(String...) > > How does a caller know what the structure may contain? Should we document the platform specific structures? I've added an example of how to print the platform-dependent structure. Should we document anyhow? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1143055969 From stsypanov at openjdk.org Tue Mar 21 09:07:59 2023 From: stsypanov at openjdk.org (Sergey Tsypanov) Date: Tue, 21 Mar 2023 09:07:59 GMT Subject: RFR: 8304591: UnixPath.stringValue shouldn't be volatile Message-ID: `UnixPath.stringValue` can be computed by multiple threads safely as the actual value is the same, so the race is benign in this case. ------------- Commit messages: - Fix build - UnixPath.stringValue shouldn't be volatile Changes: https://git.openjdk.org/jdk/pull/13115/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=13115&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8304591 Stats: 9 lines in 1 file changed: 3 ins; 0 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/13115.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13115/head:pull/13115 PR: https://git.openjdk.org/jdk/pull/13115 From alanb at openjdk.org Tue Mar 21 09:11:56 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 21 Mar 2023 09:11:56 GMT Subject: RFR: 8304591: UnixPath.stringValue shouldn't be volatile In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 09:00:19 GMT, Sergey Tsypanov wrote: > `UnixPath.stringValue` can be computed by multiple threads safely as the actual value is the same, so the race is benign in this case. Can you rename this PR to align with the JBS issue? src/java.base/unix/classes/sun/nio/fs/UnixPath.java line 48: > 46: import static sun.nio.fs.UnixConstants.O_RDONLY; > 47: import static sun.nio.fs.UnixNativeDispatcher.open; > 48: import static sun.nio.fs.UnixNativeDispatcher.realpath; Please revert this change. ------------- PR Comment: https://git.openjdk.org/jdk/pull/13115#issuecomment-1477489271 PR Review Comment: https://git.openjdk.org/jdk/pull/13115#discussion_r1143068235 From stsypanov at openjdk.org Tue Mar 21 09:27:17 2023 From: stsypanov at openjdk.org (Sergey Tsypanov) Date: Tue, 21 Mar 2023 09:27:17 GMT Subject: RFR: 8304591: (fs) UnixPath.stringValue need not be volatile [v2] In-Reply-To: References: Message-ID: > `UnixPath.stringValue` can be computed by multiple threads safely as the actual value is the same, so the race is benign in this case. Sergey Tsypanov has updated the pull request incrementally with one additional commit since the last revision: Revert ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13115/files - new: https://git.openjdk.org/jdk/pull/13115/files/2cc5ee57..88cd6cfc Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13115&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13115&range=00-01 Stats: 4 lines in 1 file changed: 0 ins; 2 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/13115.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13115/head:pull/13115 PR: https://git.openjdk.org/jdk/pull/13115 From stsypanov at openjdk.org Tue Mar 21 09:27:18 2023 From: stsypanov at openjdk.org (Sergey Tsypanov) Date: Tue, 21 Mar 2023 09:27:18 GMT Subject: RFR: 8304591: (fs) UnixPath.stringValue need not be volatile In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 09:00:19 GMT, Sergey Tsypanov wrote: > `UnixPath.stringValue` can be computed by multiple threads safely as the actual value is the same, so the race is benign in this case. Renamed ------------- PR Comment: https://git.openjdk.org/jdk/pull/13115#issuecomment-1477503987 From stsypanov at openjdk.org Tue Mar 21 09:27:18 2023 From: stsypanov at openjdk.org (Sergey Tsypanov) Date: Tue, 21 Mar 2023 09:27:18 GMT Subject: RFR: 8304591: (fs) UnixPath.stringValue need not be volatile [v2] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 09:08:34 GMT, Alan Bateman wrote: >> Sergey Tsypanov has updated the pull request incrementally with one additional commit since the last revision: >> >> Revert > > src/java.base/unix/classes/sun/nio/fs/UnixPath.java line 48: > >> 46: import static sun.nio.fs.UnixConstants.O_RDONLY; >> 47: import static sun.nio.fs.UnixNativeDispatcher.open; >> 48: import static sun.nio.fs.UnixNativeDispatcher.realpath; > > Please revert this change. Done ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13115#discussion_r1143084473 From mcimadamore at openjdk.org Tue Mar 21 09:52:54 2023 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 21 Mar 2023 09:52:54 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 00:35:40 GMT, Paul Sandoz wrote: >> Per Minborg has updated the pull request incrementally with one additional commit since the last revision: >> >> Add example for Option::captureStateLayout > > src/java.base/share/classes/java/lang/foreign/Linker.java line 479: > >> 477: * Otherwise, the invocation throws {@link WrongThreadException}; and
  • >> 478: *
  • {@code A} is kept alive during the invocation. For instance, if {@code A} has been obtained using a >> 479: * {@linkplain Arena#ofConfined() confined arena}, any attempt to {@linkplain Arena#close() close} > > Is that correct? Do you mean a shared arena? The text is correct (you can still close a confined arena from inside an upcall), but I agree that perhaps using a shared arena might be a bit more intuitive. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1143118512 From mcimadamore at openjdk.org Tue Mar 21 09:56:53 2023 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 21 Mar 2023 09:56:53 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 08:57:42 GMT, Per Minborg wrote: >> src/java.base/share/classes/java/lang/foreign/Linker.java line 621: >> >>> 619: * to a downcall handle linked with {@link #captureCallState(String...)}} >>> 620: * >>> 621: * @see #captureCallState(String...) >> >> How does a caller know what the structure may contain? Should we document the platform specific structures? > > I've added an example of how to print the platform-dependent structure. Should we document anyhow? I'm not sure about this. Honestly, the example probably doesn't help much if somebody didn't get the idea of what the layout might be. I think it might be better to say something like, `on Windows/x64 the returned layout might be as follows...` and then you write the code to create the layout instance corresponding to the returned layout. But we have to be careful to make the text non-normative (as the set of values might change). ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1143125997 From mcimadamore at openjdk.org Tue Mar 21 10:01:53 2023 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 21 Mar 2023 10:01:53 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 09:02:29 GMT, Per Minborg wrote: >> API changes for the FFM API (third preview) >> >> Specdiff: >> https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html >> >> Javadoc: >> https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html > > Per Minborg has updated the pull request incrementally with one additional commit since the last revision: > > Add example for Option::captureStateLayout src/java.base/share/classes/java/lang/foreign/Linker.java line 480: > 478: * Otherwise, the invocation throws {@link WrongThreadException}; and
  • > 479: *
  • {@code A} is kept alive during the invocation. For instance, if {@code A} has been obtained using a > 480: * {@linkplain Arena#ofShared()} confined arena}, any attempt to {@linkplain Arena#close() close} the description of the link still says "confined" ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1143131392 From pminborg at openjdk.org Tue Mar 21 12:02:44 2023 From: pminborg at openjdk.org (Per Minborg) Date: Tue, 21 Mar 2023 12:02:44 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 09:54:16 GMT, Maurizio Cimadamore wrote: >> I've added an example of how to print the platform-dependent structure. Should we document anyhow? > > I'm not sure about this. Honestly, the example probably doesn't help much if somebody didn't get the idea of what the layout might be. I think it might be better to say something like, `on Windows/x64 the returned layout might be as follows...` and then you write the code to create the layout instance corresponding to the returned layout. But we have to be careful to make the text non-normative (as the set of values might change). What about adding something like this? Here is a list of valid names for some platforms: Linux: "errno" macOS: "errno" Windows: "GetLastError", "WSAGetLastError" and "errno" ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1143265595 From mcimadamore at openjdk.org Tue Mar 21 12:08:45 2023 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 21 Mar 2023 12:08:45 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: <5VeF6VvOJ94JGz-slAqtwU-mtnhdV4qSk2KsbqVy-x4=.47d9a1f7-b86b-4d07-8ab1-f9abbaf3e5ed@github.com> On Tue, 21 Mar 2023 11:58:28 GMT, Per Minborg wrote: >> I'm not sure about this. Honestly, the example probably doesn't help much if somebody didn't get the idea of what the layout might be. I think it might be better to say something like, `on Windows/x64 the returned layout might be as follows...` and then you write the code to create the layout instance corresponding to the returned layout. But we have to be careful to make the text non-normative (as the set of values might change). > > What about adding something like this? > > > Here is a list of valid names for some platforms: > Linux: > "errno" > macOS: > "errno" > Windows: > "GetLastError", "WSAGetLastError" and "errno" I'd prefer something more informal: "Examples of valid names are "errno" on Linux/x64, or "GetLastError" on Windows/x64". And maybe add that "The precise set of platform-dependent supported names can be queried using the returned group layout". ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1143272764 From mcimadamore at openjdk.org Tue Mar 21 12:17:43 2023 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 21 Mar 2023 12:17:43 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 00:54:10 GMT, Paul Sandoz wrote: >> Per Minborg has updated the pull request incrementally with one additional commit since the last revision: >> >> Add example for Option::captureStateLayout > > src/java.base/share/classes/java/lang/foreign/Linker.java line 621: > >> 619: * to a downcall handle linked with {@link #captureCallState(String...)}} >> 620: * >> 621: * @see #captureCallState(String...) > > How does a caller know what the structure may contain? Should we document the platform specific structures? Back to @PaulSandoz question - "how does the caller know what the structure contains?". The caller typically doesn't care too much about what the returned struct is. But it might care about which "values" can be captured. That said, the set of such interesting values, is not too surprising. As demonstrated in the example in the `Linker.capturedCallState` method, once the client knows the name (and "errno" is likely to be 90% case), everything else follows from there - as the layout can be used to obtain var handles for the required values. But, perhaps, now that I write this, I realize that what @PaulSandoz might _really_ be asking is: how do I know that e.g. the returned struct will not contain e.g. nested structs, sequences, or other non-sense. So we might specify (in a normative way) that the returned layout is a struct layout, whose member layouts are one or more value layouts (possibly with some added padding layouts). The names of the value layouts reflect the names of the values that can be captured. And _then_ we show an example of the layout we return for Windows/x64 (as that's more interesting). ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1143281164 From pminborg at openjdk.org Tue Mar 21 13:39:05 2023 From: pminborg at openjdk.org (Per Minborg) Date: Tue, 21 Mar 2023 13:39:05 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 09:02:29 GMT, Per Minborg wrote: >> API changes for the FFM API (third preview) >> >> Specdiff: >> https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html >> >> Javadoc: >> https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html > > Per Minborg has updated the pull request incrementally with one additional commit since the last revision: > > Add example for Option::captureStateLayout A review of all the copyright years shall be made in this PR. ------------- PR Comment: https://git.openjdk.org/jdk/pull/13079#issuecomment-1477849414 From bpb at openjdk.org Tue Mar 21 16:52:58 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 21 Mar 2023 16:52:58 GMT Subject: RFR: 4842457: (bf spec) Clarify meaning of "(optional operation)" Message-ID: Add some clarifying verbiage about optional operations to the type-specific buffer classes. ------------- Commit messages: - 4842457: (bf spec) Clarify meaning of "(optional operation)" Changes: https://git.openjdk.org/jdk/pull/13123/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=13123&range=00 Issue: https://bugs.openjdk.org/browse/JDK-4842457 Stats: 9 lines in 1 file changed: 9 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/13123.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13123/head:pull/13123 PR: https://git.openjdk.org/jdk/pull/13123 From alanb at openjdk.org Tue Mar 21 18:28:44 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 21 Mar 2023 18:28:44 GMT Subject: RFR: 4842457: (bf spec) Clarify meaning of "(optional operation)" In-Reply-To: References: Message-ID: <3PaX6kHzWJIHXgM_P0Hy-y36iYsdEMXImPnQ3kRwupc=.01cacfcb-9ac8-49e8-95c0-28b4f500ed60@github.com> On Tue, 21 Mar 2023 16:43:02 GMT, Brian Burkhalter wrote: > Add some clarifying verbiage about optional operations to the type-specific buffer classes. src/java.base/share/classes/java/nio/X-Buffer.java.template line 269: > 267: * however, instead throw an {@linkplain UnsupportedOperationException} if the > 268: * $Type$Buffer is not backed by an {@linkplain Buffer#hasArray accessible > 269: * $type$ array} irrespective of whether the $Type$Buffer is read-only. This mostly looks okay but I think it would be clearer to drop "however, instead" and put "irrespective ..." in parentheses. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13123#discussion_r1143835767 From alanb at openjdk.org Tue Mar 21 18:29:42 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 21 Mar 2023 18:29:42 GMT Subject: RFR: 8304591: (fs) UnixPath.stringValue need not be volatile [v2] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 09:27:17 GMT, Sergey Tsypanov wrote: >> `UnixPath.stringValue` can be computed by multiple threads safely as the actual value is the same, so the race is benign in this case. > > Sergey Tsypanov has updated the pull request incrementally with one additional commit since the last revision: > > Revert Marked as reviewed by alanb (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/13115#pullrequestreview-1351157281 From bpb at openjdk.org Tue Mar 21 19:19:43 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 21 Mar 2023 19:19:43 GMT Subject: RFR: 4842457: (bf spec) Clarify meaning of "(optional operation)" In-Reply-To: <3PaX6kHzWJIHXgM_P0Hy-y36iYsdEMXImPnQ3kRwupc=.01cacfcb-9ac8-49e8-95c0-28b4f500ed60@github.com> References: <3PaX6kHzWJIHXgM_P0Hy-y36iYsdEMXImPnQ3kRwupc=.01cacfcb-9ac8-49e8-95c0-28b4f500ed60@github.com> Message-ID: <7047fcqji3CNkvaAfl9dkZVI_7feLAmM7jkkBAzZa9A=.9a4316c7-4e23-48dc-8a42-89fabe80ecec@github.com> On Tue, 21 Mar 2023 18:26:10 GMT, Alan Bateman wrote: >> Add some clarifying verbiage about optional operations to the type-specific buffer classes. > > src/java.base/share/classes/java/nio/X-Buffer.java.template line 269: > >> 267: * however, instead throw an {@linkplain UnsupportedOperationException} if the >> 268: * $Type$Buffer is not backed by an {@linkplain Buffer#hasArray accessible >> 269: * $type$ array} irrespective of whether the $Type$Buffer is read-only. > > This mostly looks okay but I think it would be clearer to drop "however, instead" and put "irrespective ..." in parentheses. I thought "however, instead" a bit clunky as well; will update. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13123#discussion_r1143886924 From bpb at openjdk.org Tue Mar 21 19:22:41 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 21 Mar 2023 19:22:41 GMT Subject: RFR: 8304591: (fs) UnixPath.stringValue need not be volatile [v2] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 09:27:17 GMT, Sergey Tsypanov wrote: >> `UnixPath.stringValue` can be computed by multiple threads safely as the actual value is the same, so the race is benign in this case. > > Sergey Tsypanov has updated the pull request incrementally with one additional commit since the last revision: > > Revert Marked as reviewed by bpb (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/13115#pullrequestreview-1351236400 From bpb at openjdk.org Tue Mar 21 19:30:30 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 21 Mar 2023 19:30:30 GMT Subject: RFR: 4842457: (bf spec) Clarify meaning of "(optional operation)" [v2] In-Reply-To: References: Message-ID: > Add some clarifying verbiage about optional operations to the type-specific buffer classes. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 4842457: Refine verbiage ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13123/files - new: https://git.openjdk.org/jdk/pull/13123/files/3d18bd29..12e5fe15 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13123&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13123&range=00-01 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/13123.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13123/head:pull/13123 PR: https://git.openjdk.org/jdk/pull/13123 From bpb at openjdk.org Tue Mar 21 19:30:33 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 21 Mar 2023 19:30:33 GMT Subject: RFR: 4842457: (bf spec) Clarify meaning of "(optional operation)" [v2] In-Reply-To: <7047fcqji3CNkvaAfl9dkZVI_7feLAmM7jkkBAzZa9A=.9a4316c7-4e23-48dc-8a42-89fabe80ecec@github.com> References: <3PaX6kHzWJIHXgM_P0Hy-y36iYsdEMXImPnQ3kRwupc=.01cacfcb-9ac8-49e8-95c0-28b4f500ed60@github.com> <7047fcqji3CNkvaAfl9dkZVI_7feLAmM7jkkBAzZa9A=.9a4316c7-4e23-48dc-8a42-89fabe80ecec@github.com> Message-ID: On Tue, 21 Mar 2023 19:16:44 GMT, Brian Burkhalter wrote: >> src/java.base/share/classes/java/nio/X-Buffer.java.template line 269: >> >>> 267: * however, instead throw an {@linkplain UnsupportedOperationException} if the >>> 268: * $Type$Buffer is not backed by an {@linkplain Buffer#hasArray accessible >>> 269: * $type$ array} irrespective of whether the $Type$Buffer is read-only. >> >> This mostly looks okay but I think it would be clearer to drop "however, instead" and put "irrespective ..." in parentheses. > > I thought "however, instead" a bit clunky as well; will update. Refined in 12e5fe15237a5f5cebbaf2ed4e81a851074e3e5c. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13123#discussion_r1143895168 From bpb at openjdk.org Tue Mar 21 19:34:28 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 21 Mar 2023 19:34:28 GMT Subject: RFR: 4842457: (bf spec) Clarify meaning of "(optional operation)" [v3] In-Reply-To: References: Message-ID: > Add some clarifying verbiage about optional operations to the type-specific buffer classes. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 4842457: Remove vestigial comma; clean up ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13123/files - new: https://git.openjdk.org/jdk/pull/13123/files/12e5fe15..93febb09 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13123&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13123&range=01-02 Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/13123.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13123/head:pull/13123 PR: https://git.openjdk.org/jdk/pull/13123 From alanb at openjdk.org Tue Mar 21 20:17:41 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 21 Mar 2023 20:17:41 GMT Subject: RFR: 4842457: (bf spec) Clarify meaning of "(optional operation)" [v3] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 19:34:28 GMT, Brian Burkhalter wrote: >> Add some clarifying verbiage about optional operations to the type-specific buffer classes. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 4842457: Remove vestigial comma; clean up Marked as reviewed by alanb (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/13123#pullrequestreview-1351315477 From duke at openjdk.org Tue Mar 21 21:48:02 2023 From: duke at openjdk.org (Markus KARG) Date: Tue, 21 Mar 2023 21:48:02 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v21] In-Reply-To: <4J5-MPn4DiG3ZB4rDGy9725Qb7VrqQJIjb-mW7cI520=.c75c7030-834c-4ed4-93b0-03bb18231785@github.com> References: <4J5-MPn4DiG3ZB4rDGy9725Qb7VrqQJIjb-mW7cI520=.c75c7030-834c-4ed4-93b0-03bb18231785@github.com> Message-ID: On Mon, 20 Mar 2023 21:00:31 GMT, Brian Burkhalter wrote: >> Markus KARG has updated the pull request incrementally with one additional commit since the last revision: >> >> Abstracted testMoreThanTwoGB > > test/jdk/java/nio/channels/Channels/TransferToBase.java line 222: > >> 220: byte[] rndBytes = createRandomBytes(BYTES_PER_WRITE, 0); >> 221: ByteBuffer src = ByteBuffer.wrap(rndBytes); >> 222: fc.write(src); > > Should this code perhaps allow for a short write, i.e., `FileChannel::write` returning a value less than `rndBytes.length`? For example, something like this: > > long pos = initPos; > while (pos < BYTES_WRITTEN) { > int len = Math.toIntExact(BYTES_WRITTEN - pos); > if (len > BYTES_PER_WRITE) > len = BYTES_PER_WRITE; > byte[] rndBytes = createRandomBytes(len, 0); > ByteBuffer src = ByteBuffer.wrap(rndBytes); > pos += fc.write(src); > } I have incorporated your proposed code change. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/6711#discussion_r1144017216 From duke at openjdk.org Tue Mar 21 21:47:59 2023 From: duke at openjdk.org (Markus KARG) Date: Tue, 21 Mar 2023 21:47:59 GMT Subject: RFR: 8278268 - (ch) InputStream returned by Channels.newInputStream should have fast path for FileChannel targets [v22] In-Reply-To: References: Message-ID: > This sub-issue defines the work to be done to implement JDK-8265891 solely for the particular case of FileChannel.transferFrom(ReadableByteChannel), including special treatment of SelectableByteChannel. Markus KARG has updated the pull request incrementally with one additional commit since the last revision: Dealing with FileChannel::write returning a value less than rndBytes.length ------------- Changes: - all: https://git.openjdk.org/jdk/pull/6711/files - new: https://git.openjdk.org/jdk/pull/6711/files/fb198503..8b7425ba Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=6711&range=21 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=6711&range=20-21 Stats: 7 lines in 1 file changed: 3 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/6711.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/6711/head:pull/6711 PR: https://git.openjdk.org/jdk/pull/6711 From bpb at openjdk.org Tue Mar 21 23:38:28 2023 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 21 Mar 2023 23:38:28 GMT Subject: RFR: 8202110: (fs) Default file system provider should load UnixNativeDispatcher lazily Message-ID: <5oiN2L_4Epw_KePlEYYZm7qSG60RUSzG4HcJeBJE40U=.a40ea209-1529-41d6-94bb-f3cf2ba8c6c2@github.com> Initialize `needToResolveAgainstDefaultDirectory` instance variable lazily thereby deferring early use of `UnixNativeDispatcher` and hopefully loading `libnio`. ------------- Commit messages: - 8202110: (fs) Default file system provider should load UnixNativeDispatcher lazily Changes: https://git.openjdk.org/jdk/pull/13132/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=13132&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8202110 Stats: 53 lines in 1 file changed: 29 ins; 24 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/13132.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13132/head:pull/13132 PR: https://git.openjdk.org/jdk/pull/13132 From fjiang at openjdk.org Wed Mar 22 08:18:42 2023 From: fjiang at openjdk.org (Feilong Jiang) Date: Wed, 22 Mar 2023 08:18:42 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: <2VN0jPv3cwJuzneVkIBwHReDI-2zj0qBknbc_AeFy1k=.d306ef5f-c104-496b-b6fd-f404ffc33d39@github.com> On Tue, 21 Mar 2023 13:35:25 GMT, Per Minborg wrote: >> Per Minborg has updated the pull request incrementally with one additional commit since the last revision: >> >> Add example for Option::captureStateLayout > > A review of all the copyright years shall be made in this PR. Hi @minborg, looks like some changes were missed on riscv port. I've added these changes and submitted tests on linux-riscv. `jdk_foreign` still passed with release & fatdebug build. Could you please add these extra changes for riscv? Thanks. Here is the patch: [foreign_riscv_port_patch.txt](https://github.com/openjdk/jdk/files/11037700/foreign_riscv_port_patch.txt) ------------- PR Comment: https://git.openjdk.org/jdk/pull/13079#issuecomment-1479083052 From aturbanov at openjdk.org Wed Mar 22 09:18:50 2023 From: aturbanov at openjdk.org (Andrey Turbanov) Date: Wed, 22 Mar 2023 09:18:50 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 09:02:29 GMT, Per Minborg wrote: >> API changes for the FFM API (third preview) >> >> Specdiff: >> https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html >> >> Javadoc: >> https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html > > Per Minborg has updated the pull request incrementally with one additional commit since the last revision: > > Add example for Option::captureStateLayout src/java.base/share/classes/java/lang/foreign/Arena.java line 224: > 222: static Arena global() { > 223: class Holder { > 224: final static Arena GLOBAL = MemorySessionImpl.GLOBAL.asArena(); Nit: use blessed modifiers order Suggestion: static final Arena GLOBAL = MemorySessionImpl.GLOBAL.asArena(); src/java.base/share/classes/jdk/internal/foreign/abi/aarch64/windows/WindowsAArch64Linker.java line 58: > 56: @Override > 57: protected UpcallStubFactory arrangeUpcall(MethodType targetType, FunctionDescriptor function, LinkerOptions options) { > 58: return CallArranger.WINDOWS.arrangeUpcall(targetType, function, options); Suggestion: return CallArranger.WINDOWS.arrangeUpcall(targetType, function, options); src/java.base/share/classes/jdk/internal/foreign/abi/fallback/LibFallback.java line 159: > 157: */ > 158: static void getStructOffsets(MemorySegment structType, MemorySegment offsetsOut, FFIABI abi) > 159: throws IllegalStateException { Suggestion: throws IllegalStateException { test/jdk/java/foreign/trivial/TestTrivial.java line 74: > 72: VarHandle vhX = bigLayout.varHandle(MemoryLayout.PathElement.groupElement("x")); > 73: VarHandle vhY = bigLayout.varHandle(MemoryLayout.PathElement.groupElement("y")); > 74: try (Arena arena = Arena.ofConfined()) { nit: Suggestion: try (Arena arena = Arena.ofConfined()) { test/jdk/java/foreign/trivial/TestTrivial.java line 89: > 87: StructLayout capturedStateLayout = Linker.Option.captureStateLayout(); > 88: VarHandle errnoHandle = capturedStateLayout.varHandle(MemoryLayout.PathElement.groupElement("errno")); > 89: try (Arena arena = Arena.ofConfined()) { nit Suggestion: try (Arena arena = Arena.ofConfined()) { ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1144450371 PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1144453152 PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1144452221 PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1144451413 PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1144451634 From mcimadamore at openjdk.org Wed Mar 22 11:57:46 2023 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 22 Mar 2023 11:57:46 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 09:02:29 GMT, Per Minborg wrote: >> API changes for the FFM API (third preview) >> >> Specdiff: >> https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html >> >> Javadoc: >> https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html > > Per Minborg has updated the pull request incrementally with one additional commit since the last revision: > > Add example for Option::captureStateLayout src/java.base/share/classes/java/lang/foreign/MemoryLayout.java line 694: > 692: * @param bitSize the padding size in bits. > 693: * @return the new selector layout. > 694: * @throws IllegalArgumentException if {@code bitSize < 0} or {@code bitSize % 8 != 0} I'm not sure if this change in the `@throws` was deliberate - e.g. the new API seems to allow creation of padding layouts of zero size (which I did not realize). This can hide issues for code generators (e.g. I stumbled upon this when working on jextract, which was silently emitting zero-length paddings in some struct layouts). Perhaps better to revert to old semantics? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1144680066 From pminborg at openjdk.org Wed Mar 22 12:10:46 2023 From: pminborg at openjdk.org (Per Minborg) Date: Wed, 22 Mar 2023 12:10:46 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v5] In-Reply-To: References: Message-ID: On Wed, 22 Mar 2023 11:55:09 GMT, Maurizio Cimadamore wrote: >> Per Minborg has updated the pull request incrementally with one additional commit since the last revision: >> >> Add example for Option::captureStateLayout > > src/java.base/share/classes/java/lang/foreign/MemoryLayout.java line 694: > >> 692: * @param bitSize the padding size in bits. >> 693: * @return the new selector layout. >> 694: * @throws IllegalArgumentException if {@code bitSize < 0} or {@code bitSize % 8 != 0} > > I'm not sure if this change in the `@throws` was deliberate - e.g. the new API seems to allow creation of padding layouts of zero size (which I did not realize). This can hide issues for code generators (e.g. I stumbled upon this when working on jextract, which was silently emitting zero-length paddings in some struct layouts). Perhaps better to revert to old semantics? Agreed. Having zero-length padding does not make much sense. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1144707890 From pminborg at openjdk.org Wed Mar 22 12:28:14 2023 From: pminborg at openjdk.org (Per Minborg) Date: Wed, 22 Mar 2023 12:28:14 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v6] In-Reply-To: References: Message-ID: > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html Per Minborg has updated the pull request incrementally with two additional commits since the last revision: - Fix formating and modifier order - Fix typo -> shared arena ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13079/files - new: https://git.openjdk.org/jdk/pull/13079/files/21ef0607..192050d6 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=04-05 Stats: 6 lines in 5 files changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/13079.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13079/head:pull/13079 PR: https://git.openjdk.org/jdk/pull/13079 From pminborg at openjdk.org Wed Mar 22 13:58:35 2023 From: pminborg at openjdk.org (Per Minborg) Date: Wed, 22 Mar 2023 13:58:35 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v7] In-Reply-To: References: Message-ID: > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html Per Minborg has updated the pull request incrementally with one additional commit since the last revision: Disalow padding layouts of size zero ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13079/files - new: https://git.openjdk.org/jdk/pull/13079/files/192050d6..45febe9d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=05-06 Stats: 10 lines in 2 files changed: 3 ins; 5 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/13079.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13079/head:pull/13079 PR: https://git.openjdk.org/jdk/pull/13079 From pminborg at openjdk.org Wed Mar 22 14:09:07 2023 From: pminborg at openjdk.org (Per Minborg) Date: Wed, 22 Mar 2023 14:09:07 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v8] In-Reply-To: References: Message-ID: <4JIhKmX2VnDfArfFl-1YJfoUzGGBVA5Uvd3mdpatW-s=.5d86f29e-5475-4a4a-91df-d6418356e204@github.com> > API changes for the FFM API (third preview) > > Specdiff: > https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html > > Javadoc: > https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html Per Minborg has updated the pull request incrementally with one additional commit since the last revision: Improve javadocs for Linker::captureStateLayout ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13079/files - new: https://git.openjdk.org/jdk/pull/13079/files/45febe9d..6df28a78 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13079&range=06-07 Stats: 11 lines in 1 file changed: 0 ins; 1 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/13079.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13079/head:pull/13079 PR: https://git.openjdk.org/jdk/pull/13079 From pminborg at openjdk.org Wed Mar 22 14:09:56 2023 From: pminborg at openjdk.org (Per Minborg) Date: Wed, 22 Mar 2023 14:09:56 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v8] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 12:12:15 GMT, Maurizio Cimadamore wrote: >> src/java.base/share/classes/java/lang/foreign/Linker.java line 621: >> >>> 619: * to a downcall handle linked with {@link #captureCallState(String...)}} >>> 620: * >>> 621: * @see #captureCallState(String...) >> >> How does a caller know what the structure may contain? Should we document the platform specific structures? > > Back to @PaulSandoz question - "how does the caller know what the structure contains?". The caller typically doesn't care too much about what the returned struct is. But it might care about which "values" can be captured. That said, the set of such interesting values, is not too surprising. As demonstrated in the example in the `Linker.capturedCallState` method, once the client knows the name (and "errno" is likely to be 90% case), everything else follows from there - as the layout can be used to obtain var handles for the required values. > > But, perhaps, now that I write this, I realize that what @PaulSandoz might _really_ be asking is: how do I know that e.g. the returned struct will not contain e.g. nested structs, sequences, or other non-sense. So we might specify (in a normative way) that the returned layout is a struct layout, whose member layouts are one or more value layouts (possibly with some added padding layouts). The names of the value layouts reflect the names of the values that can be captured. > > And _then_ we show an example of the layout we return for Windows/x64 (as that's more interesting). I've updated the specs as per how I interpret the comments above. Let me know your thoughts on this. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13079#discussion_r1144872865 From alanb at openjdk.org Wed Mar 22 14:14:49 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 22 Mar 2023 14:14:49 GMT Subject: RFR: 8202110: (fs) Default file system provider should load UnixNativeDispatcher lazily In-Reply-To: <5oiN2L_4Epw_KePlEYYZm7qSG60RUSzG4HcJeBJE40U=.a40ea209-1529-41d6-94bb-f3cf2ba8c6c2@github.com> References: <5oiN2L_4Epw_KePlEYYZm7qSG60RUSzG4HcJeBJE40U=.a40ea209-1529-41d6-94bb-f3cf2ba8c6c2@github.com> Message-ID: On Tue, 21 Mar 2023 23:31:22 GMT, Brian Burkhalter wrote: > Initialize `needToResolveAgainstDefaultDirectory` instance variable lazily thereby deferring early use of `UnixNativeDispatcher` and hopefully loading `libnio`. src/java.base/unix/classes/sun/nio/fs/UnixFileSystem.java line 123: > 121: this.needToResolveAgainstDefaultDirectory = !defaultIsCwd; > 122: } > 123: } Making needToResolveAgainstDefaultDirectory a tri-state is a bit ugly. If we really need to defer this then it might be better to just move the field to Holder class, that way this method would return holder.needToResolveAgainstDefaultDirectory. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/13132#discussion_r1144879997 From mcimadamore at openjdk.org Wed Mar 22 14:38:30 2023 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 22 Mar 2023 14:38:30 GMT Subject: RFR: 8304265: Implementation of Foreign Function and Memory API (Third Preview) [v8] In-Reply-To: <4JIhKmX2VnDfArfFl-1YJfoUzGGBVA5Uvd3mdpatW-s=.5d86f29e-5475-4a4a-91df-d6418356e204@github.com> References: <4JIhKmX2VnDfArfFl-1YJfoUzGGBVA5Uvd3mdpatW-s=.5d86f29e-5475-4a4a-91df-d6418356e204@github.com> Message-ID: On Wed, 22 Mar 2023 14:09:07 GMT, Per Minborg wrote: >> API changes for the FFM API (third preview) >> >> Specdiff: >> https://cr.openjdk.org/~pminborg/panama/21/v1/specdiff/overview-summary.html >> >> Javadoc: >> https://cr.openjdk.org/~pminborg/panama/21/v1/javadoc/java.base/module-summary.html > > Per Minborg has updated the pull request incrementally with one additional commit since the last revision: > > Improve javadocs for Linker::captureStateLayout src/java.base/share/classes/java/lang/foreign/Linker.java line 628: > 626: * and possibly {@linkplain PaddingLayout padding layouts}. > 627: * As an example, on Windows, the returned layout might contain three value layouts named: > 628: *