From alan.bateman at oracle.com Sun Jan 5 09:53:08 2025 From: alan.bateman at oracle.com (Alan Bateman) Date: Sun, 5 Jan 2025 09:53:08 +0000 Subject: NOT_INTERRUPTIBLE OpenOption and how that should fit with InterruptibleChanne In-Reply-To: References: Message-ID: On 26/12/2024 10:15, Bruno Ploumhans wrote: > Hi, > > I opened a PR that adds StandardOpenOptions.NOT_INTERRUPTIBLE to > expose and supersede FileChannelImpl#setUninterruptible. However, this > needs more discussion because FileChannel implements > InterruptibleChannel. > > Here are some thoughts to kick-start the discussion: > - The documentation of InterruptibleChannel does not currently account > for implementations that might not be interruptible. Same for the > documentation of Thread.interrupt(). That needs changing. > - Alan suggested in previous discussion (2018) a new `boolean > isInterruptible()` method. > - I wonder if the ability to pass a `boolean uninterruptible` flag > should be exposed in AbstractUninterruptibleChannel, or if that should > be kept an implementation detail of FileChannelImpl. > > In any case, I am happy to experiment with various solutions. > This is a topic that I've looked several times including explorations and prototypes of four options: 1. Changing FileChannel to not extend AbstractInterruptibleChannel 2. An OpenOption to opt out of interrupt closing the channel 3. An?OpenOption to opt in to interrupt closing the channel 4. Specifying that interrupt does not close the channel but specified to allow for compatibility with long standing behavior There are pros and cons to all. For the options explored then most of the change is specification with the starting point for all was to re-specify async close of FileChannel to align with long standing behavior. If you look at how async close of a FileChannel works on Windows or with virtual threads then you'll see what I mean. There is also the issue that the lock methods specify that FileLockInterruptionException is thrown without closing the channel but the implementation differs. My view on this has changed from when I first looked at it a few years ago. Right now, I think #4 may be the best option. The spec changes that I have from that option are: - Changes to FileChannel to specify that it deviates from InterruptibleChannel to allow for close to wait for all outstanding I/O operations to end. This allows for long standing behavior. As part of this, FileChannel overrides the specification of implCloseChannel. - Changes to FileChannel to specify that it deviates from InterruptibleChannel so that interrupt does not close the channel but allows for an implementation specific means for interrupt to close the channel. - Changes to the specification of Thread.interrupt, ClosedByInterruptException, all methods in FileChannel that throw ClosedByInterruptException, FileLockInterruptionException and the FileChannel lock methods. The main thing with this option is identifying any common usages where ClosedByInterruptException may be depended on, e.g. trying to shutdown something when threads are blocked indefinitely in file I/O operations. Such usages are likely file access over a network or maybe transfer to/from a channel backed by a network connection. Any help to identify such usages would be helpful to assess the compatibility impact of the options that change default behavior. Your exploration is #2. The spec changes that I did when prototyping this option including changes to all the above spec areas, and also everywhere that opens a file, e.g. AsynchronousFileChannel. It is clearly the least risky from a compatibility perspective as it doesn't change default behavior but also the least satisfactory option too. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From bpb at openjdk.org Mon Jan 6 17:49:40 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 6 Jan 2025 17:49:40 GMT Subject: Integrated: 8345432: (ch, fs) Replace anonymous Thread with InnocuousThread In-Reply-To: References: Message-ID: On Thu, 12 Dec 2024 02:33:30 GMT, Brian Burkhalter wrote: > Change internal instances of anonymous `Thread`s to instances of `InnocuousThread`. This pull request has now been integrated. Changeset: d723597d Author: Brian Burkhalter URL: https://git.openjdk.org/jdk/commit/d723597dd953c3fb7c3ae04a1c6cce7e10869406 Stats: 19 lines in 5 files changed: 5 ins; 6 del; 8 mod 8345432: (ch, fs) Replace anonymous Thread with InnocuousThread Reviewed-by: alanb ------------- PR: https://git.openjdk.org/jdk/pull/22697 From alanb at openjdk.org Tue Jan 7 10:37:52 2025 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 7 Jan 2025 10:37:52 GMT Subject: Integrated: 8346573: Can't use custom default file system provider with custom system class loader In-Reply-To: References: Message-ID: On Fri, 20 Dec 2024 09:47:32 GMT, Alan Bateman wrote: > If an application is deployed to use a custom default file system provider, and a custom system class loader, and initialisation of the custom system class loader uses the file system APIs, then it leads to a recursive initialisation issue. This issue has always existed but has only recently been reported. Using both configuration knobs at the same time is probably very rare but it does require attention in advance of any proposals that re-implement the java.io file classes on java.nio.file. > > The proposed change here is to re-specify FileSystems.getDefault to use the default system class loader when locating the custom default file system provider. In spec terms, this is changing "system class loader" to "default system class loader" with a link to its meaning in ClassLoader.getSystemClassLoader. As part of the change I've expanded the wording to specify that the custom default file system provider must be public in a package exported to at least java.base, and that the one-arg constructor also be public. This additional clarification specifies long standing behavior and does not change any implementation. > > The SetDefaultProvider.java test is expanded to add a new test the runs with both a custom default file system provider and custom system class loader, and where where the custom system class loader uses the file APIs in its initialisation. This pull request has now been integrated. Changeset: f1196638 Author: Alan Bateman URL: https://git.openjdk.org/jdk/commit/f1196638b347c9a2e7d7dc1795c6c24ea7efe711 Stats: 72 lines in 3 files changed: 58 ins; 2 del; 12 mod 8346573: Can't use custom default file system provider with custom system class loader Reviewed-by: mkartashev, bpb ------------- PR: https://git.openjdk.org/jdk/pull/22842 From jkern at openjdk.org Tue Jan 7 13:56:52 2025 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 7 Jan 2025 13:56:52 GMT Subject: Integrated: 8211851: (ch) java/nio/channels/AsynchronousSocketChannel/StressLoopback.java times out (aix) In-Reply-To: References: Message-ID: On Mon, 9 Dec 2024 09:53:26 GMT, Joachim Kern wrote: > Test /jdk/java/nio/channels/AsynchronousSocketChannel/StressLoopback.java times out on aix. > > The problem is that in queueControlEvent() for every event added to the controlQueue one byte is written to the ctlSP pipe to wakeup the poll thread running in poll(). The poll() function of the poll thread is just taking this one byte from the pipe but handling then all events in the controlQueue. Afterwards the controlQueue is empty, but the pipe is only 1 byte shorter. This is an asymmetric and therefore false behavior. So I changed the drain1() to a drain() meaning drain_all() which heals this asymmetry. > The problem leading to the timeout was, that more and more (StressLoopback: nomen est omen) wakeup bytes were written to the pipe, but the poll thread runs in much lower frequency. So, at some time the pipe was full on OS side, resulting in a blocking Pollset.interrupt(ctlSp[1]) in queueControlEvent(). Unfortunately, at this moment the controlQueue sync object is in locked state, resulting that the poll thread cannot acquire this lock to read from the pipe. No reading from the pipe means the pipe remains full and the dead lock is there. > Additionally, I found a flaw in the emulator of the ?one shot? semantic. Here the original implantation was not save against deleting a just registered fd before the next poll. To fix this I separated the emulation of the ?one shot? from the rest of the poll() actions. Now, the ?one shot? emulation is encapsulated in the controlLock.lock()block, which guards the poll itself. This pull request has now been integrated. Changeset: 8b22517c Author: Joachim Kern URL: https://git.openjdk.org/jdk/commit/8b22517cb0b24c4134a2dbf22591f6f84d7d866c Stats: 55 lines in 2 files changed: 21 ins; 16 del; 18 mod 8211851: (ch) java/nio/channels/AsynchronousSocketChannel/StressLoopback.java times out (aix) Reviewed-by: mdoerr, varadam ------------- PR: https://git.openjdk.org/jdk/pull/22633 From bpb at openjdk.org Tue Jan 7 20:55:18 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 7 Jan 2025 20:55:18 GMT Subject: RFR: 8346671: java/nio/file/Files/probeContentType/Basic.java fails on Windows 2025 Message-ID: <8FVl7NqVVWdYZZplEEFW6wtQh9RsKEAl_rPYgtifXqM=.5ce14fb5-fd75-489a-8009-f9536f04afc8@github.com> Modify the test to expect the same content types on Windows Server 2025 as it does on Windows 11. ------------- Commit messages: - 8346671: java/nio/file/Files/probeContentType/Basic.java fails on Windows 2025 Changes: https://git.openjdk.org/jdk/pull/22957/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=22957&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8346671 Stats: 4 lines in 1 file changed: 2 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/22957.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22957/head:pull/22957 PR: https://git.openjdk.org/jdk/pull/22957 From bpb at openjdk.org Tue Jan 7 20:55:18 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 7 Jan 2025 20:55:18 GMT Subject: RFR: 8346671: java/nio/file/Files/probeContentType/Basic.java fails on Windows 2025 In-Reply-To: <8FVl7NqVVWdYZZplEEFW6wtQh9RsKEAl_rPYgtifXqM=.5ce14fb5-fd75-489a-8009-f9536f04afc8@github.com> References: <8FVl7NqVVWdYZZplEEFW6wtQh9RsKEAl_rPYgtifXqM=.5ce14fb5-fd75-489a-8009-f9536f04afc8@github.com> Message-ID: <5czuiIjHKo5R6l09g_gCnAP3AaXxPisVVdfO82Vo8rU=.fd233416-51aa-4556-b0e6-d126d86dc401@github.com> On Tue, 7 Jan 2025 20:50:28 GMT, Brian Burkhalter wrote: > Modify the test to expect the same content types on Windows Server 2025 as it does on Windows 11. This change has been verified in the CI on all platforms except Windows Server 2025 and for the latter on a separate machine. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22957#issuecomment-2576197238 From bpb at openjdk.org Tue Jan 7 23:23:53 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 7 Jan 2025 23:23:53 GMT Subject: RFR: 8346671: java/nio/file/Files/probeContentType/Basic.java fails on Windows 2025 [v2] In-Reply-To: <8FVl7NqVVWdYZZplEEFW6wtQh9RsKEAl_rPYgtifXqM=.5ce14fb5-fd75-489a-8009-f9536f04afc8@github.com> References: <8FVl7NqVVWdYZZplEEFW6wtQh9RsKEAl_rPYgtifXqM=.5ce14fb5-fd75-489a-8009-f9536f04afc8@github.com> Message-ID: > Modify the test to expect the same content types on Windows Server 2025 as it does on Windows 11. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8346671: Improve OS name test; use StaticProperty ------------- Changes: - all: https://git.openjdk.org/jdk/pull/22957/files - new: https://git.openjdk.org/jdk/pull/22957/files/a9c407cd..a1322fa1 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=22957&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22957&range=00-01 Stats: 4 lines in 1 file changed: 1 ins; 1 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/22957.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22957/head:pull/22957 PR: https://git.openjdk.org/jdk/pull/22957 From jpai at openjdk.org Wed Jan 8 01:28:48 2025 From: jpai at openjdk.org (Jaikiran Pai) Date: Wed, 8 Jan 2025 01:28:48 GMT Subject: RFR: 8346671: java/nio/file/Files/probeContentType/Basic.java fails on Windows 2025 [v2] In-Reply-To: References: <8FVl7NqVVWdYZZplEEFW6wtQh9RsKEAl_rPYgtifXqM=.5ce14fb5-fd75-489a-8009-f9536f04afc8@github.com> Message-ID: <1s7XmiFB6LWRRueR6hN3rJOD83iA0Aq-6vnR-bH4FeY=.4e0e5c4b-84cb-4cc5-b831-753fc0a97d77@github.com> On Tue, 7 Jan 2025 23:23:53 GMT, Brian Burkhalter wrote: >> Modify the test to expect the same content types on Windows Server 2025 as it does on Windows 11. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8346671: Improve OS name test; use StaticProperty Marked as reviewed by jpai (Reviewer). test/jdk/java/nio/file/Files/probeContentType/Basic.java line 195: > 193: // Windows Server 2025 > 194: if (OperatingSystem.isWindows() && > 195: (StaticProperty.osName().matches("^.*[11|2025]$") || Hello Brian, the change looks reasonable to me. If we want to be extra "strict" on which Windows version this conditional is applicable for, then maybe we should include the regex pattern for one or more whitespace before the the `[11|2025]`. But given what this test is for, I think it's fine in this proposed form too. ------------- PR Review: https://git.openjdk.org/jdk/pull/22957#pullrequestreview-2535653987 PR Review Comment: https://git.openjdk.org/jdk/pull/22957#discussion_r1906218076 From bpb at openjdk.org Wed Jan 8 01:45:34 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 8 Jan 2025 01:45:34 GMT Subject: RFR: 8346671: java/nio/file/Files/probeContentType/Basic.java fails on Windows 2025 [v2] In-Reply-To: <1s7XmiFB6LWRRueR6hN3rJOD83iA0Aq-6vnR-bH4FeY=.4e0e5c4b-84cb-4cc5-b831-753fc0a97d77@github.com> References: <8FVl7NqVVWdYZZplEEFW6wtQh9RsKEAl_rPYgtifXqM=.5ce14fb5-fd75-489a-8009-f9536f04afc8@github.com> <1s7XmiFB6LWRRueR6hN3rJOD83iA0Aq-6vnR-bH4FeY=.4e0e5c4b-84cb-4cc5-b831-753fc0a97d77@github.com> Message-ID: <2UXd9bP02uzugTdef7ZgaO8qif0JzHEhtac7IEHVJJM=.a3a1ae2f-132d-4a2a-9c6c-1562694b373d@github.com> On Wed, 8 Jan 2025 01:25:43 GMT, Jaikiran Pai wrote: > maybe we should include the regex pattern for one or more whitespace before the the `[11|2025]`. But given what this test is for, I think it's fine in this proposed form too. Hi @jaikiran. Thanks for the prompt review. I'll give it some more thought tomorrow, but might just leave it as is. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22957#discussion_r1906226997 From alanb at openjdk.org Wed Jan 8 11:08:35 2025 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 8 Jan 2025 11:08:35 GMT Subject: RFR: 8346671: java/nio/file/Files/probeContentType/Basic.java fails on Windows 2025 [v2] In-Reply-To: References: <8FVl7NqVVWdYZZplEEFW6wtQh9RsKEAl_rPYgtifXqM=.5ce14fb5-fd75-489a-8009-f9536f04afc8@github.com> Message-ID: On Tue, 7 Jan 2025 23:23:53 GMT, Brian Burkhalter wrote: >> Modify the test to expect the same content types on Windows Server 2025 as it does on Windows 11. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8346671: Improve OS name test; use StaticProperty Maybe at some point we should just remove the bulk of these extensions from this test. It only needs one or two stable extensions, the rest just add test maintenance. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22957#issuecomment-2577402025 From bpb at openjdk.org Wed Jan 8 17:34:30 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 8 Jan 2025 17:34:30 GMT Subject: RFR: 8346671: java/nio/file/Files/probeContentType/Basic.java fails on Windows 2025 [v2] In-Reply-To: References: <8FVl7NqVVWdYZZplEEFW6wtQh9RsKEAl_rPYgtifXqM=.5ce14fb5-fd75-489a-8009-f9536f04afc8@github.com> Message-ID: <6iFJwi5efAGvCJht7lUQzjiS2KSG4vdGSdeRgg1J_cg=.f29cc721-10aa-4556-9d18-3e8e96e6a5ad@github.com> On Wed, 8 Jan 2025 11:06:21 GMT, Alan Bateman wrote: > Maybe at some point we should just remove the bulk of these extensions from this test. It only needs one or two stable extensions, the rest just add test maintenance. I filed [JDK-8347286](https://bugs.openjdk.org/browse/JDK-8347286) to track this suggestion. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22957#issuecomment-2578234238 From bpb at openjdk.org Wed Jan 8 17:39:49 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 8 Jan 2025 17:39:49 GMT Subject: RFR: 8346671: java/nio/file/Files/probeContentType/Basic.java fails on Windows 2025 [v2] In-Reply-To: <2UXd9bP02uzugTdef7ZgaO8qif0JzHEhtac7IEHVJJM=.a3a1ae2f-132d-4a2a-9c6c-1562694b373d@github.com> References: <8FVl7NqVVWdYZZplEEFW6wtQh9RsKEAl_rPYgtifXqM=.5ce14fb5-fd75-489a-8009-f9536f04afc8@github.com> <1s7XmiFB6LWRRueR6hN3rJOD83iA0Aq-6vnR-bH4FeY=.4e0e5c4b-84cb-4cc5-b831-753fc0a97d77@github.com> <2UXd9bP02uzugTdef7ZgaO8qif0JzHEhtac7IEHVJJM=.a3a1ae2f-132d-4a2a-9c6c-1562694b373d@github.com> Message-ID: On Wed, 8 Jan 2025 01:43:14 GMT, Brian Burkhalter wrote: > I'll give it some more thought tomorrow, but might just leave it as is. Given the newly filed issue [JDK-8347286](https://bugs.openjdk.org/browse/JDK-8347286), I think I'll leave this suggestion until later, as it might simply vanish. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22957#discussion_r1907556854 From hannesw at openjdk.org Wed Jan 8 19:47:13 2025 From: hannesw at openjdk.org (Hannes =?UTF-8?B?V2FsbG7DtmZlcg==?=) Date: Wed, 8 Jan 2025 19:47:13 GMT Subject: RFR: 8347121: Add missing @serial tags to module java.base Message-ID: Please review a doc-only change to add missing `@serial` javadoc tags in module `java.base`. This is a sub-task of [JDK-8286931] to allow us to re-enable the javadoc `-serialwarn` option in the JDK doc build, which has been disabled since JDK 19. [JDK-8286931]: https://bugs.openjdk.org/browse/JDK-8286931 For private and package-private serialized fields that already have a doc comment, the main description is converted to a block tag by prepending `@serial` since these fields do not require a main description. (For protected and public serialized fields we would have appended an empty `@serial` block tag instead with the same effect, but there are no protected/public fields in this particular sub-task.) For those fields that do not have a doc comment, a doc comment with an empty `@serial` tag is added. There is one special case in `java/util/UUID.java` where two private fields had plain traditional comments containing `@serial` tags instead of doc comments. Converting these comments to doc-comments is the only change in this PR that causes a change in the generated documentation, adding documentation for these two fields to the `serialized-form.html` page. ------------- Commit messages: - 8347121: Add missing @serial tags to module java.base Changes: https://git.openjdk.org/jdk/pull/22978/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=22978&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8347121 Stats: 305 lines in 89 files changed: 83 ins; 4 del; 218 mod Patch: https://git.openjdk.org/jdk/pull/22978.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22978/head:pull/22978 PR: https://git.openjdk.org/jdk/pull/22978 From dfuchs at openjdk.org Wed Jan 8 19:57:45 2025 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Wed, 8 Jan 2025 19:57:45 GMT Subject: RFR: 8347121: Add missing @serial tags to module java.base In-Reply-To: References: Message-ID: <4QE9pLQdvn8djpGnGMzgcoI9PSLLsD6RpGQAiVB6D2Q=.477d3db7-3d47-4bbe-8a3b-88f09e73ded6@github.com> On Wed, 8 Jan 2025 19:41:42 GMT, Hannes Walln?fer wrote: > Please review a doc-only change to add missing `@serial` javadoc tags in module `java.base`. This is a sub-task of [JDK-8286931] to allow us to re-enable the javadoc `-serialwarn` option in the JDK doc build, which has been disabled since JDK 19. > > [JDK-8286931]: https://bugs.openjdk.org/browse/JDK-8286931 > > For private and package-private serialized fields that already have a doc comment, the main description is converted to a block tag by prepending `@serial` since these fields do not require a main description. (For protected and public serialized fields we would have appended an empty `@serial` block tag instead with the same effect, but there are no protected/public fields in this particular sub-task.) For those fields that do not have a doc comment, a doc comment with an empty `@serial` tag is added. > > There is one special case in `java/util/UUID.java` where two private fields had plain traditional comments containing `@serial` tags instead of doc comments. Converting these comments to doc-comments is the only change in this PR that causes a change in the generated documentation, adding documentation for these two fields to the `serialized-form.html` page. Changes to `java.net` look OK. ------------- PR Review: https://git.openjdk.org/jdk/pull/22978#pullrequestreview-2538086167 From bpb at openjdk.org Wed Jan 8 20:49:49 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 8 Jan 2025 20:49:49 GMT Subject: Integrated: 8346671: java/nio/file/Files/probeContentType/Basic.java fails on Windows 2025 In-Reply-To: <8FVl7NqVVWdYZZplEEFW6wtQh9RsKEAl_rPYgtifXqM=.5ce14fb5-fd75-489a-8009-f9536f04afc8@github.com> References: <8FVl7NqVVWdYZZplEEFW6wtQh9RsKEAl_rPYgtifXqM=.5ce14fb5-fd75-489a-8009-f9536f04afc8@github.com> Message-ID: On Tue, 7 Jan 2025 20:50:28 GMT, Brian Burkhalter wrote: > Modify the test to expect the same content types on Windows Server 2025 as it does on Windows 11. This pull request has now been integrated. Changeset: 2e00816a Author: Brian Burkhalter URL: https://git.openjdk.org/jdk/commit/2e00816ac30373df28a29dc938f7ce5489a3edfc Stats: 6 lines in 1 file changed: 2 ins; 0 del; 4 mod 8346671: java/nio/file/Files/probeContentType/Basic.java fails on Windows 2025 Reviewed-by: jpai ------------- PR: https://git.openjdk.org/jdk/pull/22957 From bpb at openjdk.org Wed Jan 8 20:56:16 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 8 Jan 2025 20:56:16 GMT Subject: RFR: 8347171: (dc) java/nio/channels/DatagramChannel/InterruptibleOrNot.java fails with virtual thread factory Message-ID: Do not run `testInterruptBeforeUninterruptibleReceive` when the thread is virtual. ------------- Commit messages: - 8347171: (dc) java/nio/channels/DatagramChannel/InterruptibleOrNot.java fails with virtual thread factory Changes: https://git.openjdk.org/jdk/pull/22981/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=22981&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8347171 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/22981.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22981/head:pull/22981 PR: https://git.openjdk.org/jdk/pull/22981 From alanb at openjdk.org Wed Jan 8 21:08:49 2025 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 8 Jan 2025 21:08:49 GMT Subject: RFR: 8347171: (dc) java/nio/channels/DatagramChannel/InterruptibleOrNot.java fails with virtual thread factory In-Reply-To: References: Message-ID: On Wed, 8 Jan 2025 20:49:45 GMT, Brian Burkhalter wrote: > Do not run `testInterruptBeforeUninterruptibleReceive` when the thread is virtual. Marked as reviewed by alanb (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/22981#pullrequestreview-2538294193 From lmesnik at openjdk.org Wed Jan 8 22:21:37 2025 From: lmesnik at openjdk.org (Leonid Mesnik) Date: Wed, 8 Jan 2025 22:21:37 GMT Subject: RFR: 8347171: (dc) java/nio/channels/DatagramChannel/InterruptibleOrNot.java fails with virtual thread factory In-Reply-To: References: Message-ID: <1Qc4JCLA90MMbImCBnY5-u65lMW62MlsHl7KpCYAvKs=.a5a19a64-1ab0-4340-b9e2-d978cebe21d9@github.com> On Wed, 8 Jan 2025 20:49:45 GMT, Brian Burkhalter wrote: > Do not run `testInterruptBeforeUninterruptibleReceive` when the thread is virtual. Marked as reviewed by lmesnik (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/22981#pullrequestreview-2538394741 From alanb at openjdk.org Thu Jan 9 09:11:36 2025 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 9 Jan 2025 09:11:36 GMT Subject: RFR: 8347121: Add missing @serial tags to module java.base In-Reply-To: References: Message-ID: On Wed, 8 Jan 2025 19:41:42 GMT, Hannes Walln?fer wrote: > Please review a doc-only change to add missing `@serial` javadoc tags in module `java.base`. This is a sub-task of [JDK-8286931] to allow us to re-enable the javadoc `-serialwarn` option in the JDK doc build, which has been disabled since JDK 19. > > [JDK-8286931]: https://bugs.openjdk.org/browse/JDK-8286931 > > For private and package-private serialized fields that already have a doc comment, the main description is converted to a block tag by prepending `@serial` since these fields do not require a main description. (For protected and public serialized fields we would have appended an empty `@serial` block tag instead with the same effect, but there are no protected/public fields in this particular sub-task.) For those fields that do not have a doc comment, a doc comment with an empty `@serial` tag is added. > > There is one special case in `java/util/UUID.java` where two private fields had plain traditional comments containing `@serial` tags instead of doc comments. Converting these comments to doc-comments is the only change in this PR that causes a change in the generated documentation, adding documentation for these two fields to the `serialized-form.html` page. I skimmed through this, including the UUID comments, and looks fine. ------------- Marked as reviewed by alanb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/22978#pullrequestreview-2539183890 From hannesw at openjdk.org Thu Jan 9 11:51:40 2025 From: hannesw at openjdk.org (Hannes =?UTF-8?B?V2FsbG7DtmZlcg==?=) Date: Thu, 9 Jan 2025 11:51:40 GMT Subject: Integrated: 8347121: Add missing @serial tags to module java.base In-Reply-To: References: Message-ID: On Wed, 8 Jan 2025 19:41:42 GMT, Hannes Walln?fer wrote: > Please review a doc-only change to add missing `@serial` javadoc tags in module `java.base`. This is a sub-task of [JDK-8286931] to allow us to re-enable the javadoc `-serialwarn` option in the JDK doc build, which has been disabled since JDK 19. > > [JDK-8286931]: https://bugs.openjdk.org/browse/JDK-8286931 > > For private and package-private serialized fields that already have a doc comment, the main description is converted to a block tag by prepending `@serial` since these fields do not require a main description. (For protected and public serialized fields we would have appended an empty `@serial` block tag instead with the same effect, but there are no protected/public fields in this particular sub-task.) For those fields that do not have a doc comment, a doc comment with an empty `@serial` tag is added. > > There is one special case in `java/util/UUID.java` where two private fields had plain traditional comments containing `@serial` tags instead of doc comments. Converting these comments to doc-comments is the only change in this PR that causes a change in the generated documentation, adding documentation for these two fields to the `serialized-form.html` page. This pull request has now been integrated. Changeset: ceae2b97 Author: Hannes Walln?fer URL: https://git.openjdk.org/jdk/commit/ceae2b977dac58a9b2c09e42cb256c94eff9222b Stats: 305 lines in 89 files changed: 83 ins; 4 del; 218 mod 8347121: Add missing @serial tags to module java.base Reviewed-by: alanb ------------- PR: https://git.openjdk.org/jdk/pull/22978 From duke at openjdk.org Thu Jan 9 15:28:48 2025 From: duke at openjdk.org (Volkan =?UTF-8?B?WWF6xLFjxLE=?=) Date: Thu, 9 Jan 2025 15:28:48 GMT Subject: RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk Message-ID: Switches from using the temporary directory to work directory in `LoopingTruncate`. This should effectively provide sufficient space for the test finish. Solution is tipped by @bplb. `tier1,2` tests' results will be attached to the ticket once they finish. ------------- Commit messages: - Switch from using the temporary directory to work directory in `LoopingTruncate` Changes: https://git.openjdk.org/jdk/pull/23011/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23011&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8346972 Stats: 10 lines in 1 file changed: 5 ins; 2 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/23011.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/23011/head:pull/23011 PR: https://git.openjdk.org/jdk/pull/23011 From bpb at openjdk.org Thu Jan 9 15:29:40 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 9 Jan 2025 15:29:40 GMT Subject: Integrated: 8347171: (dc) java/nio/channels/DatagramChannel/InterruptibleOrNot.java fails with virtual thread factory In-Reply-To: References: Message-ID: On Wed, 8 Jan 2025 20:49:45 GMT, Brian Burkhalter wrote: > Do not run `testInterruptBeforeUninterruptibleReceive` when the thread is virtual. This pull request has now been integrated. Changeset: 1ef77cdd Author: Brian Burkhalter URL: https://git.openjdk.org/jdk/commit/1ef77cdd51b91f6d6d3367444a37a3f0f2e4bc99 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod 8347171: (dc) java/nio/channels/DatagramChannel/InterruptibleOrNot.java fails with virtual thread factory Reviewed-by: alanb, lmesnik ------------- PR: https://git.openjdk.org/jdk/pull/22981 From bpb at openjdk.org Thu Jan 9 15:42:35 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 9 Jan 2025 15:42:35 GMT Subject: RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk In-Reply-To: References: Message-ID: On Thu, 9 Jan 2025 15:23:55 GMT, Volkan Yaz?c? wrote: > Switches from using the temporary directory to work directory in `LoopingTruncate`. This should effectively provide sufficient space for the test finish. Solution is tipped by @bplb. `tier1,2` tests' results will be attached to the ticket once they finish. Looks fine to me. ------------- Marked as reviewed by bpb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/23011#pullrequestreview-2540270514 From alanb at openjdk.org Thu Jan 9 15:51:44 2025 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 9 Jan 2025 15:51:44 GMT Subject: RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk In-Reply-To: References: Message-ID: On Thu, 9 Jan 2025 15:23:55 GMT, Volkan Yaz?c? wrote: > Switches from using the temporary directory to work directory in `LoopingTruncate`. This should effectively provide sufficient space for the test finish. Solution is tipped by @bplb. `tier1,2` tests' results will be attached to the ticket once they finish. test/jdk/java/nio/channels/FileChannel/LoopingTruncate.java line 53: > 51: > 52: public static void main(String[] args) throws Throwable { > 53: Path dir = Path.of(StaticProperty.userDir()); If you change this to Path.of(".") then it would avoid using a JDK internal class, Path.of("user.dir") will work too. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/23011#discussion_r1909057657 From bpb at openjdk.org Thu Jan 9 15:59:37 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 9 Jan 2025 15:59:37 GMT Subject: RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk In-Reply-To: References: Message-ID: On Thu, 9 Jan 2025 15:48:49 GMT, Alan Bateman wrote: >> Switches from using the temporary directory to work directory in `LoopingTruncate`. This should effectively provide sufficient space for the test finish. Solution is tipped by @bplb. `tier1,2` tests' results will be attached to the ticket once they finish. > > test/jdk/java/nio/channels/FileChannel/LoopingTruncate.java line 53: > >> 51: >> 52: public static void main(String[] args) throws Throwable { >> 53: Path dir = Path.of(StaticProperty.userDir()); > > If you change this to Path.of(".") then it would avoid using a JDK internal class, Path.of(System.getProperty("user.dir")) will work too. If that approach is taken, then line 27 `@modules ...` can be removed. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/23011#discussion_r1909072628 From liach at openjdk.org Fri Jan 10 08:25:36 2025 From: liach at openjdk.org (Chen Liang) Date: Fri, 10 Jan 2025 08:25:36 GMT Subject: RFR: 8316882: Add NOT_INTERRUPTIBLE OpenOption and use it in ZipFS In-Reply-To: References: Message-ID: <16OW58lt-tWhc8B1G-yLedW2jG6XJwUCQumjKrwD7nU=.5daa8ce2-c4fa-4869-b302-bfa1008a6461@github.com> On Wed, 25 Dec 2024 23:55:18 GMT, Bruno Ploumhans wrote: > Here is a fix for https://bugs.openjdk.org/browse/JDK-8316882. > > Following discussion in nio-dev a while ago, I have opted to add a new `NOT_INTERRUPTIBLE` open option, superseding `FileChannelImpl#setUninterruptible`. src/java.base/share/classes/jdk/internal/jimage/BasicImageReader.java line 112: > 110: opts.add(StandardOpenOption.READ); > 111: // No lambdas during bootstrap > 112: AccessController.doPrivileged(new PrivilegedAction() { Can probably remove this AccessController wrapping on JDK 25+, and just additionally catch SecurityException in case this fails in older releases where security manager exists. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22882#discussion_r1909985130 From duke at openjdk.org Fri Jan 10 09:04:20 2025 From: duke at openjdk.org (Volkan =?UTF-8?B?WWF6xLFjxLE=?=) Date: Fri, 10 Jan 2025 09:04:20 GMT Subject: RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk [v2] In-Reply-To: References: Message-ID: <2aFX33Ck279aZFBSv15u5_HWOKjxHGtfRh2apiUNdKk=.f8dfac45-ee40-46e4-ad40-5f5500ddf9d6@github.com> On Thu, 9 Jan 2025 15:56:52 GMT, Brian Burkhalter wrote: >> test/jdk/java/nio/channels/FileChannel/LoopingTruncate.java line 53: >> >>> 51: >>> 52: public static void main(String[] args) throws Throwable { >>> 53: Path dir = Path.of(StaticProperty.userDir()); >> >> If you change this to Path.of(".") then it would avoid using a JDK internal class, Path.of(System.getProperty("user.dir")) will work too. > > If that approach is taken, then line 27 `@modules ...` can be removed. Switch to `Path.of(System.getProperty("user.dir"))` and documented the rationale in 8fc22379f774331be9a0dfea9a067483707c48df. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/23011#discussion_r1910022593 From duke at openjdk.org Fri Jan 10 09:04:20 2025 From: duke at openjdk.org (Volkan =?UTF-8?B?WWF6xLFjxLE=?=) Date: Fri, 10 Jan 2025 09:04:20 GMT Subject: RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk [v2] In-Reply-To: References: Message-ID: <0cnlSLtN3jdszmv_gOGCPm9alx0TzVrBCaVmD4aUnEE=.e6363ef9-d77f-4c9b-b5bb-a83781cd8201@github.com> > Switches from using the temporary directory to work directory in `LoopingTruncate`. This should effectively provide sufficient space for the test finish. Solution is tipped by @bplb. `tier1,2` tests' results will be attached to the ticket once they finish. Volkan Yaz?c? has updated the pull request incrementally with one additional commit since the last revision: Simplify and document changes in `LoopingTruncate` ------------- Changes: - all: https://git.openjdk.org/jdk/pull/23011/files - new: https://git.openjdk.org/jdk/pull/23011/files/2132a925..8fc22379 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=23011&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23011&range=00-01 Stats: 7 lines in 1 file changed: 3 ins; 2 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/23011.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/23011/head:pull/23011 PR: https://git.openjdk.org/jdk/pull/23011 From alanb at openjdk.org Fri Jan 10 12:41:46 2025 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 10 Jan 2025 12:41:46 GMT Subject: RFR: 8346722: (fs) Files.probeContentType throws ClassCastException with custom file system provider In-Reply-To: References: Message-ID: On Thu, 9 Jan 2025 20:56:20 GMT, Brian Burkhalter wrote: > Ensure the file type detection works with a custom default file system provider. src/java.base/unix/classes/sun/nio/fs/DefaultFileTypeDetector.java line 36: > 34: > 35: public static FileTypeDetector create() { > 36: FileSystemProvider provider = FileSystems.getDefault().provider(); I assume the imports can be pruned now. test/jdk/java/nio/file/spi/testapp/testapp/Main.java line 65: > 63: String fileType = Files.probeContentType(Path.of(".")); > 64: if (fileType != null) > 65: throw new RuntimeException("File type non-null: " + fileType); Calling Files.probeContentType is enough to exercise the code, I would be worried that it returns something based on the name of the directory, is that possible? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/23019#discussion_r1910321513 PR Review Comment: https://git.openjdk.org/jdk/pull/23019#discussion_r1910323722 From mullan at openjdk.org Fri Jan 10 13:17:36 2025 From: mullan at openjdk.org (Sean Mullan) Date: Fri, 10 Jan 2025 13:17:36 GMT Subject: RFR: 8316882: Add NOT_INTERRUPTIBLE OpenOption and use it in ZipFS In-Reply-To: <16OW58lt-tWhc8B1G-yLedW2jG6XJwUCQumjKrwD7nU=.5daa8ce2-c4fa-4869-b302-bfa1008a6461@github.com> References: <16OW58lt-tWhc8B1G-yLedW2jG6XJwUCQumjKrwD7nU=.5daa8ce2-c4fa-4869-b302-bfa1008a6461@github.com> Message-ID: On Fri, 10 Jan 2025 08:22:37 GMT, Chen Liang wrote: >> Here is a fix for https://bugs.openjdk.org/browse/JDK-8316882. >> >> Following discussion in nio-dev a while ago, I have opted to add a new `NOT_INTERRUPTIBLE` open option, superseding `FileChannelImpl#setUninterruptible`. > > src/java.base/share/classes/jdk/internal/jimage/BasicImageReader.java line 112: > >> 110: opts.add(StandardOpenOption.READ); >> 111: // No lambdas during bootstrap >> 112: AccessController.doPrivileged(new PrivilegedAction() { > > Can probably remove this AccessController wrapping on JDK 25+, and just additionally catch SecurityException in case this fails in older releases where security manager exists. Why catch the `SecurityException` - why do we need to make JDK 25+ code work on older releases? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22882#discussion_r1910363981 From alanb at openjdk.org Fri Jan 10 13:28:40 2025 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 10 Jan 2025 13:28:40 GMT Subject: RFR: 8316882: Add NOT_INTERRUPTIBLE OpenOption and use it in ZipFS In-Reply-To: References: <16OW58lt-tWhc8B1G-yLedW2jG6XJwUCQumjKrwD7nU=.5daa8ce2-c4fa-4869-b302-bfa1008a6461@github.com> Message-ID: On Fri, 10 Jan 2025 13:14:36 GMT, Sean Mullan wrote: >> src/java.base/share/classes/jdk/internal/jimage/BasicImageReader.java line 112: >> >>> 110: opts.add(StandardOpenOption.READ); >>> 111: // No lambdas during bootstrap >>> 112: AccessController.doPrivileged(new PrivilegedAction() { >> >> Can probably remove this AccessController wrapping on JDK 25+, and just additionally catch SecurityException in case this fails in older releases where security manager exists. > > Why catch the `SecurityException` - why do we need to make JDK 25+ code work on older releases? The jrtfs provider is compiled --release 8 and packaging into jrt-fs.jar to allow tools on JDK 8+ to access the classes/resources in the run-time image. In any case, it's nothing to do with the topic that this PR is about, see discussion on nio-dev instead. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22882#discussion_r1910376495 From bpb at openjdk.org Fri Jan 10 16:11:46 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 10 Jan 2025 16:11:46 GMT Subject: RFR: 8346722: (fs) Files.probeContentType throws ClassCastException with custom file system provider In-Reply-To: References: Message-ID: On Fri, 10 Jan 2025 12:39:09 GMT, Alan Bateman wrote: > I would be worried that it returns something based on the name of the directory, is that possible? I doubt it, but it might be prudent to ignore the returned value to avert future noise. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/23019#discussion_r1910593145 From bpb at openjdk.org Fri Jan 10 16:16:02 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 10 Jan 2025 16:16:02 GMT Subject: RFR: 8346722: (fs) Files.probeContentType throws ClassCastException with custom file system provider [v2] In-Reply-To: References: Message-ID: > Ensure the file type detection works with a custom default file system provider. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8346722: Ignore value returned by probeContentType; prune imports in DefaultFileTypeDetector ------------- Changes: - all: https://git.openjdk.org/jdk/pull/23019/files - new: https://git.openjdk.org/jdk/pull/23019/files/f026d83b..1be60915 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=23019&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23019&range=00-01 Stats: 4 lines in 2 files changed: 0 ins; 4 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/23019.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/23019/head:pull/23019 PR: https://git.openjdk.org/jdk/pull/23019 From bpb at openjdk.org Fri Jan 10 16:18:38 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 10 Jan 2025 16:18:38 GMT Subject: RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk [v2] In-Reply-To: <0cnlSLtN3jdszmv_gOGCPm9alx0TzVrBCaVmD4aUnEE=.e6363ef9-d77f-4c9b-b5bb-a83781cd8201@github.com> References: <0cnlSLtN3jdszmv_gOGCPm9alx0TzVrBCaVmD4aUnEE=.e6363ef9-d77f-4c9b-b5bb-a83781cd8201@github.com> Message-ID: <0e6VJoDvSYulub-R_DhkJIG8U-Ht3oig8SV9DIbMhy0=.2040a4f5-5a11-4291-9de2-824f862b24bb@github.com> On Fri, 10 Jan 2025 09:04:20 GMT, Volkan Yaz?c? wrote: >> Switches from using the temporary directory to work directory in `LoopingTruncate`. This should effectively provide sufficient space for the test finish. Solution is tipped by @bplb. `tier1,2` tests' results will be attached to the ticket once they finish. > > Volkan Yaz?c? has updated the pull request incrementally with one additional commit since the last revision: > > Simplify and document changes in `LoopingTruncate` test/jdk/java/nio/channels/FileChannel/LoopingTruncate.java line 52: > 50: public static void main(String[] args) throws Throwable { > 51: Path path = Files.createTempFile( > 52: // Intentionally opting out from the default `java.io.tmpdir`. A comment within the parameter list seems a bit weird as does the indentation. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/23011#discussion_r1910603829 From bpb at openjdk.org Fri Jan 10 16:32:43 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 10 Jan 2025 16:32:43 GMT Subject: RFR: 8346722: (fs) Files.probeContentType throws ClassCastException with custom file system provider [v2] In-Reply-To: References: Message-ID: On Fri, 10 Jan 2025 16:08:54 GMT, Brian Burkhalter wrote: >> test/jdk/java/nio/file/spi/testapp/testapp/Main.java line 65: >> >>> 63: String fileType = Files.probeContentType(Path.of(".")); >>> 64: if (fileType != null) >>> 65: throw new RuntimeException("File type non-null: " + fileType); >> >> Calling Files.probeContentType is enough to exercise the code, I would be worried that it returns something based on the name of the directory, is that possible? > >> I would be worried that it returns something based on the name of the directory, is that possible? > > I doubt it, but it might be prudent to ignore the returned value to avert future noise. Returned value ignored in 1be6091. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/23019#discussion_r1910626328 From bpb at openjdk.org Fri Jan 10 16:32:42 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 10 Jan 2025 16:32:42 GMT Subject: RFR: 8346722: (fs) Files.probeContentType throws ClassCastException with custom file system provider [v2] In-Reply-To: References: Message-ID: On Fri, 10 Jan 2025 12:37:13 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8346722: Ignore value returned by probeContentType; prune imports in DefaultFileTypeDetector > > src/java.base/unix/classes/sun/nio/fs/DefaultFileTypeDetector.java line 36: > >> 34: >> 35: public static FileTypeDetector create() { >> 36: FileSystemProvider provider = FileSystems.getDefault().provider(); > > I assume the imports can be pruned now. Pruned in 1be6091. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/23019#discussion_r1910626243 From alanb at openjdk.org Fri Jan 10 16:57:45 2025 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 10 Jan 2025 16:57:45 GMT Subject: RFR: 8346722: (fs) Files.probeContentType throws ClassCastException with custom file system provider [v2] In-Reply-To: References: Message-ID: <_JsPmgcKZy5LqwnKl6trcaJpiIYIrbZWs-qLMk2B1Yw=.89251d46-cb29-4a89-b48e-aa8a9874d057@github.com> On Fri, 10 Jan 2025 16:16:02 GMT, Brian Burkhalter wrote: >> Ensure the file type detection works with a custom default file system provider. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8346722: Ignore value returned by probeContentType; prune imports in DefaultFileTypeDetector Marked as reviewed by alanb (Reviewer). We should probably add the @bug tag in SetDefaultProvider as we've been adding to this when adding tests for custom file system providers. ------------- PR Review: https://git.openjdk.org/jdk/pull/23019#pullrequestreview-2542968471 PR Comment: https://git.openjdk.org/jdk/pull/23019#issuecomment-2583259389 From bpb at openjdk.org Fri Jan 10 17:11:26 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 10 Jan 2025 17:11:26 GMT Subject: RFR: 8346722: (fs) Files.probeContentType throws ClassCastException with custom file system provider [v3] In-Reply-To: References: Message-ID: <81ZakjKclT52hn5YrEKbH9C2f0IDm9YQRQThx4byq0M=.5f379cce-fb9e-4225-896e-28b9b6711cc9@github.com> > Ensure the file type detection works with a custom default file system provider. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8346722: Add bug tag to test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/23019/files - new: https://git.openjdk.org/jdk/pull/23019/files/1be60915..a8b205a7 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=23019&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23019&range=01-02 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/23019.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/23019/head:pull/23019 PR: https://git.openjdk.org/jdk/pull/23019 From bpb at openjdk.org Fri Jan 10 17:11:26 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 10 Jan 2025 17:11:26 GMT Subject: RFR: 8346722: (fs) Files.probeContentType throws ClassCastException with custom file system provider [v2] In-Reply-To: <_JsPmgcKZy5LqwnKl6trcaJpiIYIrbZWs-qLMk2B1Yw=.89251d46-cb29-4a89-b48e-aa8a9874d057@github.com> References: <_JsPmgcKZy5LqwnKl6trcaJpiIYIrbZWs-qLMk2B1Yw=.89251d46-cb29-4a89-b48e-aa8a9874d057@github.com> Message-ID: On Fri, 10 Jan 2025 16:54:34 GMT, Alan Bateman wrote: > We should probably add the @bug tag in SetDefaultProvider [...]. Done in [a8b205a](https://github.com/openjdk/jdk/pull/23019/commits/a8b205a70655c31e58acda0376a50708c30beac4). ------------- PR Comment: https://git.openjdk.org/jdk/pull/23019#issuecomment-2583283562 From alanb at openjdk.org Fri Jan 10 17:13:42 2025 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 10 Jan 2025 17:13:42 GMT Subject: RFR: 8346722: (fs) Files.probeContentType throws ClassCastException with custom file system provider [v3] In-Reply-To: <81ZakjKclT52hn5YrEKbH9C2f0IDm9YQRQThx4byq0M=.5f379cce-fb9e-4225-896e-28b9b6711cc9@github.com> References: <81ZakjKclT52hn5YrEKbH9C2f0IDm9YQRQThx4byq0M=.5f379cce-fb9e-4225-896e-28b9b6711cc9@github.com> Message-ID: On Fri, 10 Jan 2025 17:11:26 GMT, Brian Burkhalter wrote: >> Ensure the file type detection works with a custom default file system provider. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8346722: Add bug tag to test Marked as reviewed by alanb (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/23019#pullrequestreview-2543001980 From bpb at openjdk.org Fri Jan 10 18:53:58 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 10 Jan 2025 18:53:58 GMT Subject: Integrated: 8346722: (fs) Files.probeContentType throws ClassCastException with custom file system provider In-Reply-To: References: Message-ID: On Thu, 9 Jan 2025 20:56:20 GMT, Brian Burkhalter wrote: > Ensure the file type detection works with a custom default file system provider. This pull request has now been integrated. Changeset: 4b554b52 Author: Brian Burkhalter URL: https://git.openjdk.org/jdk/commit/4b554b52e627b3a0f5e15a623b12089bff6bfafa Stats: 11 lines in 3 files changed: 4 ins; 3 del; 4 mod 8346722: (fs) Files.probeContentType throws ClassCastException with custom file system provider Reviewed-by: alanb ------------- PR: https://git.openjdk.org/jdk/pull/23019 From syan at openjdk.org Mon Jan 13 03:19:07 2025 From: syan at openjdk.org (SendaoYan) Date: Mon, 13 Jan 2025 03:19:07 GMT Subject: [jdk24] RFR: 8347171: (dc) java/nio/channels/DatagramChannel/InterruptibleOrNot.java fails with virtual thread factory Message-ID: <7kT3-oGr1Z8kzkhzXT4o6V0pARyqeDkLjHowIzwhCmE=.63ac6a5d-428a-49ba-8d45-1f61d27d56d6@github.com> Hi all, This pull request contains a backport of commit [1ef77cdd](https://github.com/openjdk/jdk/commit/1ef77cdd51b91f6d6d3367444a37a3f0f2e4bc99) from the [openjdk/jdk](https://git.openjdk.org/jdk) repository. The commit being backported was authored by Brian Burkhalter on 9 Jan 2025 and was reviewed by Alan Bateman and Leonid Mesnik. Thanks! ------------- Commit messages: - Backport 1ef77cdd51b91f6d6d3367444a37a3f0f2e4bc99 Changes: https://git.openjdk.org/jdk/pull/23060/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23060&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8347171 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/23060.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/23060/head:pull/23060 PR: https://git.openjdk.org/jdk/pull/23060 From duke at openjdk.org Mon Jan 13 07:58:56 2025 From: duke at openjdk.org (Volkan =?UTF-8?B?WWF6xLFjxLE=?=) Date: Mon, 13 Jan 2025 07:58:56 GMT Subject: RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk [v3] In-Reply-To: References: Message-ID: <0nSLz809cW2xfxZ71E_IMT3Yf7piBhFn_cViXU0chvQ=.299c671b-017f-4960-b101-0afce0f191cf@github.com> > Switches from using the temporary directory to work directory in `LoopingTruncate`. This should effectively provide sufficient space for the test finish. Solution is tipped by @bplb. CI results are attached to the ticket. Volkan Yaz?c? has updated the pull request incrementally with one additional commit since the last revision: Improve comment style ------------- Changes: - all: https://git.openjdk.org/jdk/pull/23011/files - new: https://git.openjdk.org/jdk/pull/23011/files/8fc22379..909572e5 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=23011&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23011&range=01-02 Stats: 5 lines in 1 file changed: 0 ins; 1 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/23011.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/23011/head:pull/23011 PR: https://git.openjdk.org/jdk/pull/23011 From duke at openjdk.org Mon Jan 13 07:58:57 2025 From: duke at openjdk.org (Volkan =?UTF-8?B?WWF6xLFjxLE=?=) Date: Mon, 13 Jan 2025 07:58:57 GMT Subject: RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk [v2] In-Reply-To: <0e6VJoDvSYulub-R_DhkJIG8U-Ht3oig8SV9DIbMhy0=.2040a4f5-5a11-4291-9de2-824f862b24bb@github.com> References: <0cnlSLtN3jdszmv_gOGCPm9alx0TzVrBCaVmD4aUnEE=.e6363ef9-d77f-4c9b-b5bb-a83781cd8201@github.com> <0e6VJoDvSYulub-R_DhkJIG8U-Ht3oig8SV9DIbMhy0=.2040a4f5-5a11-4291-9de2-824f862b24bb@github.com> Message-ID: <6BQebf0WcZNKRpnzUe7KVvQ55zR4u-XScJGb16NytkE=.5efd8cdf-a219-4e41-83dd-1a193830dad8@github.com> On Fri, 10 Jan 2025 16:15:59 GMT, Brian Burkhalter wrote: >> Volkan Yaz?c? has updated the pull request incrementally with one additional commit since the last revision: >> >> Simplify and document changes in `LoopingTruncate` > > test/jdk/java/nio/channels/FileChannel/LoopingTruncate.java line 52: > >> 50: public static void main(String[] args) throws Throwable { >> 51: Path path = Files.createTempFile( >> 52: // Intentionally opting out from the default `java.io.tmpdir`. > > A comment within the parameter list seems a bit weird as does the indentation. I chose this style since the comment applies to a single argument and is only relevant within `createTempFile()`. Reverted to the old style in 909572e56d87d6a1411c2dba23cfd4c75eaa52ae. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/23011#discussion_r1912787403 From ihse at openjdk.org Mon Jan 13 12:22:53 2025 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 13 Jan 2025 12:22:53 GMT Subject: RFR: 8337143: (fc, fs) Move filesystem-related native objects from libnio to libjava [v7] In-Reply-To: References: <1UT8TbjP6frWkE-NRX1jv14uRfOy2RtrBHufgh6TCfg=.b2711a71-7935-46a9-8f39-b22ce95ef901@github.com> Message-ID: On Thu, 12 Sep 2024 15:43:32 GMT, Alan Bateman wrote: >>> Thanks for making the changes. LGTM, assuming that tests still pass. >> >> The tests passed the JDK tiers 1-3 tests on Linux, macOS, and Windows. In any case, I will run another round of tests before integrating. > >> The tests passed the JDK tiers 1-3 tests on Linux, macOS, and Windows. In any case, I will run another round of tests before integrating. > > Can you hold off integrating, I plan to go over the changes soon. @AlanBateman I think what is holding this PR back is that you said you wanted to review it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/20317#issuecomment-2586958882 From alanb at openjdk.org Mon Jan 13 12:37:38 2025 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 13 Jan 2025 12:37:38 GMT Subject: RFR: 8337143: (fc, fs) Move filesystem-related native objects from libnio to libjava [v7] In-Reply-To: References: <1UT8TbjP6frWkE-NRX1jv14uRfOy2RtrBHufgh6TCfg=.b2711a71-7935-46a9-8f39-b22ce95ef901@github.com> Message-ID: On Thu, 12 Sep 2024 15:43:32 GMT, Alan Bateman wrote: >>> Thanks for making the changes. LGTM, assuming that tests still pass. >> >> The tests passed the JDK tiers 1-3 tests on Linux, macOS, and Windows. In any case, I will run another round of tests before integrating. > >> The tests passed the JDK tiers 1-3 tests on Linux, macOS, and Windows. In any case, I will run another round of tests before integrating. > > Can you hold off integrating, I plan to go over the changes soon. > @AlanBateman I think what is holding this PR back is that you said you wanted to review it. I asked Brian to hold off integrating this as I need time to work through several issues. Note that this is just one part of overall solution, we don't know yet if this changes in this PR will help the startup issue that motivated it (in isolation, the changes in this PR don't make sense of course). ------------- PR Comment: https://git.openjdk.org/jdk/pull/20317#issuecomment-2586989708 From ihse at openjdk.org Mon Jan 13 13:25:51 2025 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 13 Jan 2025 13:25:51 GMT Subject: RFR: 8337143: (fc, fs) Move filesystem-related native objects from libnio to libjava [v11] In-Reply-To: <0Wfl1u5_aLuRYUaugkrqW-VOFwz8-r-RHmfG7ikXm_I=.fd8a5950-e155-47ba-847c-91e2397c6dd1@github.com> References: <0Wfl1u5_aLuRYUaugkrqW-VOFwz8-r-RHmfG7ikXm_I=.fd8a5950-e155-47ba-847c-91e2397c6dd1@github.com> Message-ID: <3a81ilsvQuU-PhG4LjAyj2mri7Yn2XikJgpmk3nwAro=.27e1ca09-4d2d-45e5-bbaa-3024450d6d39@github.com> On Thu, 5 Dec 2024 18:44:06 GMT, Brian Burkhalter wrote: >> This proposed change would move the native objects required for NIO file interaction from the libnio native library to the libjava native library on Linux, macOS, and Windows. > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 11 commits: > > - Merge > - Merge > - Merge > - 8337143: Minor makefile tweak > - 8337143: Clean up to address reviewer comments > - Merge > - 8337143: Remove loading libnet from Inet6AddressImpl; delete commented out #include in Windows IOUtil.c > - Merge > - 8337143: Removed dependency of libjava on headers in libnio/ch > - 8337143: Move natives to /native/libjava/nio/{ch,fs} as a function of their original location in libnio > - ... and 1 more: https://git.openjdk.org/jdk/compare/bedb68ab...2cb82267 I see. ------------- PR Comment: https://git.openjdk.org/jdk/pull/20317#issuecomment-2587094183 From bpb at openjdk.org Mon Jan 13 16:28:40 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 13 Jan 2025 16:28:40 GMT Subject: RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk [v3] In-Reply-To: <0nSLz809cW2xfxZ71E_IMT3Yf7piBhFn_cViXU0chvQ=.299c671b-017f-4960-b101-0afce0f191cf@github.com> References: <0nSLz809cW2xfxZ71E_IMT3Yf7piBhFn_cViXU0chvQ=.299c671b-017f-4960-b101-0afce0f191cf@github.com> Message-ID: On Mon, 13 Jan 2025 07:58:56 GMT, Volkan Yaz?c? wrote: >> Switches from using the temporary directory to work directory in `LoopingTruncate`. This should effectively provide sufficient space for the test finish. Solution is tipped by @bplb. CI results are attached to the ticket. > > Volkan Yaz?c? has updated the pull request incrementally with one additional commit since the last revision: > > Improve comment style Looks good. ------------- Marked as reviewed by bpb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/23011#pullrequestreview-2547166666 From duke at openjdk.org Mon Jan 13 18:58:53 2025 From: duke at openjdk.org (duke) Date: Mon, 13 Jan 2025 18:58:53 GMT Subject: RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk [v3] In-Reply-To: <0nSLz809cW2xfxZ71E_IMT3Yf7piBhFn_cViXU0chvQ=.299c671b-017f-4960-b101-0afce0f191cf@github.com> References: <0nSLz809cW2xfxZ71E_IMT3Yf7piBhFn_cViXU0chvQ=.299c671b-017f-4960-b101-0afce0f191cf@github.com> Message-ID: On Mon, 13 Jan 2025 07:58:56 GMT, Volkan Yaz?c? wrote: >> Switches from using the temporary directory to work directory in `LoopingTruncate`. This should effectively provide sufficient space for the test finish. Solution is tipped by @bplb. CI results are attached to the ticket. > > Volkan Yaz?c? has updated the pull request incrementally with one additional commit since the last revision: > > Improve comment style @vy Your change (at version 909572e56d87d6a1411c2dba23cfd4c75eaa52ae) is now ready to be sponsored by a Committer. ------------- PR Comment: https://git.openjdk.org/jdk/pull/23011#issuecomment-2587949929 From duke at openjdk.org Mon Jan 13 19:02:46 2025 From: duke at openjdk.org (Volkan =?UTF-8?B?WWF6xLFjxLE=?=) Date: Mon, 13 Jan 2025 19:02:46 GMT Subject: Integrated: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk In-Reply-To: References: Message-ID: On Thu, 9 Jan 2025 15:23:55 GMT, Volkan Yaz?c? wrote: > Switches from using the temporary directory to work directory in `LoopingTruncate`. This should effectively provide sufficient space for the test finish. Solution is tipped by @bplb. CI results are attached to the ticket. This pull request has now been integrated. Changeset: 4e0ffda5 Author: Volkan Yaz?c? Committer: Brian Burkhalter URL: https://git.openjdk.org/jdk/commit/4e0ffda5b1d82449d2d6f639be7641b69d6cb520 Stats: 10 lines in 1 file changed: 5 ins; 2 del; 3 mod 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk Co-authored-by: Brian Burkhalter Reviewed-by: bpb ------------- PR: https://git.openjdk.org/jdk/pull/23011 From bpb at openjdk.org Mon Jan 13 19:24:16 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 13 Jan 2025 19:24:16 GMT Subject: RFR: 8347286: (fs) Remove some extensions from java/nio/file/Files/probeContentType/Basic.java Message-ID: Remove testing on Windows of file extensions which do not consistently map to the same content type across different versions of Windows. ------------- Commit messages: - 8347286: (fs) Remove some extensions from java/nio/file/Files/probeContentType/Basic.java Changes: https://git.openjdk.org/jdk/pull/23085/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23085&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8347286 Stats: 12 lines in 1 file changed: 0 ins; 10 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/23085.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/23085/head:pull/23085 PR: https://git.openjdk.org/jdk/pull/23085 From bpb at openjdk.org Mon Jan 13 19:24:16 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 13 Jan 2025 19:24:16 GMT Subject: RFR: 8347286: (fs) Remove some extensions from java/nio/file/Files/probeContentType/Basic.java In-Reply-To: References: Message-ID: On Mon, 13 Jan 2025 19:18:20 GMT, Brian Burkhalter wrote: > Remove testing on Windows of file extensions which do not consistently map to the same content type across different versions of Windows. This change should eliminate future problems which are likely to occur as new versions of Windows appear. ------------- PR Comment: https://git.openjdk.org/jdk/pull/23085#issuecomment-2587994428 From syan at openjdk.org Tue Jan 14 01:47:06 2025 From: syan at openjdk.org (SendaoYan) Date: Tue, 14 Jan 2025 01:47:06 GMT Subject: [jdk24] RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk Message-ID: Hi all, This pull request contains a backport of commit [4e0ffda5](https://github.com/openjdk/jdk/commit/4e0ffda5b1d82449d2d6f639be7641b69d6cb520) from the [openjdk/jdk](https://git.openjdk.org/jdk) repository to jdk24. This backport make test more rubustness, test-fix only, no risk. Thanks! ------------- Commit messages: - Backport 4e0ffda5b1d82449d2d6f639be7641b69d6cb520 Changes: https://git.openjdk.org/jdk/pull/23094/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23094&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8346972 Stats: 10 lines in 1 file changed: 5 ins; 2 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/23094.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/23094/head:pull/23094 PR: https://git.openjdk.org/jdk/pull/23094 From alanb at openjdk.org Tue Jan 14 07:18:41 2025 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 14 Jan 2025 07:18:41 GMT Subject: RFR: 8347286: (fs) Remove some extensions from java/nio/file/Files/probeContentType/Basic.java In-Reply-To: References: Message-ID: On Mon, 13 Jan 2025 19:18:20 GMT, Brian Burkhalter wrote: > Remove testing on Windows of file extensions which do not consistently map to the same content type across different versions of Windows. Marked as reviewed by alanb (Reviewer). test/jdk/java/nio/file/Files/probeContentType/Basic.java line 196: > 194: if (OperatingSystem.isWindows() && > 195: (StaticProperty.osName().matches("^.*[11|2025]$") || > 196: new OSVersion(10, 0).compareTo(OSVersion.current()) > 0)) { I assume the import of OSVersion isn't needed now. ------------- PR Review: https://git.openjdk.org/jdk/pull/23085#pullrequestreview-2548958623 PR Review Comment: https://git.openjdk.org/jdk/pull/23085#discussion_r1914349985 From jpai at openjdk.org Tue Jan 14 09:30:35 2025 From: jpai at openjdk.org (Jaikiran Pai) Date: Tue, 14 Jan 2025 09:30:35 GMT Subject: RFR: 8347286: (fs) Remove some extensions from java/nio/file/Files/probeContentType/Basic.java In-Reply-To: References: Message-ID: On Mon, 13 Jan 2025 19:18:20 GMT, Brian Burkhalter wrote: > Remove testing on Windows of file extensions which do not consistently map to the same content type across different versions of Windows. Looks OK to me. ------------- Marked as reviewed by jpai (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/23085#pullrequestreview-2549201550 From bpb at openjdk.org Tue Jan 14 15:30:01 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 14 Jan 2025 15:30:01 GMT Subject: RFR: 8347286: (fs) Remove some extensions from java/nio/file/Files/probeContentType/Basic.java [v2] In-Reply-To: References: Message-ID: > Remove testing on Windows of file extensions which do not consistently map to the same content type across different versions of Windows. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8347286: Remove import of OSVersion ------------- Changes: - all: https://git.openjdk.org/jdk/pull/23085/files - new: https://git.openjdk.org/jdk/pull/23085/files/b25fa735..0e0b66a7 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=23085&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=23085&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/23085.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/23085/head:pull/23085 PR: https://git.openjdk.org/jdk/pull/23085 From alanb at openjdk.org Tue Jan 14 15:30:01 2025 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 14 Jan 2025 15:30:01 GMT Subject: RFR: 8347286: (fs) Remove some extensions from java/nio/file/Files/probeContentType/Basic.java [v2] In-Reply-To: References: Message-ID: On Tue, 14 Jan 2025 15:26:58 GMT, Brian Burkhalter wrote: >> Remove testing on Windows of file extensions which do not consistently map to the same content type across different versions of Windows. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8347286: Remove import of OSVersion Marked as reviewed by alanb (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/23085#pullrequestreview-2550084640 From bpb at openjdk.org Tue Jan 14 15:30:01 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 14 Jan 2025 15:30:01 GMT Subject: RFR: 8347286: (fs) Remove some extensions from java/nio/file/Files/probeContentType/Basic.java [v2] In-Reply-To: References: Message-ID: On Tue, 14 Jan 2025 07:15:37 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8347286: Remove import of OSVersion > > test/jdk/java/nio/file/Files/probeContentType/Basic.java line 196: > >> 194: if (OperatingSystem.isWindows() && >> 195: (StaticProperty.osName().matches("^.*[11|2025]$") || >> 196: new OSVersion(10, 0).compareTo(OSVersion.current()) > 0)) { > > I assume the import of OSVersion isn't needed now. Removed in 0e0b66a. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/23085#discussion_r1915033959 From bpb at openjdk.org Tue Jan 14 15:55:50 2025 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 14 Jan 2025 15:55:50 GMT Subject: Integrated: 8347286: (fs) Remove some extensions from java/nio/file/Files/probeContentType/Basic.java In-Reply-To: References: Message-ID: <69bataQt3C7Yr66c2_HA5_cB1H_g-PmZGQ9wWEU4yoQ=.6b59ad77-f9e3-4c6e-a87f-14353a527802@github.com> On Mon, 13 Jan 2025 19:18:20 GMT, Brian Burkhalter wrote: > Remove testing on Windows of file extensions which do not consistently map to the same content type across different versions of Windows. This pull request has now been integrated. Changeset: d002933c Author: Brian Burkhalter URL: https://git.openjdk.org/jdk/commit/d002933c260921d0d582724516d15ebd130b851f Stats: 13 lines in 1 file changed: 0 ins; 11 del; 2 mod 8347286: (fs) Remove some extensions from java/nio/file/Files/probeContentType/Basic.java Reviewed-by: alanb, jpai ------------- PR: https://git.openjdk.org/jdk/pull/23085 From eirbjo at openjdk.org Thu Jan 16 20:42:52 2025 From: eirbjo at openjdk.org (Eirik =?UTF-8?B?QmrDuHJzbsO4cw==?=) Date: Thu, 16 Jan 2025 20:42:52 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes Message-ID: Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. The classes were identified using an automated analysis. See CSR for details. Besides simply adding the `final` access modifier, the PR: * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` * Updates the copyright year of affected source files ------------- Commit messages: - Merge branch 'master' into non-subclassable-final - Update copyright years - Runtime class is marked final, remove it from the CheckCSMs test - Leave out the constructor access for now since the are not relevant for the CSR review - Reword note about immutability and object immutability to reference object instances instead of the class. - Resurrect note about immutability and object identity, but without any reference to subtypes - Mark non-subclassable classes in 'java.base' final Changes: https://git.openjdk.org/jdk/pull/22389/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=22389&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8344943 Stats: 35 lines in 22 files changed: 0 ins; 3 del; 32 mod Patch: https://git.openjdk.org/jdk/pull/22389.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22389/head:pull/22389 PR: https://git.openjdk.org/jdk/pull/22389 From eirbjo at openjdk.org Thu Jan 16 20:42:53 2025 From: eirbjo at openjdk.org (Eirik =?UTF-8?B?QmrDuHJzbsO4cw==?=) Date: Thu, 16 Jan 2025 20:42:53 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes In-Reply-To: References: Message-ID: On Tue, 26 Nov 2024 18:49:07 GMT, Alan Bateman wrote: >> Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. >> >> The classes were identified using an automated analysis. See CSR for details. >> >> Besides simply adding the `final` access modifier, the PR: >> >> * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. >> * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` >> * Updates the copyright year of affected source files > > src/java.base/share/classes/java/lang/module/ModuleDescriptor.java line 1318: > >> 1316: Version version, >> 1317: Set modifiers, >> 1318: Set requires, > > I assume you'll fix the alignment if this goes ahead. Thanks! Yes, that's good feedback. There are some non-API changes that I'd like to make, such as this and updating the comment for the `InterfaceAddress` constructor. I prefered to make the initial PR draft contain just the CSR changes, such that PR would be useful and clean as a Specification change, since the diff is a bit noisy. I'll keep your review comment here unresolved for now and get back to it pending CSR approval. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22389#discussion_r1859084454 From liach at openjdk.org Thu Jan 16 20:42:52 2025 From: liach at openjdk.org (Chen Liang) Date: Thu, 16 Jan 2025 20:42:52 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes In-Reply-To: References: Message-ID: On Tue, 26 Nov 2024 13:04:41 GMT, Eirik Bj?rsn?s wrote: > Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. > > The classes were identified using an automated analysis. See CSR for details. > > Besides simply adding the `final` access modifier, the PR: > > * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. > * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` > * Updates the copyright year of affected source files Changes requested by liach (Reviewer). Looks good for 25. src/java.base/share/classes/java/lang/constant/DynamicCallSiteDesc.java line 44: > 42: * {@code invokedynamic} call site. > 43: * > 44: *

This class is immutable and its behavior does not rely on object identity Suggestion: *

A {@code DynamicCallSiteDesc} is immutable and its behavior does not * rely on object identity. This describes an object, not a class, and please close sentences with a period. (We use fragments for param and return block tags instead) src/java.base/share/classes/java/lang/constant/DynamicCallSiteDesc.java line 45: > 43: * > 44: *

Concrete subtypes of {@linkplain DynamicCallSiteDesc} should be immutable > 45: * and their behavior should not rely on object identity. Please reword this to something like: {@code DynamicCallSiteDesc} is immutable and its behavior does not rely on object identity. This is given in `ConstantDesc`, but `DynamicCallSiteDesc` does not extend `ConstantDesc` so the removal is dubious. src/java.base/share/classes/java/net/InterfaceAddress.java line 45: > 43: > 44: /* > 45: * Package private constructor. Can't be built directly, instances are The comment is outdated. This constructor is actually called via JNI like `(*env)->NewObject(env, ni_ibcls, ni_ibctrID)` in `NetworkInterface.c`. I think we need to consult a network engineer to decide what is the best way to comment on this constructor. ------------- PR Review: https://git.openjdk.org/jdk/pull/22389#pullrequestreview-2462343034 Marked as reviewed by liach (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/22389#pullrequestreview-2500067616 PR Review Comment: https://git.openjdk.org/jdk/pull/22389#discussion_r1859112730 PR Review Comment: https://git.openjdk.org/jdk/pull/22389#discussion_r1859019019 PR Review Comment: https://git.openjdk.org/jdk/pull/22389#discussion_r1859157909 From eirbjo at openjdk.org Thu Jan 16 20:42:53 2025 From: eirbjo at openjdk.org (Eirik =?UTF-8?B?QmrDuHJzbsO4cw==?=) Date: Thu, 16 Jan 2025 20:42:53 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes In-Reply-To: References: Message-ID: On Tue, 26 Nov 2024 18:55:50 GMT, Eirik Bj?rsn?s wrote: >> src/java.base/share/classes/java/lang/module/ModuleDescriptor.java line 1318: >> >>> 1316: Version version, >>> 1317: Set modifiers, >>> 1318: Set requires, >> >> I assume you'll fix the alignment if this goes ahead. > > Thanks! Yes, that's good feedback. There are some non-API changes that I'd like to make, such as this and updating the comment for the `InterfaceAddress` constructor. > > I prefered to make the initial PR draft contain just the CSR changes, such that PR would be useful and clean as a Specification change, since the diff is a bit noisy. > > I'll keep your review comment here unresolved for now and get back to it pending CSR approval. For the record: It was not completely clear to me whether restricting these two package-private constructors to private is considered a specification change in the CSR sense. I opted to point them out in the CSR, just to make the change abundantly clear. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22389#discussion_r1859098445 From alanb at openjdk.org Thu Jan 16 20:42:53 2025 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 16 Jan 2025 20:42:53 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes In-Reply-To: References: Message-ID: On Tue, 26 Nov 2024 13:04:41 GMT, Eirik Bj?rsn?s wrote: > Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. > > The classes were identified using an automated analysis. See CSR for details. > > Besides simply adding the `final` access modifier, the PR: > > * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. > * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` > * Updates the copyright year of affected source files src/java.base/share/classes/java/lang/module/ModuleDescriptor.java line 1318: > 1316: Version version, > 1317: Set modifiers, > 1318: Set requires, I assume you'll fix the alignment if this goes ahead. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22389#discussion_r1859076834 From eirbjo at openjdk.org Thu Jan 16 20:42:52 2025 From: eirbjo at openjdk.org (Eirik =?UTF-8?B?QmrDuHJzbsO4cw==?=) Date: Thu, 16 Jan 2025 20:42:52 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes In-Reply-To: References: Message-ID: <_1XUsFhYyvc8iVWXtjOIkOD_zIsfb8LTT5HwgMq3I_g=.4c6b4570-5a83-4be6-ab30-37878978313e@github.com> On Tue, 26 Nov 2024 13:04:41 GMT, Eirik Bj?rsn?s wrote: > Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. > > The classes were identified using an automated analysis. See CSR for details. > > Besides simply adding the `final` access modifier, the PR: > > * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. > * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` > * Updates the copyright year of affected source files Now that the CSR for this change is "final-ly" approved, I'm marking this PR ready for review. Since the CSR approval, the PR has merged with a recent master and the copyright headers are updated for 2025. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22389#issuecomment-2596848026 From eirbjo at openjdk.org Thu Jan 16 20:42:52 2025 From: eirbjo at openjdk.org (Eirik =?UTF-8?B?QmrDuHJzbsO4cw==?=) Date: Thu, 16 Jan 2025 20:42:52 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes In-Reply-To: References: Message-ID: On Tue, 26 Nov 2024 18:00:21 GMT, Chen Liang wrote: >> Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. >> >> The classes were identified using an automated analysis. See CSR for details. >> >> Besides simply adding the `final` access modifier, the PR: >> >> * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. >> * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` >> * Updates the copyright year of affected source files > > Changes requested by liach (Reviewer). Following some offline discussion with @liach, we decided to leave out the constructor access updates in `ModuleDescriptor` and `InterfaceAddress` for now. They are not considered a specification change and adds noise for the CSR review. These changes may be revisited in this PR pending CSR approval, or can be addressed in follow-up PRs. > src/java.base/share/classes/java/lang/constant/DynamicCallSiteDesc.java line 44: > >> 42: * {@code invokedynamic} call site. >> 43: * >> 44: *

This class is immutable and its behavior does not rely on object identity > > Suggestion: > > *

A {@code DynamicCallSiteDesc} is immutable and its behavior does not > * rely on object identity. > > This describes an object, not a class, and please close sentences with a period. > (We use fragments for param and return block tags instead) Thanks, fixed in 6156b46 :) > src/java.base/share/classes/java/lang/constant/DynamicCallSiteDesc.java line 45: > >> 43: * >> 44: *

Concrete subtypes of {@linkplain DynamicCallSiteDesc} should be immutable >> 45: * and their behavior should not rely on object identity. > > Please reword this to something like: > > {@code DynamicCallSiteDesc} is immutable and its behavior does not rely on object identity. > > This is given in `ConstantDesc`, but `DynamicCallSiteDesc` does not extend `ConstantDesc` so the removal is dubious. Thanks, I've updated the PR and CSR to retain the note about immutability and object identity, removing the reference to subtypes: This class is immutable and its behavior does not rely on object identity I opted to replace the self-reference `{@code DynamicCallSiteDesc}` with just "This class" here. Let me know if you prefer spelling out the class name. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22389#issuecomment-2501808953 PR Review Comment: https://git.openjdk.org/jdk/pull/22389#discussion_r1859123165 PR Review Comment: https://git.openjdk.org/jdk/pull/22389#discussion_r1859051463 From liach at openjdk.org Thu Jan 16 20:42:53 2025 From: liach at openjdk.org (Chen Liang) Date: Thu, 16 Jan 2025 20:42:53 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes In-Reply-To: References: Message-ID: On Tue, 26 Nov 2024 19:08:26 GMT, Eirik Bj?rsn?s wrote: >> Thanks! Yes, that's good feedback. There are some non-API changes that I'd like to make, such as this and updating the comment for the `InterfaceAddress` constructor. >> >> I prefered to make the initial PR draft contain just the CSR changes, such that PR would be useful and clean as a Specification change, since the diff is a bit noisy. >> >> I'll keep your review comment here unresolved for now and get back to it pending CSR approval. > > For the record: > > It was not completely clear to me whether restricting these two package-private constructors to private is considered a specification change in the CSR sense. I opted to point them out in the CSR, just to make the change abundantly clear. This constructor is not part of the public API: package-private is not accessible, and it does not appear in the API docs: https://docs.oracle.com/en/java/javase/23/docs/api/java.base/java/lang/module/ModuleDescriptor.html ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22389#discussion_r1859114697 From liach at openjdk.org Thu Jan 16 20:42:53 2025 From: liach at openjdk.org (Chen Liang) Date: Thu, 16 Jan 2025 20:42:53 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes In-Reply-To: References: Message-ID: <4ifcIGdOnz0NbgxfK5Yn8YMoEOKHOvRv4rrG7XOvwZQ=.9d79707f-3c28-4a3a-86a1-bd1c7f5233c3@github.com> On Tue, 26 Nov 2024 20:03:39 GMT, Chen Liang wrote: >> Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. >> >> The classes were identified using an automated analysis. See CSR for details. >> >> Besides simply adding the `final` access modifier, the PR: >> >> * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. >> * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` >> * Updates the copyright year of affected source files > > src/java.base/share/classes/java/net/InterfaceAddress.java line 45: > >> 43: >> 44: /* >> 45: * Package private constructor. Can't be built directly, instances are > > The comment is outdated. This constructor is actually called via JNI like `(*env)->NewObject(env, ni_ibcls, ni_ibctrID)` in `NetworkInterface.c`. I think we need to consult a network engineer to decide what is the best way to comment on this constructor. A pleasant surprise - since we aren't modifying constructor modifiers, we can put these intricacies on hold. Thanks for limiting the scope of this patch. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22389#discussion_r1859162790 From eirbjo at openjdk.org Thu Jan 16 20:42:53 2025 From: eirbjo at openjdk.org (Eirik =?UTF-8?B?QmrDuHJzbsO4cw==?=) Date: Thu, 16 Jan 2025 20:42:53 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes In-Reply-To: <4ifcIGdOnz0NbgxfK5Yn8YMoEOKHOvRv4rrG7XOvwZQ=.9d79707f-3c28-4a3a-86a1-bd1c7f5233c3@github.com> References: <4ifcIGdOnz0NbgxfK5Yn8YMoEOKHOvRv4rrG7XOvwZQ=.9d79707f-3c28-4a3a-86a1-bd1c7f5233c3@github.com> Message-ID: <1Osv6B_tLqnArJDx89PKGl7Qy_o_oBU5Nxb2wpKiSQY=.59527d3c-7ded-49d8-8944-b84bf154d501@github.com> On Tue, 26 Nov 2024 20:08:28 GMT, Chen Liang wrote: >> src/java.base/share/classes/java/net/InterfaceAddress.java line 45: >> >>> 43: >>> 44: /* >>> 45: * Package private constructor. Can't be built directly, instances are >> >> The comment is outdated. This constructor is actually called via JNI like `(*env)->NewObject(env, ni_ibcls, ni_ibctrID)` in `NetworkInterface.c`. I think we need to consult a network engineer to decide what is the best way to comment on this constructor. > > A pleasant surprise - since we aren't modifying constructor modifiers, we can put these intricacies on hold. Thanks for limiting the scope of this patch. Indeed, this makes this PR much simpler. I have filed separate JBS issues to track these constructor updates: https://bugs.openjdk.org/browse/JDK-8345074 for`InterfaceAddress` https://bugs.openjdk.org/browse/JDK-8345075 for `ModuleDescriptor` ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22389#discussion_r1859196447 From bruno.ploumhans at gmail.com Thu Jan 16 21:42:06 2025 From: bruno.ploumhans at gmail.com (Bruno Ploumhans) Date: Thu, 16 Jan 2025 22:42:06 +0100 Subject: NOT_INTERRUPTIBLE OpenOption and how that should fit with InterruptibleChanne In-Reply-To: References: Message-ID: The different options are a good way to see it. I think that #1 is too breaking. Among the others, - #2 does not change default behavior so it is the least breaking, - #3 does change default behavior, but allows opting back into old behavior. - #4 does change default behavior, with no option to revert to the old behavior. #3 is weird in that it changes the default behavior, yet acknowledges that it's a breaking change that might be undesirable. Beyond the compatibility impact, implementing #4 also removes the ability to interrupt potentially blocking network file operations, which might already make it a no-go? See [1] for an example of a FileChannel that wraps a blocking network operation; I am not sure how relevant it is. > #2 [...] is also the least satisfactory option too. My $0.02 as a low-experience NIO user: I can understand why a FileChannel would be interruptible, and as long as I can opt out of that behavior I am happy. One would just have to learn the hard way that FileChannels are interruptible by default... Best, Bruno [1]: https://github.com/apache/mina-sshd/blob/master/sshd-sftp/src/main/java/org/apache/sshd/sftp/client/impl/SftpRemotePathChannel.java Le dim. 5 janv. 2025 ? 10:53, Alan Bateman a ?crit : > > On 26/12/2024 10:15, Bruno Ploumhans wrote: > > Hi, > > I opened a PR that adds StandardOpenOptions.NOT_INTERRUPTIBLE to > expose and supersede FileChannelImpl#setUninterruptible. However, this > needs more discussion because FileChannel implements > InterruptibleChannel. > > Here are some thoughts to kick-start the discussion: > - The documentation of InterruptibleChannel does not currently account > for implementations that might not be interruptible. Same for the > documentation of Thread.interrupt(). That needs changing. > - Alan suggested in previous discussion (2018) a new `boolean > isInterruptible()` method. > - I wonder if the ability to pass a `boolean uninterruptible` flag > should be exposed in AbstractUninterruptibleChannel, or if that should > be kept an implementation detail of FileChannelImpl. > > In any case, I am happy to experiment with various solutions. > > This is a topic that I've looked several times including explorations and prototypes of four options: > > 1. Changing FileChannel to not extend AbstractInterruptibleChannel > 2. An OpenOption to opt out of interrupt closing the channel > 3. An OpenOption to opt in to interrupt closing the channel > 4. Specifying that interrupt does not close the channel but specified to allow for compatibility with long standing behavior > > There are pros and cons to all. > > For the options explored then most of the change is specification with the starting point for all was to re-specify async close of FileChannel to align with long standing behavior. If you look at how async close of a FileChannel works on Windows or with virtual threads then you'll see what I mean. There is also the issue that the lock methods specify that FileLockInterruptionException is thrown without closing the channel but the implementation differs. > > My view on this has changed from when I first looked at it a few years ago. Right now, I think #4 may be the best option. The spec changes that I have from that option are: > > - Changes to FileChannel to specify that it deviates from InterruptibleChannel to allow for close to wait for all outstanding I/O operations to end. This allows for long standing behavior. As part of this, FileChannel overrides the specification of implCloseChannel. > > - Changes to FileChannel to specify that it deviates from InterruptibleChannel so that interrupt does not close the channel but allows for an implementation specific means for interrupt to close the channel. > > - Changes to the specification of Thread.interrupt, ClosedByInterruptException, all methods in FileChannel that throw ClosedByInterruptException, FileLockInterruptionException and the FileChannel lock methods. > > The main thing with this option is identifying any common usages where ClosedByInterruptException may be depended on, e.g. trying to shutdown something when threads are blocked indefinitely in file I/O operations. Such usages are likely file access over a network or maybe transfer to/from a channel backed by a network connection. Any help to identify such usages would be helpful to assess the compatibility impact of the options that change default behavior. > > Your exploration is #2. The spec changes that I did when prototyping this option including changes to all the above spec areas, and also everywhere that opens a file, e.g. AsynchronousFileChannel. It is clearly the least risky from a compatibility perspective as it doesn't change default behavior but also the least satisfactory option too. > > -Alan > From syan at openjdk.org Fri Jan 17 07:23:35 2025 From: syan at openjdk.org (SendaoYan) Date: Fri, 17 Jan 2025 07:23:35 GMT Subject: [jdk24] RFR: 8346972: Test java/nio/channels/FileChannel/LoopingTruncate.java fails sometimes with IOException: There is not enough space on the disk In-Reply-To: References: Message-ID: On Tue, 14 Jan 2025 01:41:26 GMT, SendaoYan wrote: > Hi all, > > This pull request contains a backport of commit [4e0ffda5](https://github.com/openjdk/jdk/commit/4e0ffda5b1d82449d2d6f639be7641b69d6cb520) from the [openjdk/jdk](https://git.openjdk.org/jdk) repository to jdk24. This backport make test more rubustness, test-fix only, no risk. > > Thanks! GHA report fails 'wget exited with exit code 4' at 'Build JTReg' stage, seems like environmental issue or infrastructure issue, it's unrelated to this backport PR. ------------- PR Comment: https://git.openjdk.org/jdk/pull/23094#issuecomment-2597590605 From djelinski at openjdk.org Fri Jan 17 12:28:39 2025 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Fri, 17 Jan 2025 12:28:39 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes In-Reply-To: References: Message-ID: On Tue, 26 Nov 2024 13:04:41 GMT, Eirik Bj?rsn?s wrote: > Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. > > The classes were identified using an automated analysis. See CSR for details. > > Besides simply adding the `final` access modifier, the PR: > > * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. > * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` > * Updates the copyright year of affected source files LGTM, the class changes are pretty trivial. I suppose the CheckCSMs test could be cleaned up a little (will we ever need KNOWN_NON_FINAL_CSMS again?), but that can be done in a separate PR. ------------- Marked as reviewed by djelinski (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/22389#pullrequestreview-2558892291 From mullan at openjdk.org Fri Jan 17 13:51:45 2025 From: mullan at openjdk.org (Sean Mullan) Date: Fri, 17 Jan 2025 13:51:45 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes In-Reply-To: References: Message-ID: On Tue, 26 Nov 2024 13:04:41 GMT, Eirik Bj?rsn?s wrote: > Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. > > The classes were identified using an automated analysis. See CSR for details. > > Besides simply adding the `final` access modifier, the PR: > > * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. > * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` > * Updates the copyright year of affected source files @bradfordwetmore @wangweij please review the change to `DRBGParameters` and make sure we are ok with it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22389#issuecomment-2598407604 From eirbjo at openjdk.org Fri Jan 17 18:19:56 2025 From: eirbjo at openjdk.org (Eirik =?UTF-8?B?QmrDuHJzbsO4cw==?=) Date: Fri, 17 Jan 2025 18:19:56 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes [v2] In-Reply-To: References: Message-ID: > Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. > > The classes were identified using an automated analysis. See CSR for details. > > Besides simply adding the `final` access modifier, the PR: > > * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. > * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` > * Updates the copyright year of affected source files Eirik Bj?rsn?s has updated the pull request incrementally with one additional commit since the last revision: Update copyright headers for 2025 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/22389/files - new: https://git.openjdk.org/jdk/pull/22389/files/d93dcf8b..12c3e153 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=22389&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22389&range=00-01 Stats: 19 lines in 19 files changed: 0 ins; 0 del; 19 mod Patch: https://git.openjdk.org/jdk/pull/22389.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22389/head:pull/22389 PR: https://git.openjdk.org/jdk/pull/22389 From weijun at openjdk.org Fri Jan 17 20:05:38 2025 From: weijun at openjdk.org (Weijun Wang) Date: Fri, 17 Jan 2025 20:05:38 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes [v2] In-Reply-To: References: Message-ID: <3_SAo71Dt1lxEL32D0GqZyLdm6aQjgk46bCJMIIgO84=.b9a6acd6-ae8c-485b-8145-e09eb594cf1d@github.com> On Fri, 17 Jan 2025 18:19:56 GMT, Eirik Bj?rsn?s wrote: >> Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. >> >> The classes were identified using an automated analysis. See CSR for details. >> >> Besides simply adding the `final` access modifier, the PR: >> >> * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. >> * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` >> * Updates the copyright year of affected source files > > Eirik Bj?rsn?s has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright headers for 2025 Marked as reviewed by weijun (Reviewer). The `DRBGParameters` change looks good. It?s a utility class with static methods and a private constructor, so any attempt to extend it should result in a compile-time error. ------------- PR Review: https://git.openjdk.org/jdk/pull/22389#pullrequestreview-2559870005 PR Comment: https://git.openjdk.org/jdk/pull/22389#issuecomment-2599101417 From shade at openjdk.org Mon Jan 20 12:59:09 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Mon, 20 Jan 2025 12:59:09 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner Message-ID: DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. We can migrate DBBs to use `java.lang.ref.Cleaner`. There are two pecularities during this rewrite. First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing are in DBB cleaning action, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 Additional testing: - [x] Linux x86_64 server fastdebug, `java/nio java/io` - [x] Linux AArch64 server fastdebug, `java/nio java/io` - [ ] Linux x86_64 server fastdebug, `all` - [ ] Linux AArch64 server fastdebug, `all` ------------- Commit messages: - Cleaner does not accept null cleanables - Fix Changes: https://git.openjdk.org/jdk/pull/22165/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8344332 Stats: 270 lines in 8 files changed: 15 ins; 234 del; 21 mod Patch: https://git.openjdk.org/jdk/pull/22165.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22165/head:pull/22165 PR: https://git.openjdk.org/jdk/pull/22165 From vklang at openjdk.org Mon Jan 20 13:27:37 2025 From: vklang at openjdk.org (Viktor Klang) Date: Mon, 20 Jan 2025 13:27:37 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner In-Reply-To: References: Message-ID: <9-qwIxCcUu35IeZ28A63BEAkOSbDkcuDX9RS-C_LVKQ=.1e6632a9-25b2-4cd1-a104-5deab2d1f120@github.com> On Fri, 15 Nov 2024 20:54:56 GMT, Aleksey Shipilev wrote: > DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. > > We can migrate DBBs to use `java.lang.ref.Cleaner`. > > There are two pecularities during this rewrite. > > First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. > > Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing are in DBB cleaning action, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. > > Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 > > Additional testing: > - [x] Linux x86_64 server fastdebug, `java/nio java/io` > - [x] Linux AArch64 server fastdebug, `java/nio java/io` > - [ ] Linux x86_64 server fastdebug, `all` > - [ ] Linux AArch64 server fastdebug, `all` src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template line 88: > 86: // Old jdk.internal.ref.Cleaner behavior: when it fails, VM exits. > 87: if (System.err != null) { > 88: new Error("Cleaner terminated abnormally", x).printStackTrace(); @shipilev If this line throws, should the exit(1) be skipped? ? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1922401820 From shade at openjdk.org Mon Jan 20 13:55:43 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Mon, 20 Jan 2025 13:55:43 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner In-Reply-To: <9-qwIxCcUu35IeZ28A63BEAkOSbDkcuDX9RS-C_LVKQ=.1e6632a9-25b2-4cd1-a104-5deab2d1f120@github.com> References: <9-qwIxCcUu35IeZ28A63BEAkOSbDkcuDX9RS-C_LVKQ=.1e6632a9-25b2-4cd1-a104-5deab2d1f120@github.com> Message-ID: On Mon, 20 Jan 2025 13:25:26 GMT, Viktor Klang wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template line 88: > >> 86: // Old jdk.internal.ref.Cleaner behavior: when it fails, VM exits. >> 87: if (System.err != null) { >> 88: new Error("Cleaner terminated abnormally", x).printStackTrace(); > > @shipilev If this line throws, should the exit(1) be skipped? ? This is verbatim from the old code in removed `Cleaner.java`. This line does not throw, it is there for printing the stack trace, I think :) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1922439813 From shade at openjdk.org Mon Jan 20 16:43:08 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Mon, 20 Jan 2025 16:43:08 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v2] In-Reply-To: References: Message-ID: <00hUbsbC0k4Q12lEVRbIbflJflQPlwH51VHIztwhdxc=.900a197b-3448-40b2-a25b-b124449b42ba@github.com> > DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. > > We can migrate DBBs to use `java.lang.ref.Cleaner`. > > There are two pecularities during this rewrite. > > First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. > > Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. > > Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 > > Additional testing: > - [x] Linux x86_64 server fastdebug, `java/nio java/io` > - [x] Linux AArch64 server fastdebug, `java/nio java/io` > - [ ] Linux x86_64 server fastdebug, `all` > - [ ] Linux AArch64 server fastdebug, `all` Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Lazy initialization to avoid bootstrap circularities ------------- Changes: - all: https://git.openjdk.org/jdk/pull/22165/files - new: https://git.openjdk.org/jdk/pull/22165/files/dc021ce4..0d32ea15 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=00-01 Stats: 15 lines in 1 file changed: 12 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/22165.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22165/head:pull/22165 PR: https://git.openjdk.org/jdk/pull/22165 From shade at openjdk.org Mon Jan 20 16:43:08 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Mon, 20 Jan 2025 16:43:08 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner In-Reply-To: References: Message-ID: On Fri, 15 Nov 2024 20:54:56 GMT, Aleksey Shipilev wrote: > DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. > > We can migrate DBBs to use `java.lang.ref.Cleaner`. > > There are two pecularities during this rewrite. > > First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. > > Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. > > Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 > > Additional testing: > - [x] Linux x86_64 server fastdebug, `java/nio java/io` > - [x] Linux AArch64 server fastdebug, `java/nio java/io` > - [ ] Linux x86_64 server fastdebug, `all` > - [ ] Linux AArch64 server fastdebug, `all` FTR, wider testing shows two tests that install custom system classloader fail with the following circularity: TEST: runtime/cds/appcds/SpecifySysLoaderProp.java TEST: java/security/SignedJar/SignedJarWithCustomClassLoader.java Caused by: java.lang.IllegalStateException: getSystemClassLoader cannot be called during the system class loader instantiation at java.lang.ClassLoader.getSystemClassLoader(java.base at 25-internal/ClassLoader.java:1849) at jdk.internal.misc.InnocuousThread.newThread(java.base at 25-internal/InnocuousThread.java:68) at jdk.internal.ref.CleanerImpl$InnocuousThreadFactory.newThread(java.base at 25-internal/CleanerImpl.java:208) at jdk.internal.ref.CleanerImpl.start(java.base at 25-internal/CleanerImpl.java:110) at java.lang.ref.Cleaner.create(java.base at 25-internal/Cleaner.java:174) at java.nio.DirectByteBuffer.(java.base at 25-internal/DirectByteBuffer.java:95) at jdk.internal.perf.Perf.createLong(java.base at 25-internal/Native Method) at jdk.internal.perf.PerfCounter.(java.base at 25-internal/PerfCounter.java:63) at jdk.internal.perf.PerfCounter.newPerfCounter(java.base at 25-internal/PerfCounter.java:69) at jdk.internal.perf.PerfCounter$CoreCounters.(java.base at 25-internal/PerfCounter.java:126) at jdk.internal.perf.PerfCounter.getZipFileOpenTime(java.base at 25-internal/PerfCounter.java:176) at java.util.zip.ZipFile.(java.base at 25-internal/ZipFile.java:203) at java.util.zip.ZipFile.(java.base at 25-internal/ZipFile.java:148) at java.util.jar.JarFile.(java.base at 25-internal/JarFile.java:333) at jdk.internal.loader.URLClassPath$JarLoader.getJarFile(java.base at 25-internal/URLClassPath.java:682) at jdk.internal.loader.URLClassPath$JarLoader.ensureOpen(java.base at 25-internal/URLClassPath.java:653) at jdk.internal.loader.URLClassPath$JarLoader.(java.base at 25-internal/URLClassPath.java:622) at jdk.internal.loader.URLClassPath.getLoader(java.base at 25-internal/URLClassPath.java:465) at jdk.internal.loader.URLClassPath.getLoader(java.base at 25-internal/URLClassPath.java:409) at jdk.internal.loader.URLClassPath.getResource(java.base at 25-internal/URLClassPath.java:331) at jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(java.base at 25-internal/BuiltinClassLoader.java:688) at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base at 25-internal/BuiltinClassLoader.java:620) at jdk.internal.loader.BuiltinClassLoader.loadClass(java.base at 25-internal/BuiltinClassLoader.java:578) at java.lang.ClassLoader.loadClass(java.base at 25-internal/ClassLoader.java:490) at java.lang.Class.forName0(java.base at 25-internal/Native Method) at java.lang.Class.forName(java.base at 25-internal/Class.java:543) at java.lang.ClassLoader.initSystemClassLoader(java.base at 25-internal/ClassLoader.java:1887) at java.lang.System.initPhase3(java.base at 25-internal/System.java:1964) This can be avoided if we use the lazy holder pattern to delay `Cleaner` initialization until the actual use. Done in new commit. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2602865622 From shade at openjdk.org Mon Jan 20 16:48:49 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Mon, 20 Jan 2025 16:48:49 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v3] In-Reply-To: References: Message-ID: > DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. > > We can migrate DBBs to use `java.lang.ref.Cleaner`. > > There are two pecularities during this rewrite. > > First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. > > Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. > > Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 > > Additional testing: > - [x] Linux x86_64 server fastdebug, `java/nio java/io` > - [x] Linux AArch64 server fastdebug, `java/nio java/io` > - [ ] Linux x86_64 server fastdebug, `all` > - [ ] Linux AArch64 server fastdebug, `all` Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Visibility and whitespace ------------- Changes: - all: https://git.openjdk.org/jdk/pull/22165/files - new: https://git.openjdk.org/jdk/pull/22165/files/0d32ea15..b115ec46 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=01-02 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/22165.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22165/head:pull/22165 PR: https://git.openjdk.org/jdk/pull/22165 From alanb at openjdk.org Mon Jan 20 17:39:37 2025 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 20 Jan 2025 17:39:37 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v3] In-Reply-To: References: Message-ID: On Mon, 20 Jan 2025 16:48:49 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Visibility and whitespace No objection to move this to use jl.ref.Cleaner. Note that the long standing recommendation has always been to cache/reuse direct buffers rather than discard like the reproducer does. Now that the FFM API is permanent then we should redirect developers looking for deterministic release to use the new API and they can get a BB view if needed. I assume you'll update the copyright header before this change is integrated. src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template line 86: > 84: Bits.unreserveMemory(size, capacity); > 85: } catch (Throwable x) { > 86: // Old jdk.internal.ref.Cleaner behavior: when it fails, VM exits. I'd prefer not reference a class that no longer exists, instead just say that it preserves long standing behavior exit the VM. In the case of DBB, I can't think what exceptions/errors are possible here. src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template line 100: > 98: private static final Cleaner CLEANER = Cleaner.create(); > 99: > 100: public static Cleanable register(Object buffer, Runnable action) { Why is this public? ------------- PR Review: https://git.openjdk.org/jdk/pull/22165#pullrequestreview-2562893777 PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1922695881 PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1922696432 From uschindler at openjdk.org Mon Jan 20 18:06:35 2025 From: uschindler at openjdk.org (Uwe Schindler) Date: Mon, 20 Jan 2025 18:06:35 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v3] In-Reply-To: References: Message-ID: On Mon, 20 Jan 2025 16:48:49 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Visibility and whitespace Marked as reviewed by uschindler (Author). This looks fine for me. Older Lucene versions won't break, as they use `sun.misc.Unsafe#invokeCleaner()` which behaves as before. Code that deep reflected and made `DirectByteBuffer#cleaner()` accessible was broken for long time anyways (since Java 9), so the new names and class types should not affect uptodate code. I'd remove the `catch (Throwable)` => exit VM code completely. The called methods have no checked exceptions. If the deallocation fails for some (internal) reason it its already broken and a crush will come sooner or later anyways. ------------- PR Review: https://git.openjdk.org/jdk/pull/22165#pullrequestreview-2562957601 PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2603003680 From egahlin at openjdk.org Mon Jan 20 18:41:39 2025 From: egahlin at openjdk.org (Erik Gahlin) Date: Mon, 20 Jan 2025 18:41:39 GMT Subject: RFR: 8310996: Add JFR event for connect operations [v5] In-Reply-To: References: <9WgigdK6VDa5UjTuNSUw1A_cn0PUzhZxdl3REdSzZz4=.c3307452-0fd0-4392-89d3-6c050c587f3d@github.com> <0cyBOkicHRv3O8XkoAzzpNfC6auxjkPVtsrvRybLa44=.4b07f7e0-65a0-4e87-8758-d03f015e3661@github.com> <3bizpeZPDx1GkDuHtNSq1229cleBGYHZW6s4Cx8hGwQ=.2a6a2dc1-462f-41b8-86af-adcc36f81d8b@github.com> <9j67W3RcuigGBre_a2BfGeMsRlWvEFcoUg3iRmglPVQ=.4a0d1d85-eca9-4669-88f1-a8ed1159d3cb@github.com> Message-ID: On Thu, 19 Dec 2024 19:16:55 GMT, Erik Gahlin wrote: >> I'm not sure if one or two events are most suitable. If possible, I would like to discuss it with Markus to get some more input. He will back in January. > > Regarding one or two events. I'm fine with integrating as-is and then revisit it before FC, if needed. I talked to Markus, and we think two events would provide the greatest flexibility for future settings and events. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/21528#discussion_r1922757394 From shade at openjdk.org Mon Jan 20 19:12:22 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Mon, 20 Jan 2025 19:12:22 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v4] In-Reply-To: References: Message-ID: > DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. > > We can migrate DBBs to use `java.lang.ref.Cleaner`. > > There are two pecularities during this rewrite. > > First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. > > Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. > > Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 > > Additional testing: > - [x] Linux x86_64 server fastdebug, `java/nio java/io` > - [x] Linux AArch64 server fastdebug, `java/nio java/io` > - [ ] Linux x86_64 server fastdebug, `all` > - [ ] Linux AArch64 server fastdebug, `all` Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Review feedback and benchmarks ------------- Changes: - all: https://git.openjdk.org/jdk/pull/22165/files - new: https://git.openjdk.org/jdk/pull/22165/files/b115ec46..f8ecbd60 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=02-03 Stats: 116 lines in 7 files changed: 108 ins; 0 del; 8 mod Patch: https://git.openjdk.org/jdk/pull/22165.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22165/head:pull/22165 PR: https://git.openjdk.org/jdk/pull/22165 From shade at openjdk.org Mon Jan 20 19:12:23 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Mon, 20 Jan 2025 19:12:23 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v3] In-Reply-To: References: Message-ID: On Mon, 20 Jan 2025 17:37:19 GMT, Alan Bateman wrote: > Note that the long standing recommendation has always been to cache/reuse direct buffers rather than discard like the reproducer does The reproducer is likely overly simplistic. The performance problem we are solving is non-parallelism in GC when processing backing Cleaner lists. Actually, this reminds me that I need to add direct churn/GC tests like we did for [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704) to verify the DBB-like scenarios also improve. I pushed some benchmarks, going to see how they perform now. > src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template line 86: > >> 84: Bits.unreserveMemory(size, capacity); >> 85: } catch (Throwable x) { >> 86: // Old jdk.internal.ref.Cleaner behavior: when it fails, VM exits. > > I'd prefer not reference a class that no longer exists, instead just say that it preserves long standing behavior exit the VM. In the case of DBB, I can't think what exceptions/errors are possible here. Amended the comment. > src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template line 100: > >> 98: private static final Cleaner CLEANER = Cleaner.create(); >> 99: >> 100: public static Cleanable register(Object buffer, Runnable action) { > > Why is this public? It is a public method in a private nested class. This method is used from the enclosing class, so private would be weird? I can drop public, though, let it be package-private. Done in new commit. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2603094715 PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1922775259 PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1922775141 From shade at openjdk.org Tue Jan 21 09:46:58 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Tue, 21 Jan 2025 09:46:58 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v5] In-Reply-To: References: Message-ID: > DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. > > We can migrate DBBs to use `java.lang.ref.Cleaner`. > > There are two pecularities during this rewrite. > > First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. > > Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. > > Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 > > Additional testing: > - [x] Linux x86_64 server fastdebug, `java/nio java/io` > - [x] Linux AArch64 server fastdebug, `java/nio java/io` > - [ ] Linux x86_64 server fastdebug, `all` > - [ ] Linux AArch64 server fastdebug, `all` Aleksey Shipilev has updated the pull request incrementally with two additional commits since the last revision: - Amend benchmarks - Fix tests ------------- Changes: - all: https://git.openjdk.org/jdk/pull/22165/files - new: https://git.openjdk.org/jdk/pull/22165/files/f8ecbd60..d33e9eb8 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=03-04 Stats: 9 lines in 1 file changed: 4 ins; 0 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/22165.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22165/head:pull/22165 PR: https://git.openjdk.org/jdk/pull/22165 From shade at openjdk.org Tue Jan 21 09:46:58 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Tue, 21 Jan 2025 09:46:58 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v4] In-Reply-To: References: Message-ID: On Mon, 20 Jan 2025 19:12:22 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Review feedback and benchmarks Benchmark data on new benchmarks: Benchmark (count) (recipFreq) Mode Cnt Score Error Units # Before DirectByteBufferChurn.test N/A 128 avgt 9 11.852 ? 0.505 ns/op DirectByteBufferChurn.test N/A 256 avgt 9 9.107 ? 0.695 ns/op DirectByteBufferChurn.test N/A 512 avgt 9 7.643 ? 0.480 ns/op DirectByteBufferChurn.test N/A 1024 avgt 9 7.091 ? 0.183 ns/op DirectByteBufferChurn.test N/A 2048 avgt 9 6.734 ? 0.184 ns/op DirectByteBufferGC.test 16384 N/A avgt 9 3.122 ? 0.028 ms/op DirectByteBufferGC.test 65536 N/A avgt 9 6.079 ? 0.111 ms/op DirectByteBufferGC.test 262144 N/A avgt 9 16.892 ? 0.705 ms/op DirectByteBufferGC.test 1048576 N/A avgt 9 156.980 ? 9.673 ms/op DirectByteBufferGC.test 4194304 N/A avgt 9 725.576 ? 49.363 ms/op # After DirectByteBufferChurn.test N/A 128 avgt 9 8.697 ? 0.742 ns/op ; <--- DirectByteBufferChurn.test N/A 256 avgt 9 7.557 ? 0.353 ns/op ; <--- DirectByteBufferChurn.test N/A 512 avgt 9 7.074 ? 0.268 ns/op ; <--- DirectByteBufferChurn.test N/A 1024 avgt 9 6.853 ? 0.272 ns/op DirectByteBufferChurn.test N/A 2048 avgt 9 6.688 ? 0.298 ns/op DirectByteBufferGC.test 16384 N/A avgt 9 3.197 ? 0.231 ms/op DirectByteBufferGC.test 65536 N/A avgt 9 5.442 ? 0.323 ms/op DirectByteBufferGC.test 262144 N/A avgt 9 16.967 ? 0.485 ms/op DirectByteBufferGC.test 1048576 N/A avgt 9 50.613 ? 2.107 ms/op ; <--- DirectByteBufferGC.test 4194304 N/A avgt 9 343.177 ? 20.655 ms/op ; <--- ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2604187764 From alanb at openjdk.org Tue Jan 21 11:54:41 2025 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 21 Jan 2025 11:54:41 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v5] In-Reply-To: References: Message-ID: On Tue, 21 Jan 2025 09:46:58 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with two additional commits since the last revision: > > - Amend benchmarks > - Fix tests Good, I don't have any other comments. ------------- Marked as reviewed by alanb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/22165#pullrequestreview-2564379884 From shade at openjdk.org Tue Jan 21 12:55:31 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Tue, 21 Jan 2025 12:55:31 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v6] In-Reply-To: References: Message-ID: > DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. > > We can migrate DBBs to use `java.lang.ref.Cleaner`. > > There are two pecularities during this rewrite. > > First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. > > Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. > > Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 > > Additional testing: > - [x] Linux x86_64 server fastdebug, `java/nio java/io` > - [x] Linux AArch64 server fastdebug, `java/nio java/io` > - [ ] Linux x86_64 server fastdebug, `all` > - [ ] Linux AArch64 server fastdebug, `all` Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Track Cleaner progress with canaries ------------- Changes: - all: https://git.openjdk.org/jdk/pull/22165/files - new: https://git.openjdk.org/jdk/pull/22165/files/d33e9eb8..25cb6453 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=04-05 Stats: 195 lines in 5 files changed: 84 ins; 99 del; 12 mod Patch: https://git.openjdk.org/jdk/pull/22165.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22165/head:pull/22165 PR: https://git.openjdk.org/jdk/pull/22165 From shade at openjdk.org Tue Jan 21 12:55:32 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Tue, 21 Jan 2025 12:55:32 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v5] In-Reply-To: References: Message-ID: On Tue, 21 Jan 2025 09:46:58 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with two additional commits since the last revision: > > - Amend benchmarks > - Fix tests Unfortunately, I am seeing intermittent OOME failures in GHA on `DirectBufferAllocTest`. I managed to reproduce some of these locally. This likely hits us hard: > With the patched version, we now rely on normal Cleaner thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. So I think we need a feedback loop from a relevant Cleaner back to allocation code. I pushed a hopefully reliable version of it now. It survives the short `DirectBufferAllocTest` stress tests I throw at it, now running it for longer. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2604647588 From alanb at openjdk.org Tue Jan 21 13:48:36 2025 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 21 Jan 2025 13:48:36 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v6] In-Reply-To: References: Message-ID: <6I92Ypsg5cTFrNQnnsmXs5gXzpLmIWAE90lT6pW_ebk=.571f971f-bfbe-49d6-8647-21fa781420ed@github.com> On Tue, 21 Jan 2025 12:55:31 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Track Cleaner progress with canaries Okay, there is much more significant change now and will require time to review very closely. ------------- Changes requested by alanb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/22165#pullrequestreview-2564681328 From shade at openjdk.org Tue Jan 21 14:33:43 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Tue, 21 Jan 2025 14:33:43 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v6] In-Reply-To: <6I92Ypsg5cTFrNQnnsmXs5gXzpLmIWAE90lT6pW_ebk=.571f971f-bfbe-49d6-8647-21fa781420ed@github.com> References: <6I92Ypsg5cTFrNQnnsmXs5gXzpLmIWAE90lT6pW_ebk=.571f971f-bfbe-49d6-8647-21fa781420ed@github.com> Message-ID: <1deAxr-IkpzltLxEyXGRQ551UDQDsj59dMJCGdt_XKA=.3fe05042-e6db-49d3-9e6c-aa90f949c173@github.com> On Tue, 21 Jan 2025 13:45:50 GMT, Alan Bateman wrote: > Okay, there is much more significant change now and will require time to review very closely. Sure. I was hoping we could avoid changing the feedback path in allocation code, but testing disagreed :) I remember when we built that feedback path back in JDK 8 days; it is clumsy and hacky and arguably a tech debt, but it worked well on stress tests. Today, migrating DDB to standard Cleaner forces us to figure out that particular part of technical debt: the feedback path from Reference handler to DDB allocation code needs to be replaced with the feedback path from Cleaner. I think current version is less messy than what we had before. No stress test failures so far... ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2604896009 From shade at openjdk.org Wed Jan 22 08:50:38 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 22 Jan 2025 08:50:38 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v6] In-Reply-To: <1deAxr-IkpzltLxEyXGRQ551UDQDsj59dMJCGdt_XKA=.3fe05042-e6db-49d3-9e6c-aa90f949c173@github.com> References: <6I92Ypsg5cTFrNQnnsmXs5gXzpLmIWAE90lT6pW_ebk=.571f971f-bfbe-49d6-8647-21fa781420ed@github.com> <1deAxr-IkpzltLxEyXGRQ551UDQDsj59dMJCGdt_XKA=.3fe05042-e6db-49d3-9e6c-aa90f949c173@github.com> Message-ID: <1lr5io7nX-jRr3wfr3XThOzB4qaVHFSZQN62v0R73pw=.33448629-a662-41ff-82f2-27920f4e7ee4@github.com> On Tue, 21 Jan 2025 14:31:33 GMT, Aleksey Shipilev wrote: > No stress test failures so far... I ran a large matrix of `DirectBufferAllocTest` configs on my 5950X overnight, and it did not fail once. Several GHA re-runs also came back clean. This gives me confidence current version works well. #!/bin/bash set -euxo pipefail J=~/trunks/jdk/build/linux-x86_64-server-release/images/jdk/bin/java JOPTS="-Xmx128m -Xlog:gc" for T in 1 4 16 64 256 1024; do for M in 1 2 4 8 16 32 64 128 256; do $J $JOPTS -XX:MaxDirectMemorySize=${M}m DirectBufferAllocTest.java -r 800 -t $T done done ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2606622780 From shade at openjdk.org Wed Jan 22 15:58:14 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 22 Jan 2025 15:58:14 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v7] In-Reply-To: References: Message-ID: > DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. > > We can migrate DBBs to use `java.lang.ref.Cleaner`. > > There are two pecularities during this rewrite. > > First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. > > Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. > > Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 > > Additional testing: > - [x] Linux x86_64 server fastdebug, `java/nio java/io` > - [x] Linux AArch64 server fastdebug, `java/nio java/io` > - [ ] Linux x86_64 server fastdebug, `all` > - [ ] Linux AArch64 server fastdebug, `all` Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Remove vestigial reference to waitForReferenceProcessing in tests ------------- Changes: - all: https://git.openjdk.org/jdk/pull/22165/files - new: https://git.openjdk.org/jdk/pull/22165/files/25cb6453..2c2c4df2 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=05-06 Stats: 26 lines in 1 file changed: 0 ins; 26 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/22165.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22165/head:pull/22165 PR: https://git.openjdk.org/jdk/pull/22165 From shade at openjdk.org Wed Jan 22 17:16:48 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 22 Jan 2025 17:16:48 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v7] In-Reply-To: References: Message-ID: On Wed, 22 Jan 2025 15:58:14 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Remove vestigial reference to waitForReferenceProcessing in tests Some test code removals should disappear once we merge https://github.com/openjdk/jdk/pull/23239. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2607813845 From alanb at openjdk.org Wed Jan 22 17:28:52 2025 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 22 Jan 2025 17:28:52 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v7] In-Reply-To: References: Message-ID: On Wed, 22 Jan 2025 15:58:14 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Remove vestigial reference to waitForReferenceProcessing in tests I've done a pass over the latest version and I think it's okay but want to do another pass on tomorrow, only because this is change code in a very critical area. src/java.base/share/classes/java/nio/Bits.java line 160: > 158: } catch (InterruptedException e) { > 159: interrupted = true; > 160: break; I'm not sure that bailing out to throw OOME when interrupted is the right thing to do here. If ByteBuffer.allocateDIrect is called with the interrupt status set, or someone interrupts a thread while allocating on this slow path, then it would be surprising to have it throw OOME when it might have succeeded (with the interrupt status set). src/java.base/share/classes/java/nio/BufferCleaner.java line 48: > 46: } > 47: > 48: public static class Canary implements Runnable { Confusing for Canary to be public, it's not accessible outside of this package. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2607840604 PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1925699621 PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1925700380 From shade at openjdk.org Wed Jan 22 17:50:52 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 22 Jan 2025 17:50:52 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v8] In-Reply-To: References: Message-ID: <45N5sfuYxIBGbY8f0IUwR_L84ABni1q3dXobFdiH2dI=.838e8a67-9d2f-4a8f-9eec-e143f4a97fe2@github.com> > DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. > > We can migrate DBBs to use `java.lang.ref.Cleaner`. > > There are two pecularities during this rewrite. > > First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. > > Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. > > Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 > > Additional testing: > - [x] Linux x86_64 server fastdebug, `java/nio java/io` > - [x] Linux AArch64 server fastdebug, `java/nio java/io` > - [ ] Linux x86_64 server fastdebug, `all` > - [ ] Linux AArch64 server fastdebug, `all` Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Alan's review ------------- Changes: - all: https://git.openjdk.org/jdk/pull/22165/files - new: https://git.openjdk.org/jdk/pull/22165/files/2c2c4df2..1b8b3ff1 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=06-07 Stats: 2 lines in 2 files changed: 0 ins; 1 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/22165.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22165/head:pull/22165 PR: https://git.openjdk.org/jdk/pull/22165 From shade at openjdk.org Wed Jan 22 17:50:54 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 22 Jan 2025 17:50:54 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v7] In-Reply-To: References: Message-ID: On Wed, 22 Jan 2025 17:23:53 GMT, Alan Bateman wrote: >> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove vestigial reference to waitForReferenceProcessing in tests > > src/java.base/share/classes/java/nio/Bits.java line 160: > >> 158: } catch (InterruptedException e) { >> 159: interrupted = true; >> 160: break; > > I'm not sure that bailing out to throw OOME when interrupted is the right thing to do here. If ByteBuffer.allocateDIrect is called with the interrupt status set, or someone interrupts a thread while allocating on this slow path, then it would be surprising to have it throw OOME when it might have succeeded (with the interrupt status set). Agreed. I thought I maintained the same interrupt behavior like before, but now I see old code does not bail on interrupt, it just restores the interrupt status on both failing and passing paths. Dropped this `break` in new commit. > src/java.base/share/classes/java/nio/BufferCleaner.java line 48: > >> 46: } >> 47: >> 48: public static class Canary implements Runnable { > > Confusing for Canary to be public, it's not accessible outside of this package. Right. Can be package-private. Done in new commit. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1925728413 PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1925728718 From bchristi at openjdk.org Wed Jan 22 20:38:49 2025 From: bchristi at openjdk.org (Brent Christian) Date: Wed, 22 Jan 2025 20:38:49 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v8] In-Reply-To: <45N5sfuYxIBGbY8f0IUwR_L84ABni1q3dXobFdiH2dI=.838e8a67-9d2f-4a8f-9eec-e143f4a97fe2@github.com> References: <45N5sfuYxIBGbY8f0IUwR_L84ABni1q3dXobFdiH2dI=.838e8a67-9d2f-4a8f-9eec-e143f4a97fe2@github.com> Message-ID: On Wed, 22 Jan 2025 17:50:52 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Alan's review src/java.base/share/classes/java/lang/ref/Reference.java line 286: > 284: { > 285: synchronized (processPendingLock) { > 286: if (processPendingActive || hasReferencePendingList()) { It looks like `hasReferencePendingList()` is no longer used. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1925928978 From bchristi at openjdk.org Wed Jan 22 23:46:49 2025 From: bchristi at openjdk.org (Brent Christian) Date: Wed, 22 Jan 2025 23:46:49 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v8] In-Reply-To: <45N5sfuYxIBGbY8f0IUwR_L84ABni1q3dXobFdiH2dI=.838e8a67-9d2f-4a8f-9eec-e143f4a97fe2@github.com> References: <45N5sfuYxIBGbY8f0IUwR_L84ABni1q3dXobFdiH2dI=.838e8a67-9d2f-4a8f-9eec-e143f4a97fe2@github.com> Message-ID: <9-yWCMfc8D4YFZFryYjeYJTV97f-wz9mon-7ZjRct_0=.1bfe4f67-28b1-48f7-8454-4d95a358750d@github.com> On Wed, 22 Jan 2025 17:50:52 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Alan's review src/java.base/share/classes/java/nio/BufferCleaner.java line 31: > 29: import java.lang.ref.Cleaner.Cleanable; > 30: > 31: class BufferCleaner { It looks like `BufferCleaner` is not meant to be instantiated. A private constructor (that throws?) would make that clear. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1926125201 From shade at openjdk.org Thu Jan 23 08:51:38 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 23 Jan 2025 08:51:38 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: References: Message-ID: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> > DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. > > We can migrate DBBs to use `java.lang.ref.Cleaner`. > > There are two pecularities during this rewrite. > > First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. > > Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. > > Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 > > Additional testing: > - [x] Linux x86_64 server fastdebug, `java/nio java/io` > - [x] Linux AArch64 server fastdebug, `java/nio java/io` > - [ ] Linux x86_64 server fastdebug, `all` > - [ ] Linux AArch64 server fastdebug, `all` Aleksey Shipilev has updated the pull request incrementally with three additional commits since the last revision: - No instantiation for BufferCleaner, javadocs - Remove hasReferencePendingList - Revert test exclusion, moved to JDK-8348301 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/22165/files - new: https://git.openjdk.org/jdk/pull/22165/files/1b8b3ff1..2831bc84 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=07-08 Stats: 71 lines in 6 files changed: 49 ins; 19 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/22165.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22165/head:pull/22165 PR: https://git.openjdk.org/jdk/pull/22165 From shade at openjdk.org Thu Jan 23 08:51:38 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 23 Jan 2025 08:51:38 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: References: <45N5sfuYxIBGbY8f0IUwR_L84ABni1q3dXobFdiH2dI=.838e8a67-9d2f-4a8f-9eec-e143f4a97fe2@github.com> Message-ID: On Wed, 22 Jan 2025 20:35:49 GMT, Brent Christian wrote: >> Aleksey Shipilev has updated the pull request incrementally with three additional commits since the last revision: >> >> - No instantiation for BufferCleaner, javadocs >> - Remove hasReferencePendingList >> - Revert test exclusion, moved to JDK-8348301 > > src/java.base/share/classes/java/lang/ref/Reference.java line 286: > >> 284: { >> 285: synchronized (processPendingLock) { >> 286: if (processPendingActive || hasReferencePendingList()) { > > It looks like `hasReferencePendingList()` is no longer used. Right, we can clean this up right here too. Done in new commit. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1926586148 From shade at openjdk.org Thu Jan 23 08:51:40 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 23 Jan 2025 08:51:40 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v8] In-Reply-To: <9-yWCMfc8D4YFZFryYjeYJTV97f-wz9mon-7ZjRct_0=.1bfe4f67-28b1-48f7-8454-4d95a358750d@github.com> References: <45N5sfuYxIBGbY8f0IUwR_L84ABni1q3dXobFdiH2dI=.838e8a67-9d2f-4a8f-9eec-e143f4a97fe2@github.com> <9-yWCMfc8D4YFZFryYjeYJTV97f-wz9mon-7ZjRct_0=.1bfe4f67-28b1-48f7-8454-4d95a358750d@github.com> Message-ID: On Wed, 22 Jan 2025 23:44:02 GMT, Brent Christian wrote: >> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: >> >> Alan's review > > src/java.base/share/classes/java/nio/BufferCleaner.java line 31: > >> 29: import java.lang.ref.Cleaner.Cleanable; >> 30: >> 31: class BufferCleaner { > > It looks like `BufferCleaner` is not meant to be instantiated. A private constructor (that throws?) would make that clear. Correct, added some comments too. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1926586462 From alanb at openjdk.org Thu Jan 23 17:25:54 2025 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 23 Jan 2025 17:25:54 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> References: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> Message-ID: On Thu, 23 Jan 2025 08:51:38 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with three additional commits since the last revision: > > - No instantiation for BufferCleaner, javadocs > - Remove hasReferencePendingList > - Revert test exclusion, moved to JDK-8348301 I did another pass over the changes today, including the update that removes JVM_HasReferencePendingList. I'm happy with the changes. ------------- Marked as reviewed by alanb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/22165#pullrequestreview-2570534886 From bchristi at openjdk.org Fri Jan 24 00:06:50 2025 From: bchristi at openjdk.org (Brent Christian) Date: Fri, 24 Jan 2025 00:06:50 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> References: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> Message-ID: On Thu, 23 Jan 2025 08:51:38 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with three additional commits since the last revision: > > - No instantiation for BufferCleaner, javadocs > - Remove hasReferencePendingList > - Revert test exclusion, moved to JDK-8348301 The reference processing and Cleaner stuff all looks fine to me. ------------- Marked as reviewed by bchristi (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/22165#pullrequestreview-2571367180 From bchristi at openjdk.org Fri Jan 24 00:11:49 2025 From: bchristi at openjdk.org (Brent Christian) Date: Fri, 24 Jan 2025 00:11:49 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> References: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> Message-ID: On Thu, 23 Jan 2025 08:51:38 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with three additional commits since the last revision: > > - No instantiation for BufferCleaner, javadocs > - Remove hasReferencePendingList > - Revert test exclusion, moved to JDK-8348301 src/java.base/share/classes/java/nio/Bits.java line 151: > 149: // Trigger GC to perform reference processing and then cleaning. > 150: canary = BufferCleaner.newCanary(); > 151: System.gc(); So the Canary functions only to avoid calling `System.gc()` again if we've not yet seen progress from the previous call to `System.gc()`. Is that right? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1927908860 From shade at openjdk.org Fri Jan 24 07:09:51 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Fri, 24 Jan 2025 07:09:51 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: References: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> Message-ID: <1b5VRpKPgM2EGuxUuIfypPqdaAQCUiPYSb9es1R7jVE=.b96b6d9f-4165-4014-9325-92dc14a8016d@github.com> On Fri, 24 Jan 2025 00:09:00 GMT, Brent Christian wrote: >> Aleksey Shipilev has updated the pull request incrementally with three additional commits since the last revision: >> >> - No instantiation for BufferCleaner, javadocs >> - Remove hasReferencePendingList >> - Revert test exclusion, moved to JDK-8348301 > > src/java.base/share/classes/java/nio/Bits.java line 151: > >> 149: // Trigger GC to perform reference processing and then cleaning. >> 150: canary = BufferCleaner.newCanary(); >> 151: System.gc(); > > So the Canary functions only to avoid calling `System.gc()` again if we've not yet seen progress from the previous call to `System.gc()`. Is that right? Yes. There is no point in flooding the JVM with GC requests if Cleaner have not responded yet. It is, by nature, similar to the waiting for pending reference processing hack we had before. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1928186438 From shade at openjdk.org Fri Jan 24 11:32:56 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Fri, 24 Jan 2025 11:32:56 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> References: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> Message-ID: On Thu, 23 Jan 2025 08:51:38 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [ ] Linux x86_64 server fastdebug, `all` >> - [ ] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with three additional commits since the last revision: > > - No instantiation for BufferCleaner, javadocs > - Remove hasReferencePendingList > - Revert test exclusion, moved to JDK-8348301 Testing is still clean on my side. I would appreciate if folks can run this through their CIs. I am planning to integrate this on Monday, as long as we discover no new issues. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2612301921 From eirbjo at openjdk.org Fri Jan 24 19:31:55 2025 From: eirbjo at openjdk.org (Eirik =?UTF-8?B?QmrDuHJzbsO4cw==?=) Date: Fri, 24 Jan 2025 19:31:55 GMT Subject: RFR: 8344943: Mark not subclassable classes final in java.base exported classes [v2] In-Reply-To: References: Message-ID: On Fri, 17 Jan 2025 18:19:56 GMT, Eirik Bj?rsn?s wrote: >> Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. >> >> The classes were identified using an automated analysis. See CSR for details. >> >> Besides simply adding the `final` access modifier, the PR: >> >> * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. >> * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` >> * Updates the copyright year of affected source files > > Eirik Bj?rsn?s has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright headers for 2025 Thanks everyone for your patient reviews in this PR. I'll integrate this now. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22389#issuecomment-2613236074 From eirbjo at openjdk.org Fri Jan 24 19:31:56 2025 From: eirbjo at openjdk.org (Eirik =?UTF-8?B?QmrDuHJzbsO4cw==?=) Date: Fri, 24 Jan 2025 19:31:56 GMT Subject: Integrated: 8344943: Mark not subclassable classes final in java.base exported classes In-Reply-To: References: Message-ID: On Tue, 26 Nov 2024 13:04:41 GMT, Eirik Bj?rsn?s wrote: > Please review this PR which adds the `final` modifier to non-subclassable classes in `java.base`. > > The classes were identified using an automated analysis. See CSR for details. > > Besides simply adding the `final` access modifier, the PR: > > * Updates a note in `java.lang.constant.DynamicCallSiteDesc` to not reference subtypes. See CSR for discussion. > * Removes the class `java.lang.Runtime` from the test `test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java` > * Updates the copyright year of affected source files This pull request has now been integrated. Changeset: 8e8f8000 Author: Eirik Bj?rsn?s URL: https://git.openjdk.org/jdk/commit/8e8f800071851f479f9e8b493bc38c0a38165bab Stats: 47 lines in 22 files changed: 0 ins; 3 del; 44 mod 8344943: Mark not subclassable classes final in java.base exported classes Reviewed-by: weijun, liach, djelinski ------------- PR: https://git.openjdk.org/jdk/pull/22389 From bchristi at openjdk.org Fri Jan 24 22:04:48 2025 From: bchristi at openjdk.org (Brent Christian) Date: Fri, 24 Jan 2025 22:04:48 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: References: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> Message-ID: On Fri, 24 Jan 2025 11:30:16 GMT, Aleksey Shipilev wrote: > Testing is still clean on my side. I would appreciate if folks can run this through their CIs. > > I am planning to integrate this on Monday, as long as we discover no new issues. Did the change to `test/jdk/sun/security/provider/FileInputStreamPool/FileInputStreamPoolTest.java` get lost? I'm pretty sure it was updated, but I no longer see that in the changes, and it failed in my automated test run. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2613455704 From shade at openjdk.org Sat Jan 25 07:14:51 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Sat, 25 Jan 2025 07:14:51 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v10] In-Reply-To: References: Message-ID: > DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. > > We can migrate DBBs to use `java.lang.ref.Cleaner`. > > There are two pecularities during this rewrite. > > First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. > > Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. > > Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 > > Additional testing: > - [x] Linux x86_64 server fastdebug, `java/nio java/io` > - [x] Linux AArch64 server fastdebug, `java/nio java/io` > - [x] Linux x86_64 server fastdebug, `all` > - [x] Linux AArch64 server fastdebug, `all` Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 14 additional commits since the last revision: - Merge branch 'master' into JDK-8344332-dbb-cleaner - No instantiation for BufferCleaner, javadocs - Remove hasReferencePendingList - Revert test exclusion, moved to JDK-8348301 - Alan's review - Remove vestigial reference to waitForReferenceProcessing in tests - Track Cleaner progress with canaries - Amend benchmarks - Fix tests - Review feedback and benchmarks - ... and 4 more: https://git.openjdk.org/jdk/compare/c8e9921d...bcfab1ba ------------- Changes: - all: https://git.openjdk.org/jdk/pull/22165/files - new: https://git.openjdk.org/jdk/pull/22165/files/2831bc84..bcfab1ba Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=09 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=08-09 Stats: 14358 lines in 2002 files changed: 5683 ins; 5459 del; 3216 mod Patch: https://git.openjdk.org/jdk/pull/22165.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22165/head:pull/22165 PR: https://git.openjdk.org/jdk/pull/22165 From shade at openjdk.org Sat Jan 25 07:14:51 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Sat, 25 Jan 2025 07:14:51 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: References: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> Message-ID: <-vJ0hBTDWcQy99ecAeITRv6rvV1szyZgnVoKRqpi2V0=.87d6e7a1-f821-4545-a7ce-076a0b96ac49@github.com> On Fri, 24 Jan 2025 22:01:59 GMT, Brent Christian wrote: > Did the change to `test/jdk/sun/security/provider/FileInputStreamPool/FileInputStreamPoolTest.java` get lost? I'm pretty sure it was updated, but I no longer see that in the changes, and it failed in my automated test run. It was a test bug, I fixed it separately: [JDK-8348301](https://bugs.openjdk.org/browse/JDK-8348301). I merged current master to this PR branch now, so it should work well. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2613819873 From alanb at openjdk.org Sun Jan 26 07:33:02 2025 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 26 Jan 2025 07:33:02 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: References: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> Message-ID: On Fri, 24 Jan 2025 22:01:59 GMT, Brent Christian wrote: > Testing is still clean on my side. I would appreciate if folks can run this through their CIs. > > I am planning to integrate this on Monday, as long as we discover no new issues. I ran it through our CI and we seem to have a few tests that aren't in the jdk repo but have a dependency on jdk.internal.ref.Cleaner so they will need to be fixed and/or excluded. Would you mind holding off until Tuesday to give time to get that done? Otherwise I didn't see any issues. I see there is a referenced Graal issue as it looks like they have to do an update to align with the changes here. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2614254369 From duke at openjdk.org Sun Jan 26 15:54:53 2025 From: duke at openjdk.org (Markus KARG) Date: Sun, 26 Jan 2025 15:54:53 GMT Subject: RFR: JDK-8337974 - ChannelInputStream::skip can use splice (linux) [v3] In-Reply-To: References: Message-ID: > # Targets > > The major target of this issue is to reduce execution time of `ChannelInputStream::skip(long)`. In particular, make `skip(n)` run noticable faster than `read(new byte[n])` on pipes and sockets in the optimal case, but don't run noticable slower in the worst case. > > A side target of this issue is to provide unit testing for `ChannelInputStream::skip(long)`. In particular, provide unit testing for files, pipes and sockets. > > An appreciated benefit of this issue is reduced resource consumption (in the sense of CPU cycles, Java Heap, power consumption, CO2 footprint, etc.) of `ChannelInputStream::skip(long)`. Albeit, as it is not a target, this was not acitvely monitored. > > > # Non-Targets > > It is not a target to improve any other methods of the mentioned or any other class. Such changes should go in separate issues. > > It is not a target to add any new *public* APIs. The public API shall be kept unchanged. All changes implied by the current improvement shall be purely *internal* to OpenJDK. > > It is not a target to improve other source types besides pipes and sockets. > > > # Description > > What users expect from `InputStream::skip`, besides comfort, is "at least some" measurable benefit over `InputStream::read`. Otherwise skipping instead of reading makes no sense to users. > > For files, OpenJDK already applies an effective `seek`-based optimization. For pipes and sockets, benefits were neglectible so far, as `skip` more or less was simply an alias for `read`. > > Hence, this issue proposes optimized implementations for `ChannelInputStream::skip` in the pipes and sockets cases. > > > # Implementation > > The abstract idea behind this issue is to prevent transmission of skipped data into the JVM's on-heap memory in the pipes and socket cases. As a Java application obviously is not interested in skipped data, copying it into the Java heap is a waste of both, time and heap, and induces (albeit neglectible) GC stress without any benefit. > > Hence, this pull request changes the implementation of `ChannelInputStream::skip` in the following aspects: > 1. On *all* operating systems, for pipe and socket channels, `skip` is implemented in C. While the data *still is* transferred form the source into the OS kernel and from the OS kernel into the JVM's off-heap memory, it is *not* transferred into the JVM's on-heap memory. > 2. For *Linux* pipes only, `splice` is used with `/dev/null` as the target. Data is neither transferred from the source into the OS kernel, nor from the OS kernel into n... Markus KARG has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: - Socket.getInputStream()'s JavaDocs explain behavior of skip for timeout, non-blocking, interrupt - Removed trailing whitespace - Corrected issue ID: 8337974 - JDK-8337974 Implementing ChannelInputStream::skip using splice on Linux and C-loops elsewhere ------------- Changes: https://git.openjdk.org/jdk/pull/20489/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=20489&range=02 Stats: 1186 lines in 21 files changed: 1179 ins; 0 del; 7 mod Patch: https://git.openjdk.org/jdk/pull/20489.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/20489/head:pull/20489 PR: https://git.openjdk.org/jdk/pull/20489 From duke at openjdk.org Sun Jan 26 15:54:54 2025 From: duke at openjdk.org (Markus KARG) Date: Sun, 26 Jan 2025 15:54:54 GMT Subject: RFR: JDK-8337974 - ChannelInputStream::skip can use splice (linux) [v2] In-Reply-To: References: Message-ID: <0NogVeue-vkm0O90ZsO6Pu5Zgn_X5Ol2xtkN7ZvoCNE=.d6e95706-25c3-4848-ae30-63a5f01bbdf7@github.com> On Sun, 29 Dec 2024 08:43:37 GMT, Markus KARG wrote: >> Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. Incremental views are not available. > > Kindly asking to resume this suspended review. > @mkarg Please do not rebase or force-push to an active PR as it invalidates existing review comments. Note for future reference, the bots always squash all changes into a single commit automatically as part of the integration. See [OpenJDK Developers? Guide](https://openjdk.org/guide/#working-with-pull-requests) for more information. Intentionally rebased open PR to be up-to-date with current master branch. Kindly asking to continue the review of this PR. ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/20489#issuecomment-2614481238 From joe at chubbypenguin.net Sun Jan 26 16:16:23 2025 From: joe at chubbypenguin.net (Joseph Walker) Date: Sun, 26 Jan 2025 11:16:23 -0500 Subject: Support 5 Level Paging and allow mmap to use greater than 128TB of virtual address space on Linux Message-ID: Our use case is a dataset exceeding 200TB on a single server with elasticsearch. Enabling 5 Level Paging support requires passing a hint address greater than 47-bit to mmap. [https://docs.kernel.org/arch/x86/x86_64/5level-paging.html] The patch below works in our environment and is now used in production: diff --git a/src/java.base/unix/native/libnio/ch/UnixFileDispatcherImpl.c b/src/java.base/unix/native/libnio/ch/UnixFileDispatcherImpl.c index a24f0503e55..23b7068ff21 100644 --- a/src/java.base/unix/native/libnio/ch/UnixFileDispatcherImpl.c +++ b/src/java.base/unix/native/libnio/ch/UnixFileDispatcherImpl.c @@ -320,7 +320,7 @@ Java_sun_nio_ch_UnixFileDispatcherImpl_map0(JNIEnv *env, jclass klass, jobject f } mapAddress = mmap64( - 0, /* Let OS decide location */ + (void*)0xFFFFFFFF00000000LL, /* Let OS decide location */ len, /* Number of bytes to map */ protections, /* File permissions */ flags, /* Changes are shared */ Thanks, Joe From shade at openjdk.org Mon Jan 27 09:11:55 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Mon, 27 Jan 2025 09:11:55 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: References: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> Message-ID: <_Ma82xOEruDRHPxNMNEAVwCs1CAOp0D3ozJMVmVrSFc=.50beb18b-1587-49da-98b9-42b7601f3ea8@github.com> On Sun, 26 Jan 2025 07:30:22 GMT, Alan Bateman wrote: > Would you mind holding off until Tuesday to give time to get that done? Sure, there is no rush with this PR. I'll wait. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2615197062 From shade at openjdk.org Wed Jan 29 11:00:48 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Jan 2025 11:00:48 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: <_Ma82xOEruDRHPxNMNEAVwCs1CAOp0D3ozJMVmVrSFc=.50beb18b-1587-49da-98b9-42b7601f3ea8@github.com> References: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> <_Ma82xOEruDRHPxNMNEAVwCs1CAOp0D3ozJMVmVrSFc=.50beb18b-1587-49da-98b9-42b7601f3ea8@github.com> Message-ID: On Mon, 27 Jan 2025 09:08:56 GMT, Aleksey Shipilev wrote: > > Would you mind holding off until Tuesday to give time to get that done? > > Sure, there is no rush with this PR. I'll wait. How is it going, @AlanBateman? Any progress on closed tests? ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2621313239 From bchristi at openjdk.org Wed Jan 29 17:19:50 2025 From: bchristi at openjdk.org (Brent Christian) Date: Wed, 29 Jan 2025 17:19:50 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: References: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> <_Ma82xOEruDRHPxNMNEAVwCs1CAOp0D3ozJMVmVrSFc=.50beb18b-1587-49da-98b9-42b7601f3ea8@github.com> Message-ID: <8dL8MzvZq4DslLmaoXx5rAuq5FPcjopkblqYJ65qmBw=.d6a376cc-1271-498d-9a3d-d4bdf6468e85@github.com> On Wed, 29 Jan 2025 10:58:07 GMT, Aleksey Shipilev wrote: > > > Would you mind holding off until Tuesday to give time to get that done? > > > > > > Sure, there is no rush with this PR. I'll wait. > > How is it going, @AlanBateman? Any progress on closed tests? Nearly done - in review now. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2622281311 From kbarrett at openjdk.org Wed Jan 29 19:32:09 2025 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 29 Jan 2025 19:32:09 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v10] In-Reply-To: References: Message-ID: On Sat, 25 Jan 2025 07:14:51 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [x] Linux x86_64 server fastdebug, `all` >> - [x] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 14 additional commits since the last revision: > > - Merge branch 'master' into JDK-8344332-dbb-cleaner > - No instantiation for BufferCleaner, javadocs > - Remove hasReferencePendingList > - Revert test exclusion, moved to JDK-8348301 > - Alan's review > - Remove vestigial reference to waitForReferenceProcessing in tests > - Track Cleaner progress with canaries > - Amend benchmarks > - Fix tests > - Review feedback and benchmarks > - ... and 4 more: https://git.openjdk.org/jdk/compare/915b951b...bcfab1ba src/java.base/share/classes/java/lang/ref/Reference.java line 282: > 280: // references, or (2) the reference processing thread is > 281: // processing references. Otherwise, returns false immediately. > 282: private static boolean waitForReferenceProcessing() Removing waitForReferenceProcessing is contrary to https://bugs.openjdk.org/browse/JDK-8305186 Reference.waitForReferenceProcessing should be more accessible I've pointed to JDK-8305186 in a number of discussions about problems encountered with various tests. So I'm doubtful about removing it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1934462036 From shade at openjdk.org Wed Jan 29 19:46:01 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Jan 2025 19:46:01 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v10] In-Reply-To: References: Message-ID: On Wed, 29 Jan 2025 19:28:31 GMT, Kim Barrett wrote: >> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 14 additional commits since the last revision: >> >> - Merge branch 'master' into JDK-8344332-dbb-cleaner >> - No instantiation for BufferCleaner, javadocs >> - Remove hasReferencePendingList >> - Revert test exclusion, moved to JDK-8348301 >> - Alan's review >> - Remove vestigial reference to waitForReferenceProcessing in tests >> - Track Cleaner progress with canaries >> - Amend benchmarks >> - Fix tests >> - Review feedback and benchmarks >> - ... and 4 more: https://git.openjdk.org/jdk/compare/f2507063...bcfab1ba > > src/java.base/share/classes/java/lang/ref/Reference.java line 282: > >> 280: // references, or (2) the reference processing thread is >> 281: // processing references. Otherwise, returns false immediately. >> 282: private static boolean waitForReferenceProcessing() > > Removing waitForReferenceProcessing is contrary to > https://bugs.openjdk.org/browse/JDK-8305186 > Reference.waitForReferenceProcessing should be more accessible > I've pointed to JDK-8305186 in a number of discussions about problems encountered with various tests. > So I'm doubtful about removing it. No problem, I will revert those removals. I did the removals proactively, thinking none of this infrastructure is needed after we migrate DBB, but since there are apparently plausible use cases for it, I'll leave it in. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1934481142 From shade at openjdk.org Wed Jan 29 19:56:00 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Jan 2025 19:56:00 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v11] In-Reply-To: References: Message-ID: > DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. > > We can migrate DBBs to use `java.lang.ref.Cleaner`. > > There are two pecularities during this rewrite. > > First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. > > Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. > > Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 > > Additional testing: > - [x] Linux x86_64 server fastdebug, `java/nio java/io` > - [x] Linux AArch64 server fastdebug, `java/nio java/io` > - [x] Linux x86_64 server fastdebug, `all` > - [x] Linux AArch64 server fastdebug, `all` Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Revert waitForReferenceProcessing removals, see JDK-8305186 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/22165/files - new: https://git.openjdk.org/jdk/pull/22165/files/bcfab1ba..dc3dab7f Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=10 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=22165&range=09-10 Stats: 72 lines in 5 files changed: 70 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/22165.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/22165/head:pull/22165 PR: https://git.openjdk.org/jdk/pull/22165 From shade at openjdk.org Wed Jan 29 19:56:01 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Jan 2025 19:56:01 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: References: Message-ID: <1kiBSlffsdUX423oAb73eb5uzJqw-QGhOlSOKQyYGIg=.34e8b7da-f266-41bb-a46e-7dd6e9fc578c@github.com> On Wed, 29 Jan 2025 19:42:49 GMT, Aleksey Shipilev wrote: >> src/java.base/share/classes/java/lang/ref/Reference.java line 282: >> >>> 280: * This method returns a strong reference to the referent. This may cause >>> 281: * the garbage collector to treat it as strongly reachable until some later >>> 282: * collection cycle. The {@link #refersTo(Object) refersTo} method can be >> >> Removing waitForReferenceProcessing is contrary to >> https://bugs.openjdk.org/browse/JDK-8305186 >> Reference.waitForReferenceProcessing should be more accessible >> I've pointed to JDK-8305186 in a number of discussions about problems encountered with various tests. >> So I'm doubtful about removing it. > > No problem, I will revert those removals. I did the removals proactively, thinking none of this infrastructure is needed after we migrate DBB, but since there are apparently plausible use cases for it, I'll leave it in. Done in new commit, thanks for taking a look. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1934492345 From bchristi at openjdk.org Wed Jan 29 23:12:02 2025 From: bchristi at openjdk.org (Brent Christian) Date: Wed, 29 Jan 2025 23:12:02 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v11] In-Reply-To: References: Message-ID: On Wed, 29 Jan 2025 19:56:00 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [x] Linux x86_64 server fastdebug, `all` >> - [x] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Revert waitForReferenceProcessing removals, see JDK-8305186 Thanks for restoring `waitForReferenceProcessing()`. Closed test changes are in, so you should be good to go. ------------- Marked as reviewed by bchristi (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/22165#pullrequestreview-2582568958 From kbarrett at openjdk.org Wed Jan 29 23:55:58 2025 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 29 Jan 2025 23:55:58 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v11] In-Reply-To: References: Message-ID: On Wed, 29 Jan 2025 19:56:00 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. >> >> Tests show that DirectBufferAlloc tests are still surviving this, possibly due to exponential sleep-backoff already built in. See the reclamation path in `Bits.unreserveMemory`: https://github.com/openjdk/jdk/blob/c207cc7e705d3f449f2387324d86cfb31ce40c44/src/java.base/share/classes/java/nio/Bits.java#L106-L186 >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [x] Linux x86_64 server fastdebug, `all` >> - [x] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Revert waitForReferenceProcessing removals, see JDK-8305186 src/java.base/share/classes/java/nio/Bits.java line 146: > 144: } > 145: > 146: if (canary == null || canary.isDead()) { If we're keeping Reference.waitForPendingReferences, why not continue to use it, rather than introducing the canary as a new, ad hoc, reference processing progress detector? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1934820754 From alanb at openjdk.org Thu Jan 30 08:40:00 2025 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 30 Jan 2025 08:40:00 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v11] In-Reply-To: References: Message-ID: On Wed, 29 Jan 2025 19:56:00 GMT, Aleksey Shipilev wrote: >> DirectByteBuffers are still using old `jdk.internal.ref.Cleaner` implementation. That implementation carries a doubly-linked list, and so makes DBB suffer from the same issue fixed for generic `java.lang.ref.Cleaner` users with [JDK-8343704](https://bugs.openjdk.org/browse/JDK-8343704). See the bug for the reproducer. >> >> We can migrate DBBs to use `java.lang.ref.Cleaner`. >> >> There are two pecularities during this rewrite. >> >> First, the old ad-hoc `Cleaner` implementation used to exit the VM when cleaning action failed. I presume it was to avoid memory leak / accidental reuse of the buffer. I moved the relevant block to `Deallocator` directly. Unfortunately, I cannot easily test it. >> >> Second is quite a bit hairy. Old DBB cleaning code was hooked straight into `Reference` processing loop. This was possible because we could infer that the weak references we are processing were DBB cleaning actions, since old `Cleaner` was the only use of this code. With standard `Cleaner`, we have lost this association, and so we cannot really do this from the reference processing loop. With the patched version, we now rely on normal `Cleaner` thread to do cleanups for us. >> >> Because of this, there is a new outpacing opportunity window where reference processing might have been over, but cleaner thread has not reacted yet. This is why we need another way to check progress that involves checking if cleaner has acted. >> >> Additional testing: >> - [x] Linux x86_64 server fastdebug, `java/nio java/io` >> - [x] Linux AArch64 server fastdebug, `java/nio java/io` >> - [x] Linux x86_64 server fastdebug, `all` >> - [x] Linux AArch64 server fastdebug, `all` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Revert waitForReferenceProcessing removals, see JDK-8305186 Marked as reviewed by alanb (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/22165#pullrequestreview-2583243848 From alanb at openjdk.org Thu Jan 30 08:40:01 2025 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 30 Jan 2025 08:40:01 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v9] In-Reply-To: References: <5jyLkYGrwl0zCPsaCj2-9dpglmONIXrI8BeJk3cysSY=.fbca0181-4a60-4178-9dba-aa5c5b1e28e8@github.com> Message-ID: On Sun, 26 Jan 2025 07:30:22 GMT, Alan Bateman wrote: >>> Testing is still clean on my side. I would appreciate if folks can run this through their CIs. >>> >>> I am planning to integrate this on Monday, as long as we discover no new issues. >> >> Did the change to `test/jdk/sun/security/provider/FileInputStreamPool/FileInputStreamPoolTest.java` get lost? >> I'm pretty sure it was updated, but I no longer see that in the changes, and it failed in my automated test run. > >> Testing is still clean on my side. I would appreciate if folks can run this through their CIs. >> >> I am planning to integrate this on Monday, as long as we discover no new issues. > > I ran it through our CI and we seem to have a few tests that aren't in the jdk repo but have a dependency on jdk.internal.ref.Cleaner so they will need to be fixed and/or excluded. Would you mind holding off until Tuesday to give time to get that done? Otherwise I didn't see any issues. I see there is a referenced Graal issue as it looks like they have to do an update to align with the changes here. > How is it going, @AlanBateman? Any progress on closed tests? Thanks for giving us time to get this sorted out. ------------- PR Comment: https://git.openjdk.org/jdk/pull/22165#issuecomment-2623847692 From alanb at openjdk.org Thu Jan 30 08:40:02 2025 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 30 Jan 2025 08:40:02 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v11] In-Reply-To: References: Message-ID: <9e-kLX5DVX9v0teMvEFIrp6QaKM5g66WCaR8-pOwF2E=.a2be9c9f-804f-4228-9032-535dbcb5ff50@github.com> On Wed, 29 Jan 2025 23:52:40 GMT, Kim Barrett wrote: >> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: >> >> Revert waitForReferenceProcessing removals, see JDK-8305186 > > src/java.base/share/classes/java/nio/Bits.java line 146: > >> 144: } >> 145: >> 146: if (canary == null || canary.isDead()) { > > If we're keeping Reference.waitForPendingReferences, why not continue to use it, rather than introducing > the canary as a new, ad hoc, reference processing progress detector? @kimbarrett Do you have a change coming to allow waitForPendingReferences be used by WB? I assume this will at least add a comment to the method (or whatever it changes to) to make it clear that it's for testing. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1935209408 From kbarrett at openjdk.org Thu Jan 30 16:29:01 2025 From: kbarrett at openjdk.org (Kim Barrett) Date: Thu, 30 Jan 2025 16:29:01 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v11] In-Reply-To: <9e-kLX5DVX9v0teMvEFIrp6QaKM5g66WCaR8-pOwF2E=.a2be9c9f-804f-4228-9032-535dbcb5ff50@github.com> References: <9e-kLX5DVX9v0teMvEFIrp6QaKM5g66WCaR8-pOwF2E=.a2be9c9f-804f-4228-9032-535dbcb5ff50@github.com> Message-ID: On Thu, 30 Jan 2025 08:36:34 GMT, Alan Bateman wrote: >> src/java.base/share/classes/java/nio/Bits.java line 146: >> >>> 144: } >>> 145: >>> 146: if (canary == null || canary.isDead()) { >> >> If we're keeping Reference.waitForPendingReferences, why not continue to use it, rather than introducing >> the canary as a new, ad hoc, reference processing progress detector? > > @kimbarrett Do you have a change coming to allow waitForPendingReferences be used by WB? I assume this will at least add a comment to the method (or whatever it changes to) to make it clear that it's for testing. @AlanBateman I've not done any work on JDK-8305186. There has also been discussion about making that function non-private or even public (though with concerns about specification difficulty) for use in places like this. @shipilev I'm working on a reply, but it might be long-ish. That outpacing issue for some tests is why this code wasn't switched away from jdk.internal.ref.Cleaner a long time ago. I'm still looking at it, but I currently don't think the canary provides a reliable solution to that. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1935907473 From alanb at openjdk.org Thu Jan 30 16:57:00 2025 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 30 Jan 2025 16:57:00 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v11] In-Reply-To: <9e-kLX5DVX9v0teMvEFIrp6QaKM5g66WCaR8-pOwF2E=.a2be9c9f-804f-4228-9032-535dbcb5ff50@github.com> References: <9e-kLX5DVX9v0teMvEFIrp6QaKM5g66WCaR8-pOwF2E=.a2be9c9f-804f-4228-9032-535dbcb5ff50@github.com> Message-ID: <95uXqmmcJ84nFyR6r0gw0aeGYBjFb_fdcUm_r7wtftQ=.9d0af897-c413-49dd-a09e-490a58bb1fda@github.com> On Thu, 30 Jan 2025 08:36:34 GMT, Alan Bateman wrote: >> src/java.base/share/classes/java/nio/Bits.java line 146: >> >>> 144: } >>> 145: >>> 146: if (canary == null || canary.isDead()) { >> >> If we're keeping Reference.waitForPendingReferences, why not continue to use it, rather than introducing >> the canary as a new, ad hoc, reference processing progress detector? > > @kimbarrett Do you have a change coming to allow waitForPendingReferences be used by WB? I assume this will at least add a comment to the method (or whatever it changes to) to make it clear that it's for testing. > @AlanBateman I've not done any work on JDK-8305186. There has also been discussion about making that function non-private or even public (though with concerns about specification difficulty) for use in places like this. Exposing this in this API might be scary, I think would need a lot of thinking to be confident it doesn't create an attractive nuisance. Keeping it for WB testing is okay of course. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1935953253 From shade at openjdk.org Thu Jan 30 17:07:06 2025 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 30 Jan 2025 17:07:06 GMT Subject: RFR: 8344332: (bf) Migrate DirectByteBuffer to use java.lang.ref.Cleaner [v11] In-Reply-To: <95uXqmmcJ84nFyR6r0gw0aeGYBjFb_fdcUm_r7wtftQ=.9d0af897-c413-49dd-a09e-490a58bb1fda@github.com> References: <9e-kLX5DVX9v0teMvEFIrp6QaKM5g66WCaR8-pOwF2E=.a2be9c9f-804f-4228-9032-535dbcb5ff50@github.com> <95uXqmmcJ84nFyR6r0gw0aeGYBjFb_fdcUm_r7wtftQ=.9d0af897-c413-49dd-a09e-490a58bb1fda@github.com> Message-ID: On Thu, 30 Jan 2025 16:54:16 GMT, Alan Bateman wrote: >> @kimbarrett Do you have a change coming to allow waitForPendingReferences be used by WB? I assume this will at least add a comment to the method (or whatever it changes to) to make it clear that it's for testing. > >> @AlanBateman I've not done any work on JDK-8305186. There has also been discussion about making that function non-private or even public (though with concerns about specification difficulty) for use in places like this. > > Exposing this in this API might be scary, I think would need a lot of thinking to be confident it doesn't create an attractive nuisance. Keeping it for WB testing is okay of course. The asynchronicity of buffer cleanup would IMO always be a headache, so we would need some sort of feedback loop. Cleaner canary is not a perfect vehicle for this, since it does not tell us directly that any buffer cleanup happened, like it used to be with reference handling thread invoking `jdk.internal.ref.Cleaner` directly. We could reinstate the similarly-shaped hack by tracking if buffer-attached cleaners have fired. That said, I have been in the room when this hack was introduced, we recognized it was not perfect, and I remember we were happy this kind of feedback still worked in practice, seeing no failures on `DirectBufferAllocTest` stress tests. So when introducing the canaries, I planned to arrive to the same happy point: stress`DirectBufferAllocTest` works in all practical configurations. There could probably be cases where we would throw OOME, when we could have waited a bit more or do a bit more cleanups. But then again, this would not be a correctness issue, but rather a performance pothole, which is, again, does not show up on stress tests, so the probability of it seems fairly low. So while we are discussing this, I think we should really keep an eye on practical cost-benefit balance here. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/22165#discussion_r1935969193