From alanb at openjdk.org Sat Oct 1 07:40:29 2022 From: alanb at openjdk.org (Alan Bateman) Date: Sat, 1 Oct 2022 07:40:29 GMT Subject: RFR: 8294437: java/nio/channels/FileChannel tests slow on Windows [v5] In-Reply-To: References: Message-ID: On Fri, 30 Sep 2022 20:25:27 GMT, Daniel Jeli?ski wrote: >> Please review this test-only change that improves the execution speed of a few FileChannel tests: >> - Transfer2GPlus, Transfer4GBFile, and TransferTo6GBFile were modified to use sparse files. Their speed is now consistent across platforms, provided that the platform / filesystem supports sparse files. >> - LargeMapTest was rewritten to use sparse files, and to more precisely target the issue it was written to detect. In my tests it still crashed with EXCEPTION_ACCESS_VIOLATION when JDK-8286637 was reverted. >> - MapTest: repetition was removed from `testForce` method; I'm not sure what issues it was supposed to catch, but at far as I can tell, it was only triggering timeouts, see [JDK-8289526](https://bugs.openjdk.org/browse/JDK-8289526), [JDK-8224480](https://bugs.openjdk.org/browse/JDK-8224480) > > Daniel Jeli?ski has updated the pull request incrementally with one additional commit since the last revision: > > Use try with resources Thanks for the update, I don't have any other comments. Will be good to get this change in as these tests are so slow on Windows right now. ------------- Marked as reviewed by alanb (Reviewer). PR: https://git.openjdk.org/jdk/pull/10464 From mkartashev at openjdk.org Mon Oct 3 08:25:34 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Mon, 3 Oct 2022 08:25:34 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Sun, 11 Sep 2022 18:20:57 GMT, Alan Bateman wrote: >> This is an implementation of `WatchService` based on File System Events API that is capable of generating events whenever a change occurs in an interesting directory or underneath it. Since the API naturally supports "recursive" watch, the `FILE_TREE` is supported by the watch service. >> >> Some things of note: >> * There's one "service" thread per `WatchService` instance that is inactive unless changes occur in the watched directory. The changes are grouped by introducing a time delay between when they occurred and when they are reported, which is controlled by the sensitivity modifier of the watch service. >> * Since FSEvents API reports directories only, the watch service keeps a snapshot (hierarchical if necessary) of the files in the directory being watched. The snapshot gets updated when an event in that directory or underneath it gets delivered. File changes are detected by comparing "last modified" time with a millisecond precision (`BasicFileAttributes.lastModifiedTime()`). >> * There is a slight complication with the move of an entire directory hierarchy: FSEvents API only reports about the containing directory of that move and not about any of the directories actually moved. There's a separate test for that (`Move.java`). >> * The code is careful not to do any I/O (such as reading the contents of a directory or attributes of a file) unless unavoidable. Any deviation from this line should be considered a bug (of, arguably, low priority). >> * The native part consists mostly of API wrappers with one exception of the callback function invoked by the system to report the events that occurred. The sole task of the function is to convert from C strings to Java strings and pass the array of affected directories to Java code. This can be rewritten if desired to make the code more future-proof. >> >> This commit leaves `PollingWatchService` unused. I'm not sure if I should/can do anything about it. Any advice is welcomed. >> >> ### Testing >> >> * Tested by running `test/jdk/java/nio/file` and `test/jdk/jdk/nio` on MacOS 10.15.7 (x64) and `test/jdk/java/nio/` plus `test/jdk/jdk/nio` on MacOS 12.5.1 (aarch64). >> * Also verified that new tests pass on Linux and Windows. >> * This code (albeit in a slightly modified form) has been in use at JetBrains for around half a year and a few bugs have been found and fixed during that time period. > > I did a first pass over this. It's a good start and demonstrates that the file system events / FSEventStream* API can be used to implement WatchService. > > It's really unfortunate that it requires calling CFRunLoopRun to do the run loop as that means an upcall and JNI code that we would normally try to keep out of this area. One concern is that there is a lot of UnixPath (bytes) <-> String <-> CFString conversions going on and these will need time to work through (even with UTF-8). Also > CFRunLoopThread will need to be an InnocuousThread to avoid inheriting inheritable thread locals or context class loader. > > One architectural issue is that the WatchService implementation should not be using Path.of or Files.* methods as that will cause a loop when interposing on the default provider. The provider implementation should only use use the Unix provider methods directly. > > The use of toRealPath().normalize() looks odd, I would expect the normalize to be a no-op here. This is an area that will require careful study as it's just not clear how the macOS work when the sym link is to another directory on another file system. > > Look at ptr_to_jlong and jlong_to_jptr to go between points and jlong and that should eliminate some of the casts. > > The code includes SUBTREE support. I had hope this wouldn't be included initially as it adds a bit more complexity and could be added later. > > Style-wise the code is very different to the existing code and I think we'll need to do a few cleanup passes (overly long lines, naming, formatting, overdue of final, ... all minor stuff and there is a lot to go through before these things). @AlanBateman Do you have any follow-up comments? I'd like to move this one forward. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From alanb at openjdk.org Mon Oct 3 08:38:28 2022 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 3 Oct 2022 08:38:28 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Sun, 11 Sep 2022 18:20:57 GMT, Alan Bateman wrote: >> This is an implementation of `WatchService` based on File System Events API that is capable of generating events whenever a change occurs in an interesting directory or underneath it. Since the API naturally supports "recursive" watch, the `FILE_TREE` is supported by the watch service. >> >> Some things of note: >> * There's one "service" thread per `WatchService` instance that is inactive unless changes occur in the watched directory. The changes are grouped by introducing a time delay between when they occurred and when they are reported, which is controlled by the sensitivity modifier of the watch service. >> * Since FSEvents API reports directories only, the watch service keeps a snapshot (hierarchical if necessary) of the files in the directory being watched. The snapshot gets updated when an event in that directory or underneath it gets delivered. File changes are detected by comparing "last modified" time with a millisecond precision (`BasicFileAttributes.lastModifiedTime()`). >> * There is a slight complication with the move of an entire directory hierarchy: FSEvents API only reports about the containing directory of that move and not about any of the directories actually moved. There's a separate test for that (`Move.java`). >> * The code is careful not to do any I/O (such as reading the contents of a directory or attributes of a file) unless unavoidable. Any deviation from this line should be considered a bug (of, arguably, low priority). >> * The native part consists mostly of API wrappers with one exception of the callback function invoked by the system to report the events that occurred. The sole task of the function is to convert from C strings to Java strings and pass the array of affected directories to Java code. This can be rewritten if desired to make the code more future-proof. >> >> This commit leaves `PollingWatchService` unused. I'm not sure if I should/can do anything about it. Any advice is welcomed. >> >> ### Testing >> >> * Tested by running `test/jdk/java/nio/file` and `test/jdk/jdk/nio` on MacOS 10.15.7 (x64) and `test/jdk/java/nio/` plus `test/jdk/jdk/nio` on MacOS 12.5.1 (aarch64). >> * Also verified that new tests pass on Linux and Windows. >> * This code (albeit in a slightly modified form) has been in use at JetBrains for around half a year and a few bugs have been found and fixed during that time period. > > I did a first pass over this. It's a good start and demonstrates that the file system events / FSEventStream* API can be used to implement WatchService. > > It's really unfortunate that it requires calling CFRunLoopRun to do the run loop as that means an upcall and JNI code that we would normally try to keep out of this area. One concern is that there is a lot of UnixPath (bytes) <-> String <-> CFString conversions going on and these will need time to work through (even with UTF-8). Also > CFRunLoopThread will need to be an InnocuousThread to avoid inheriting inheritable thread locals or context class loader. > > One architectural issue is that the WatchService implementation should not be using Path.of or Files.* methods as that will cause a loop when interposing on the default provider. The provider implementation should only use use the Unix provider methods directly. > > The use of toRealPath().normalize() looks odd, I would expect the normalize to be a no-op here. This is an area that will require careful study as it's just not clear how the macOS work when the sym link is to another directory on another file system. > > Look at ptr_to_jlong and jlong_to_jptr to go between points and jlong and that should eliminate some of the casts. > > The code includes SUBTREE support. I had hope this wouldn't be included initially as it adds a bit more complexity and could be added later. > > Style-wise the code is very different to the existing code and I think we'll need to do a few cleanup passes (overly long lines, naming, formatting, overdue of final, ... all minor stuff and there is a lot to go through before these things). > @AlanBateman Do you have any follow-up comments? I'd like to move this one forward. It's on my list to do another pass on this. I think the main thing that I'd like us to explore is using interfaces that would avoid the upcall. It would be good to see if there are any options that are feasible there. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From mkartashev at openjdk.org Mon Oct 3 10:34:16 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Mon, 3 Oct 2022 10:34:16 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Mon, 3 Oct 2022 08:36:22 GMT, Alan Bateman wrote: > It's on my list to do another pass on this. Thank you! > I think the main thing that I'd like us to explore is using interfaces that would avoid the upcall. It would be good to see if there are any options that are feasible there. It greatly depends on the definition of "upcall". I understand this as any call from native to Java code. The FSEvents API employs a callback-based approach, there's nothing to be explored there. However, as Brian suggested earlier, there are still options like reducing the complexity of the upcall to a mere `Object.notify()` or other simple means of synchronization. The side effect of such a change would be a much more complicated native portion of the callback, which I understood is something to be avoided. So I'm at a crossroads and would like a clearer guidance on the matter. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From alanb at openjdk.org Mon Oct 3 10:39:22 2022 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 3 Oct 2022 10:39:22 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Mon, 3 Oct 2022 10:30:22 GMT, Maxim Kartashev wrote: > The FSEvents API employs a callback-based approach, there's nothing to be explored there. The suggestion was to explore doing this at a low level with kqueue so that we don't have callbacks. I think this was the direction that Brian was exploring at one point. I do plan to do a detailed walk through of the FSEvents prototype. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From mkartashev at openjdk.org Mon Oct 3 11:34:36 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Mon, 3 Oct 2022 11:34:36 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Mon, 3 Oct 2022 10:35:44 GMT, Alan Bateman wrote: > The suggestion was to explore doing this at a low level with kqueue so that we don't have callbacks. Vladimir Kempik had a draft implementation based on kqueue that, AFAIR, passed all the existing tests; he may be interested in getting his work merged into the mainline. My primary concern with this approach is that it doesn't give a lot of advantage over `PollingWatchService` when you need to watch for changes in a very deep hierarchy of directories, the main use case for my employer. At the same time FSEvents API naturally provide the ability for recursive watch almost for free resource-wise. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From alanb at openjdk.org Mon Oct 3 11:40:25 2022 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 3 Oct 2022 11:40:25 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Mon, 3 Oct 2022 11:32:18 GMT, Maxim Kartashev wrote: > My primary concern with this approach is that it doesn't give a lot of advantage over `PollingWatchService` when you need to watch for changes in a very deep hierarchy of directories, the main use case for my employer. At the same time FSEvents API naturally provide the ability for recursive watch almost for free resource-wise. Do you know if FSEvents is implemented on a supported/document interface? I think it would be useful to know if there are alternatives that provide a way to block for events so we don't have the callback. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From vkempik at openjdk.org Mon Oct 3 11:46:26 2022 From: vkempik at openjdk.org (Vladimir Kempik) Date: Mon, 3 Oct 2022 11:46:26 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v7] In-Reply-To: References: Message-ID: On Fri, 30 Sep 2022 05:17:43 GMT, Maxim Kartashev wrote: >> This is an implementation of `WatchService` based on File System Events API that is capable of generating events whenever a change occurs in an interesting directory or underneath it. Since the API naturally supports "recursive" watch, the `FILE_TREE` is supported by the watch service. >> >> Some things of note: >> * There's one "service" thread per `WatchService` instance that is inactive unless changes occur in the watched directory. The changes are grouped by introducing a time delay between when they occurred and when they are reported, which is controlled by the sensitivity modifier of the watch service. >> * Since FSEvents API reports directories only, the watch service keeps a snapshot (hierarchical if necessary) of the files in the directory being watched. The snapshot gets updated when an event in that directory or underneath it gets delivered. File changes are detected by comparing "last modified" time with a millisecond precision (`BasicFileAttributes.lastModifiedTime()`). >> * There is a slight complication with the move of an entire directory hierarchy: FSEvents API only reports about the containing directory of that move and not about any of the directories actually moved. There's a separate test for that (`Move.java`). >> * The code is careful not to do any I/O (such as reading the contents of a directory or attributes of a file) unless unavoidable. Any deviation from this line should be considered a bug (of, arguably, low priority). >> * The native part consists mostly of API wrappers with one exception of the callback function invoked by the system to report the events that occurred. The sole task of the function is to convert from C strings to Java strings and pass the array of affected directories to Java code. This can be rewritten if desired to make the code more future-proof. >> >> This commit leaves `PollingWatchService` unused. I'm not sure if I should/can do anything about it. Any advice is welcomed. >> >> ### Testing >> >> * Tested by running `test/jdk/java/nio/file` and `test/jdk/jdk/nio` on MacOS 10.15.7 (x64) and `test/jdk/java/nio/` plus `test/jdk/jdk/nio` on MacOS 12.5.1 (aarch64). >> * Also verified that new tests pass on Linux and Windows. >> * This code (albeit in a slightly modified form) has been in use at JetBrains for around half a year and a few bugs have been found and fixed during that time period. > > Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: > > Dropped __unused attributes from used parameters kqueue implementation is generally bad idea, example of implementation: https://github.com/VladimirKempik/jdk/commit/507f8acecb2d48d0673d63386b1959ddb8c040a3 , first of all: 1) The kqueue monitor requires a file descriptor to be opened for every file being watched. As a result, this monitor scales badly with the number of files being observed and may begin to misbehave as soon as the process runs out of file descriptors. the limit is about 10k files 2) this modifier StandardWatchEventKinds.ENTRY_MODIFY says to monitor changes of individual files in the dir, but kevent on the dir only reports changes to the dir itself ( e.g. file added, file removed) and won't says anything about changes to file in the dir ( file modified). As a result we still have to use good old periodic polling on the directories and can miss some event ( e.g. file create and removed immediately) ------------- PR: https://git.openjdk.org/jdk/pull/10140 From mkartashev at openjdk.org Mon Oct 3 12:04:31 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Mon, 3 Oct 2022 12:04:31 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Mon, 3 Oct 2022 11:36:48 GMT, Alan Bateman wrote: > Do you know if FSEvents is implemented on a supported/documenedt interface? Unfortunately, my knowledge doesn't go deep into OSX. FWIW, I haven't seen anything like that when I was researching for this implementation. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From mik3hall at gmail.com Mon Oct 3 12:37:20 2022 From: mik3hall at gmail.com (Michael Hall) Date: Mon, 3 Oct 2022 07:37:20 -0500 Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v7] In-Reply-To: References: Message-ID: <1A386793-0B6F-475E-881B-1E41835B0B03@gmail.com> > On Oct 3, 2022, at 6:46 AM, Vladimir Kempik wrote: > > 2) this modifier StandardWatchEventKinds.ENTRY_MODIFY says to monitor changes of individual files in the dir, but kevent on the dir only reports changes to the dir itself Seems somewhat off-topic to FSEvents implementation but it got me to look at some of my old attempt. You could just add all the files in the directory. [self addChildren:[NSURL fileURLWithPath:path] notifyingAbout:fflags withWatchKey:watchKey]; This of course does nothing for the possible file descriptor issues. As near as I can tell on a quick look the only time I freed up a descriptor was if the file was deleted. My implementation however, usually did no polling. I only added some polling when I ran into what seemed like deadlock issues to make sure the thread wasn?t itself deadlocked somehow. As an ?alive? test. I did a lot of googling on this but never hit on mention of descriptors. I don?t think I?ll go over the code too much more though. The current FSEvents implementation looks fine to me. Based on a officially Apple supported API and extension I think to kqueue? If I remember right they had a couple different file monitoring extensions that differed in granularity. One less granular I think could watch for any file changes on the machine. Callbacks are fairly common in Apple OS/X code aren?t they? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rriggs at openjdk.org Mon Oct 3 15:30:23 2022 From: rriggs at openjdk.org (Roger Riggs) Date: Mon, 3 Oct 2022 15:30:23 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v18] In-Reply-To: References: Message-ID: On Mon, 1 Aug 2022 21:54:22 GMT, Brian Burkhalter wrote: >> Resurrection of the proposal to add a method to obtain the filename extension originated in PR [2319](https://github.com/openjdk/jdk/pull/2319). > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8057113: Remove API note For a filename to have an extension, then I think there needs to be a non-empty root part. So: ".z", root = ".z" ext = null "z.", root = "z" ext = "" "..z", root = "." ext = "z" "...foo", root = "..", ext = "foo" ".....", root = "....", ext = "" Algorithm wise, you can find the last "." and check if there is a non-empty string before it. Scanning of the prefix is not necessary. ------------- PR: https://git.openjdk.org/jdk/pull/8066 From bpb at openjdk.org Mon Oct 3 16:24:21 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 3 Oct 2022 16:24:21 GMT Subject: RFR: 8294437: java/nio/channels/FileChannel tests slow on Windows [v5] In-Reply-To: References: Message-ID: <0t7naOuah8F7-5s1Uy3szmVCzcVRvLikV8zUnK9EcUA=.e333770e-d0ec-4cab-8fb6-14ee65085880@github.com> On Fri, 30 Sep 2022 20:25:27 GMT, Daniel Jeli?ski wrote: >> Please review this test-only change that improves the execution speed of a few FileChannel tests: >> - Transfer2GPlus, Transfer4GBFile, and TransferTo6GBFile were modified to use sparse files. Their speed is now consistent across platforms, provided that the platform / filesystem supports sparse files. >> - LargeMapTest was rewritten to use sparse files, and to more precisely target the issue it was written to detect. In my tests it still crashed with EXCEPTION_ACCESS_VIOLATION when JDK-8286637 was reverted. >> - MapTest: repetition was removed from `testForce` method; I'm not sure what issues it was supposed to catch, but at far as I can tell, it was only triggering timeouts, see [JDK-8289526](https://bugs.openjdk.org/browse/JDK-8289526), [JDK-8224480](https://bugs.openjdk.org/browse/JDK-8224480) > > Daniel Jeli?ski has updated the pull request incrementally with one additional commit since the last revision: > > Use try with resources test/jdk/java/nio/channels/FileChannel/TransferTo6GBFile.java line 69: > 67: t1 - t0, TimeUnit.NANOSECONDS.toMillis(t1 - t0)); > 68: > 69: // Setup looback connection and echo server "looback" -> "loopback" ------------- PR: https://git.openjdk.org/jdk/pull/10464 From bpb at openjdk.org Mon Oct 3 16:28:27 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 3 Oct 2022 16:28:27 GMT Subject: RFR: 8294437: java/nio/channels/FileChannel tests slow on Windows [v5] In-Reply-To: References: Message-ID: On Fri, 30 Sep 2022 20:25:27 GMT, Daniel Jeli?ski wrote: >> Please review this test-only change that improves the execution speed of a few FileChannel tests: >> - Transfer2GPlus, Transfer4GBFile, and TransferTo6GBFile were modified to use sparse files. Their speed is now consistent across platforms, provided that the platform / filesystem supports sparse files. >> - LargeMapTest was rewritten to use sparse files, and to more precisely target the issue it was written to detect. In my tests it still crashed with EXCEPTION_ACCESS_VIOLATION when JDK-8286637 was reverted. >> - MapTest: repetition was removed from `testForce` method; I'm not sure what issues it was supposed to catch, but at far as I can tell, it was only triggering timeouts, see [JDK-8289526](https://bugs.openjdk.org/browse/JDK-8289526), [JDK-8224480](https://bugs.openjdk.org/browse/JDK-8224480) > > Daniel Jeli?ski has updated the pull request incrementally with one additional commit since the last revision: > > Use try with resources Marked as reviewed by bpb (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/10464 From mr at openjdk.org Mon Oct 3 16:57:27 2022 From: mr at openjdk.org (Mark Reinhold) Date: Mon, 3 Oct 2022 16:57:27 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v18] In-Reply-To: References: Message-ID: On Mon, 1 Aug 2022 21:54:22 GMT, Brian Burkhalter wrote: >> Resurrection of the proposal to add a method to obtain the filename extension originated in PR [2319](https://github.com/openjdk/jdk/pull/2319). > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8057113: Remove API note The most surprising thing would be to consider `.foo` to have the extension `foo`, which would make it impossible to treat `.foo` together with `.foo.log` in a sensible way. Multiple leading period characters are a corner case of a corner case. ------------- PR: https://git.openjdk.org/jdk/pull/8066 From djelinski at openjdk.org Mon Oct 3 17:40:39 2022 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 3 Oct 2022 17:40:39 GMT Subject: RFR: 8294437: java/nio/channels/FileChannel tests slow on Windows [v6] In-Reply-To: References: Message-ID: > Please review this test-only change that improves the execution speed of a few FileChannel tests: > - Transfer2GPlus, Transfer4GBFile, and TransferTo6GBFile were modified to use sparse files. Their speed is now consistent across platforms, provided that the platform / filesystem supports sparse files. > - LargeMapTest was rewritten to use sparse files, and to more precisely target the issue it was written to detect. In my tests it still crashed with EXCEPTION_ACCESS_VIOLATION when JDK-8286637 was reverted. > - MapTest: repetition was removed from `testForce` method; I'm not sure what issues it was supposed to catch, but at far as I can tell, it was only triggering timeouts, see [JDK-8289526](https://bugs.openjdk.org/browse/JDK-8289526), [JDK-8224480](https://bugs.openjdk.org/browse/JDK-8224480) Daniel Jeli?ski has updated the pull request incrementally with one additional commit since the last revision: Fix typo ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10464/files - new: https://git.openjdk.org/jdk/pull/10464/files/0f22fb54..148ea2cc Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10464&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10464&range=04-05 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/10464.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10464/head:pull/10464 PR: https://git.openjdk.org/jdk/pull/10464 From djelinski at openjdk.org Mon Oct 3 17:40:39 2022 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 3 Oct 2022 17:40:39 GMT Subject: RFR: 8294437: java/nio/channels/FileChannel tests slow on Windows [v5] In-Reply-To: References: Message-ID: On Fri, 30 Sep 2022 20:25:27 GMT, Daniel Jeli?ski wrote: >> Please review this test-only change that improves the execution speed of a few FileChannel tests: >> - Transfer2GPlus, Transfer4GBFile, and TransferTo6GBFile were modified to use sparse files. Their speed is now consistent across platforms, provided that the platform / filesystem supports sparse files. >> - LargeMapTest was rewritten to use sparse files, and to more precisely target the issue it was written to detect. In my tests it still crashed with EXCEPTION_ACCESS_VIOLATION when JDK-8286637 was reverted. >> - MapTest: repetition was removed from `testForce` method; I'm not sure what issues it was supposed to catch, but at far as I can tell, it was only triggering timeouts, see [JDK-8289526](https://bugs.openjdk.org/browse/JDK-8289526), [JDK-8224480](https://bugs.openjdk.org/browse/JDK-8224480) > > Daniel Jeli?ski has updated the pull request incrementally with one additional commit since the last revision: > > Use try with resources Typo fixed. Thanks for the reviews! ------------- PR: https://git.openjdk.org/jdk/pull/10464 From djelinski at openjdk.org Mon Oct 3 17:46:47 2022 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 3 Oct 2022 17:46:47 GMT Subject: Integrated: 8294437: java/nio/channels/FileChannel tests slow on Windows In-Reply-To: References: Message-ID: On Wed, 28 Sep 2022 10:58:43 GMT, Daniel Jeli?ski wrote: > Please review this test-only change that improves the execution speed of a few FileChannel tests: > - Transfer2GPlus, Transfer4GBFile, and TransferTo6GBFile were modified to use sparse files. Their speed is now consistent across platforms, provided that the platform / filesystem supports sparse files. > - LargeMapTest was rewritten to use sparse files, and to more precisely target the issue it was written to detect. In my tests it still crashed with EXCEPTION_ACCESS_VIOLATION when JDK-8286637 was reverted. > - MapTest: repetition was removed from `testForce` method; I'm not sure what issues it was supposed to catch, but at far as I can tell, it was only triggering timeouts, see [JDK-8289526](https://bugs.openjdk.org/browse/JDK-8289526), [JDK-8224480](https://bugs.openjdk.org/browse/JDK-8224480) This pull request has now been integrated. Changeset: a4f2078b Author: Daniel Jeli?ski URL: https://git.openjdk.org/jdk/commit/a4f2078bd60d06795099c52ca8d437a8128eb8f6 Stats: 462 lines in 6 files changed: 180 ins; 105 del; 177 mod 8294437: java/nio/channels/FileChannel tests slow on Windows Reviewed-by: alanb, bpb ------------- PR: https://git.openjdk.org/jdk/pull/10464 From bpb at openjdk.org Mon Oct 3 19:09:31 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 3 Oct 2022 19:09:31 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v18] In-Reply-To: References: Message-ID: On Mon, 1 Aug 2022 21:54:22 GMT, Brian Burkhalter wrote: >> Resurrection of the proposal to add a method to obtain the filename extension originated in PR [2319](https://github.com/openjdk/jdk/pull/2319). > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8057113: Remove API note Interestingly, double clicking on both the files `.txt` and `...txt` with identical content (and no file signature) in Linux File Manager, macOS Finder, and Windows Explorer all open the respective platform default text editor. ------------- PR: https://git.openjdk.org/jdk/pull/8066 From brian.burkhalter at oracle.com Mon Oct 3 19:16:25 2022 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 3 Oct 2022 19:16:25 +0000 Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Oct 3, 2022, at 4:34 AM, Maxim Kartashev > wrote: On Mon, 3 Oct 2022 10:35:44 GMT, Alan Bateman > wrote: The suggestion was to explore doing this at a low level with kqueue so that we don't have callbacks. Vladimir Kempik had a draft implementation based on kqueue that, AFAIR, passed all the existing tests; he may be interested in getting his work merged into the mainline. As I mentioned, I wrote a prototype based on kqueue and it also (usually) passed all the tests but had problems shutting down. It also sometimes ran out of file descriptors. Unless my testing was wrong, I think that using kqueue here is likely to run into the exhausted file descriptor situation frequently if the `ENTRY_MODIFY` event is watched. `ENTRY_CREATE` and `ENTRY_DELETE` could be detected by watching the directory file descriptor alone and rescanning the directory, but modifying a contained file alone will not generate any event on the directory descriptor, hence watching for `ENTRY_MODIFY` requires an open descriptor for each file in the directory. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bpb at openjdk.org Mon Oct 3 19:22:24 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 3 Oct 2022 19:22:24 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Mon, 3 Oct 2022 11:36:48 GMT, Alan Bateman wrote: > My primary concern with this approach is that it doesn't give a lot of advantage over `PollingWatchService` [...]. Has there been any form of bechmarking comparison other than simple subjective observation? ------------- PR: https://git.openjdk.org/jdk/pull/10140 From mr at openjdk.org Mon Oct 3 20:07:31 2022 From: mr at openjdk.org (Mark Reinhold) Date: Mon, 3 Oct 2022 20:07:31 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v18] In-Reply-To: References: Message-ID: <5kWk4eK9PtFqPgTJjyymqxyiTDRC-UKdte3vxdhwA30=.49440d78-db24-4333-9c59-18ee9e82fef7@github.com> On Mon, 3 Oct 2022 19:05:34 GMT, Brian Burkhalter wrote: > Interestingly, double clicking on both the files `.txt` and `...txt` with identical content (and no file signature) in Linux File Manager, macOS Finder, and Windows Explorer all open the respective platform default text editor. Is that because those systems are interpreting the `txt` suffix, or is it because they always open files of unknown type with the platform?s default text editor? ------------- PR: https://git.openjdk.org/jdk/pull/8066 From mkartashev at openjdk.org Tue Oct 4 06:20:59 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Tue, 4 Oct 2022 06:20:59 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Mon, 3 Oct 2022 19:20:07 GMT, Brian Burkhalter wrote: > Has there been any form of bechmarking comparison other than simple subjective observation? Yes, but it was about 9 months ago and I didn't save the data. Since Vladimir kept the [kqueues-based implementation](https://github.com/VladimirKempik/jdk/commit/507f8acecb2d48d0673d63386b1959ddb8c040a3), anyone willing to experiment and collect new data can do so relatively easily. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From vkempik at openjdk.org Tue Oct 4 08:11:05 2022 From: vkempik at openjdk.org (Vladimir Kempik) Date: Tue, 4 Oct 2022 08:11:05 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Tue, 4 Oct 2022 06:50:01 GMT, Brian Burkhalter wrote: >but modifying a contained file alone will not generate any event on the directory descriptor, hence watching for `ENTRY_MODIFY` requires an open descriptor for each file in the directory. exactly, the main reason I have dropped that work. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From bpb at openjdk.org Tue Oct 4 15:41:23 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 4 Oct 2022 15:41:23 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Tue, 4 Oct 2022 08:07:27 GMT, Vladimir Kempik wrote: >> On Oct 3\, 2022\, at 4\:34 AM\, Maxim Kartashev \> wrote\: >> >> On Mon\, 3 Oct 2022 10\:35\:44 GMT\, Alan Bateman \> wrote\: >> >> The suggestion was to explore doing this at a low level with kqueue so that we don\'t have callbacks\. >> >> Vladimir Kempik had a draft implementation based on kqueue that\, AFAIR\, passed all the existing tests\; he may be interested in getting his work merged into the mainline\. >> >> As I mentioned\, I wrote a prototype based on kqueue and it also \(usually\) passed all the tests but had problems shutting down\. It also sometimes ran out of file descriptors\. Unless my testing was wrong\, I think that using kqueue here is likely to run into the exhausted file descriptor situation frequently if the \`ENTRY\_MODIFY\` event is watched\. \`ENTRY\_CREATE\` and \`ENTRY\_DELETE\` could be detected by watching the directory file descriptor alone and rescanning the directory\, but modifying a contained file alone will not generate any event on the directory descriptor\, hence watching for \`ENTRY\_MODIFY\` requires an open descriptor for each file in the directory\. >> \-\-\-\-\-\-\-\-\-\-\-\-\-\- next part \-\-\-\-\-\-\-\-\-\-\-\-\-\- >> An HTML attachment was scrubbed\.\.\. >> URL\: \ > >>but modifying a contained file alone will not generate any event on the directory descriptor, hence watching for `ENTRY_MODIFY` requires an open descriptor for each file in the directory. > > exactly, the main reason I have dropped that work. @VladimirKempik Thank you for corroborating my findings. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From cjplummer at openjdk.org Tue Oct 4 18:23:32 2022 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 4 Oct 2022 18:23:32 GMT Subject: RFR: 8294321: Fix typos in files under test/jdk/java, test/jdk/jdk, test/jdk/jni [v2] In-Reply-To: References: Message-ID: On Wed, 28 Sep 2022 15:32:11 GMT, Michael Ernst wrote: > The title was edited by someone other than me, as you can see from the PR history. The PR title needs to match the CR synopsis, so update the CR first, and then update the PR. ------------- PR: https://git.openjdk.org/jdk/pull/10029 From aivanov at openjdk.org Wed Oct 5 13:27:29 2022 From: aivanov at openjdk.org (Alexey Ivanov) Date: Wed, 5 Oct 2022 13:27:29 GMT Subject: RFR: 8294321: Fix typos in files under test/jdk/java, test/jdk/jdk, test/jdk/jni [v2] In-Reply-To: References: Message-ID: On Mon, 26 Sep 2022 16:51:36 GMT, Michael Ernst wrote: >> 8294321: Fix typos in files under test/jdk/java, test/jdk/jdk, test/jdk/jni > > Michael Ernst has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: > > - Reinstate typos in Apache code that is copied into the JDK > - Merge ../jdk-openjdk into typos-typos > - Remove file that was removed upstream > - Fix inconsistency in capitalization > - Undo change in zlip > - Fix typos Changes requested by aivanov (Reviewer). src/hotspot/share/opto/memnode.cpp line 2365: > 2363: if (x != this) return x; > 2364: > 2365: // Take apart the address into an oop and offset. ?and _an_ offset?? src/java.xml/share/classes/org/w3c/dom/Document.java line 293: > 291: * systemId, and notationName attributes are > 292: * copied. If a deep import is requested, the descendants > 293: * of the source Entity are recursively imported and This class may come from a 3rd party library. Anyone from `java.xml` can confirm it? test/hotspot/jtreg/vmTestbase/nsk/share/locks/DeadlockMaker.java line 31: > 29: /* > 30: * Class used to create deadlocked threads. It is possible create 2 or more deadlocked thread, also > 31: * is possible to specify resource of which type should lock each deadlocked thread Suggestion: * it is possible to specify resource of which type should lock each deadlocked thread It doesn't sound right without _?it?_. test/jdk/com/sun/jdi/connect/spi/GeneratedConnectors.java line 28: > 26: * @summary Unit test for "Pluggable Connectors and Transports" feature. > 27: * > 28: * When a transport service is deployed the virtual machine Suggestion: * When a transport service is deployed, the virtual machine Let's add a comma for clarity. test/jdk/java/security/testlibrary/SimpleOCSPServer.java line 445: > 443: > 444: /** > 445: * Check the status database for revocation information on one or more Suggestion: * Check the status database for revocation information of one or more test/jdk/sun/jvmstat/testlibrary/utils.sh line 181: > 179: if [ $? -eq 0 ] > 180: then > 181: # it's still lingering, now it is hard Suggestion: # it's still lingering, now hit it hard ------------- PR: https://git.openjdk.org/jdk/pull/10029 From aivanov at openjdk.org Wed Oct 5 14:17:13 2022 From: aivanov at openjdk.org (Alexey Ivanov) Date: Wed, 5 Oct 2022 14:17:13 GMT Subject: RFR: 8294321: Fix typos in files under test/jdk/java, test/jdk/jdk, test/jdk/jni [v2] In-Reply-To: References: Message-ID: On Mon, 26 Sep 2022 16:51:36 GMT, Michael Ernst wrote: >> 8294321: Fix typos in files under test/jdk/java, test/jdk/jdk, test/jdk/jni > > Michael Ernst has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: > > - Reinstate typos in Apache code that is copied into the JDK > - Merge ../jdk-openjdk into typos-typos > - Remove file that was removed upstream > - Fix inconsistency in capitalization > - Undo change in zlip > - Fix typos I agree with everyone who said the PR should be broken to smaller pieces so that it touches code / tests in one or two packages, modules. It would be easier to review, you would need to get an approval from reviewers in a one or two specific areas. At this time, this PR touches files in 11 areas according the number of labels which correspond to a specific mailing list where discussions for the area are held. ------------- PR: https://git.openjdk.org/jdk/pull/10029 From alanb at openjdk.org Wed Oct 5 15:27:05 2022 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 5 Oct 2022 15:27:05 GMT Subject: RFR: 8293810: Remove granting of RuntimePermission("stopThread") from tests Message-ID: This is a test only change to remove the granting of RuntimePermission("stopThread") from the tests. With Thread.stop changed to throw UOE it means there is nothing that requires this permission. ------------- Commit messages: - Initial commit Changes: https://git.openjdk.org/jdk/pull/10577/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10577&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8293810 Stats: 52 lines in 6 files changed: 0 ins; 51 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/10577.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10577/head:pull/10577 PR: https://git.openjdk.org/jdk/pull/10577 From dfuchs at openjdk.org Wed Oct 5 15:51:04 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Wed, 5 Oct 2022 15:51:04 GMT Subject: RFR: 8293810: Remove granting of RuntimePermission("stopThread") from tests In-Reply-To: References: Message-ID: <3kAWQXlRXyCRqCR-ocy_VaXCQls5ozpMLjGj0-Z4kH0=.674a358d-a544-449c-8d9d-b0a60a2f5a40@github.com> On Wed, 5 Oct 2022 15:01:15 GMT, Alan Bateman wrote: > This is a test only change to remove the granting of RuntimePermission("stopThread") from the tests. With Thread.stop changed to throw UOE it means there is nothing that requires this permission. Changes to java.net and javax.management look fine to me. ------------- Marked as reviewed by dfuchs (Reviewer). PR: https://git.openjdk.org/jdk/pull/10577 From mullan at openjdk.org Wed Oct 5 16:35:19 2022 From: mullan at openjdk.org (Sean Mullan) Date: Wed, 5 Oct 2022 16:35:19 GMT Subject: RFR: 8293810: Remove granting of RuntimePermission("stopThread") from tests In-Reply-To: References: Message-ID: On Wed, 5 Oct 2022 15:01:15 GMT, Alan Bateman wrote: > This is a test only change to remove the granting of RuntimePermission("stopThread") from the tests. With Thread.stop changed to throw UOE it means there is nothing that requires this permission. Marked as reviewed by mullan (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/10577 From bpb at openjdk.org Wed Oct 5 17:31:22 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 5 Oct 2022 17:31:22 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Mon, 3 Oct 2022 11:36:48 GMT, Alan Bateman wrote: > My primary concern with this approach is that it doesn't give a lot of advantage over `PollingWatchService` when you need to watch for changes in a very deep hierarchy of directories Could the upcall be replaced for example with a `socketpair` where the native code writes the event data to one end, and the Java code polls the other end and reads the event data? ------------- PR: https://git.openjdk.org/jdk/pull/10140 From mchung at openjdk.org Wed Oct 5 18:42:25 2022 From: mchung at openjdk.org (Mandy Chung) Date: Wed, 5 Oct 2022 18:42:25 GMT Subject: RFR: 8293810: Remove granting of RuntimePermission("stopThread") from tests In-Reply-To: References: Message-ID: <83iSGxB9O0_LEGHtSUf0AsrmJohOpOfVgI8ZpCs-NxQ=.1bc56ba2-eecd-492a-8d58-d2fed72a4f76@github.com> On Wed, 5 Oct 2022 15:01:15 GMT, Alan Bateman wrote: > This is a test only change to remove the granting of RuntimePermission("stopThread") from the tests. With Thread.stop changed to throw UOE it means there is nothing that requires this permission. LGTM ------------- Marked as reviewed by mchung (Reviewer). PR: https://git.openjdk.org/jdk/pull/10577 From bpb at openjdk.org Thu Oct 6 00:02:39 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 6 Oct 2022 00:02:39 GMT Subject: RFR: 8294717: (bf) DirectByteBuffer constructor will leak if allocating Deallocator or Cleaner fails with OOME Message-ID: <2ee_8Iso3nVSH7BG5pfoyOkBFWzaNYH1g8iK8A-atN4=.0b8437fe-0ebf-43a7-9d24-85061307b10e@github.com> Catch any `Throwable` from the `Deallocator` constructor or `Cleaner::create` and free memory to prevent a leak. ------------- Commit messages: - 8294717: (bf) DirectByteBuffer constructor will leak if allocating Deallocator or Cleaner fails with OOME Changes: https://git.openjdk.org/jdk/pull/10588/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10588&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8294717 Stats: 7 lines in 1 file changed: 6 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/10588.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10588/head:pull/10588 PR: https://git.openjdk.org/jdk/pull/10588 From mkartashev at openjdk.org Thu Oct 6 06:44:23 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Thu, 6 Oct 2022 06:44:23 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) In-Reply-To: References: Message-ID: On Wed, 5 Oct 2022 17:27:42 GMT, Brian Burkhalter wrote: > Could the upcall be replaced for example with a socketpair Yes, this should be possible; I think I saw something of that sort implemented in the Linux watch service. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From alanb at openjdk.org Thu Oct 6 09:02:22 2022 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 6 Oct 2022 09:02:22 GMT Subject: RFR: 8294717: (bf) DirectByteBuffer constructor will leak if allocating Deallocator or Cleaner fails with OOME In-Reply-To: <2ee_8Iso3nVSH7BG5pfoyOkBFWzaNYH1g8iK8A-atN4=.0b8437fe-0ebf-43a7-9d24-85061307b10e@github.com> References: <2ee_8Iso3nVSH7BG5pfoyOkBFWzaNYH1g8iK8A-atN4=.0b8437fe-0ebf-43a7-9d24-85061307b10e@github.com> Message-ID: On Wed, 5 Oct 2022 23:54:47 GMT, Brian Burkhalter wrote: > Catch any `Throwable` from the `Deallocator` constructor or `Cleaner::create` and free memory to prevent a leak. src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template line 148: > 146: } catch (Throwable t) { > 147: // Prevent leak if the Deallocator or Cleaner fail for any reason > 148: UNSAFE.freeMemory(base); Bits.unreserveMemory is also needed here. ------------- PR: https://git.openjdk.org/jdk/pull/10588 From chegar at openjdk.org Thu Oct 6 11:46:18 2022 From: chegar at openjdk.org (Chris Hegarty) Date: Thu, 6 Oct 2022 11:46:18 GMT Subject: RFR: 8293810: Remove granting of RuntimePermission("stopThread") from tests In-Reply-To: References: Message-ID: On Wed, 5 Oct 2022 15:01:15 GMT, Alan Bateman wrote: > This is a test only change to remove the granting of RuntimePermission("stopThread") from the tests. With Thread.stop changed to throw UOE it means there is nothing that requires this permission. LGTM ------------- Marked as reviewed by chegar (Reviewer). PR: https://git.openjdk.org/jdk/pull/10577 From prappo at openjdk.org Thu Oct 6 14:35:20 2022 From: prappo at openjdk.org (Pavel Rappo) Date: Thu, 6 Oct 2022 14:35:20 GMT Subject: RFR: 8294377: Prepare to stop auto-inheriting documentation for subclasses of exceptions whose documentation is inherited [v2] In-Reply-To: References: Message-ID: On Fri, 30 Sep 2022 19:15:17 GMT, Phil Race wrote: > If the docs end up the same as you say that is fine by me since for the two ImageIO classes that seems to be what we'd want. Since this change does not affect HTML pages for java.desktop, I take it as "approved". > But inheritDoc behaviour is still "surprising". I've never been sure I understood it and now I'm just sure I don't. > > 1. The two ImageIO methods don't have @OverRide or _anything_ and yet javadoc infers > they'd want to inherit the docs from the interface .. clever javadoc .. I guess I can see > that some doc is better than none so this is OK > 2. When it does inherit I'd expected that it copies the EXACT and ENTIRE javadoc > from the super-interface, which for your change to work can't be all its doing. > You've added JUST > /** > > * @throws SomeExceptionSubType blah-blah > */ > > and yet javadoc somehow figures out this is to be ADDED to the super-type doc for the method and not replace it .. ??? > > 3. What the old code was doing seems to me to be "natural" to the extent any of > this does and the fix you are preparing would add to the surprising behaviours .. Let me try to clarify exception documentation inheritance for you. A method **never** inherits exception documentation unless that method explicitly requests to do so. A method that wants to inherit documentation for a particular exception has two options: use a doc comment or use a method declaration. Let's see how those options work. Suppose, a method wants to inherit documentation for an exception `X`. To inherit documentation through a doc comment, the method adds the following exception tag (`@throws` or `@exception`) to that method's doc comment: ? @throws X {@inheritDoc} To inherit documentation through a method declaration, the method lists `X` in that method's `throws` clause: ? throws ..., X, ... If a method chooses neither of those options, then that method won't inherit exception documentation for `X` (assuming that exception documentation for `X` exists in the supermethod). Here's an informal, simplified, but conceptually correct version of the algorithm that javadoc uses to document exceptions thrown by a method: Step 1. For each exception tag, do the following. If an exception tag does not have `{@inheritDoc}` in its description, add an entry for the exception that this tag describes to the "Throws:" section. Otherwise, look for corresponding documentation in the supermethod, to which apply this step (Step 1) recursively. Step 2. For each exception from the `throws` clause, do the following. If an exception has not been documented on the previous step, document it using corresponding documentation in the supermethod, to which apply this algorithm (both steps in order) recursively. A few notes on the algorithm: * Exception tags are examined prior to the `throws` clause. This allows a method to **override** documentation that could be otherwise inherited from the supermethod: if the method provides exception tags for a particular exception, then documentation for that exception will be found on Step 1 and, hence, won't be looked for in the supermethod on Step 2. ? @throws X * If a method wants to **add** documentation for a particular exception, rather than **override** it, the method should both add documentation and inherit it using tags: ? @throws X ? @throws X {@inheritDoc} The above model might explain a common **misconception** according to which methods inherit documentation for checked exceptions and do not inherit it for unchecked exceptions. In reality, javadoc treats all exceptions equally. It's just that if a method throws a checked exception, that exception (or its superclass) must be listed in that method's `throws` clause. Now, if such an exception is not documented by that method but documented by the supermethod, that exception documentation is inherited. That gives a false impression that the documentation is inherited because the exception is checked. In fact, the documentation would still be inherited if that exception, listed in the `throws` clause, were unchecked. The above is how it has been working (barring bugs) since documentation comment inheritance appeared in its current form. Implicit inheritance (filling in missing text) appeared in JDK 1.3, explicit inheritance (`{@inheritDoc}`) appeared in JDK 1.4, auto-inheriting documentation for subclasses of exceptions whose documentation is inherited (JDK-4947455) appeared in JDK 5. Back to this PR change in `java.desktop`. `ImageInputStreamImpl.readBoolean` inherits `EOFException` documentation not because that method doesn't have a doc comment of its own and, thus, inherits "the exact and entire" doc comment from its supermethod (`ImageInputStream.readBoolean`). It's because that when the algorithm reaches Step 2 and tries to find documentation for `IOException` (because it is listed in the `throws` clause), JDK-4947455 kicks in. And JDK-4947455 says that if a method inherits documentation for a particular exception, it should also inherit documentation for that exception's subclasses. So, javadoc goes ahead and inherits documentation for `IOException`, because it's a direct match, and for `EOFException`, because it's a subclass of `IOException`. To inherit `EOFException` documentation after JDK-4947455 has been reverted and, thus, the subclassing :magic_wand: has gone, `ImageInputStreamImpl.readBoolean` has two options: * list `EOFException` in the `throws` clause * `@throws EOFException {@inheritDoc}` Thanks to the current implementation of documentation inheritance, any of the above can be done **before** JDK-4947455 is reverted. For now doing so just adds a redundant but harmless inheritance route, which will become the only route in time. For this PR to be less disruptive and qualify as "noreg-doc" I chose to add an exception tag rather than amend the `throws` clause. I hope that my reply clarifies the matter. Documentation inheritance can be surprising. We're trying hard to capture its model and simplify it where possible. We are constantly improving the Documentation Comment Specification for the Standard Doclet[^1] (spec), but since documentation inheritance is a popular topic which is often misunderstood, along with improving the spec in this area, we might also provide a less formal document dedicated to documentation inheritance. [^1]: https://docs.oracle.com/en/java/javase/19/docs/specs/javadoc/doc-comment-spec.html ------------- PR: https://git.openjdk.org/jdk/pull/10449 From jjg at openjdk.org Thu Oct 6 15:05:56 2022 From: jjg at openjdk.org (Jonathan Gibbons) Date: Thu, 6 Oct 2022 15:05:56 GMT Subject: RFR: 8294377: Prepare to stop auto-inheriting documentation for subclasses of exceptions whose documentation is inherited [v2] In-Reply-To: References: Message-ID: On Tue, 27 Sep 2022 12:14:23 GMT, Pavel Rappo wrote: >> This adds exception documentation to JDK methods that would otherwise lose that documentation once JDK-8287796 is integrated. While adding this exception documentation now does not change [^1] the JDK API Documentation, it will automatically counterbalance the effect that JDK-8287796 will have. >> >> JDK-8287796 seeks to remove an old, undocumented, and esoteric javadoc feature that cannot be suppressed [^2]. The feature is so little known about that IDEs either do not implement it correctly or do not implement it at all, thus rendering documentation differently from how the javadoc tool renders it. >> >> That feature was introduced in JDK-4947455 and manifests as follows. If a method inherits documentation for an exception, along with that documentation the method automatically inherits documentation for all exceptions that are subclasses of that former exception and are documented in an overridden method. >> >> To illustrate that behavior, consider the following example. A method `Subtype.m` inherits documentation for `SuperException`, while the overridden method `Supertype.m` documents `SuperException`, `SubExceptionA` and `SubExceptionB`, where the latter two exceptions are subclasses of the former exception: >> >> public class Subtype extends Supertype { >> >> @Override >> public void m() throws SuperException { >> ... >> >> public class Supertype { >> >> /** >> * @throws SuperException general problem >> * @throws SubExceptionA specific problem A >> * @throws SubExceptionB specific problem B >> */ >> public void m() throws SuperException { >> ... >> >> public class SuperException extends Exception { >> ... >> >> public class SubExceptionA extends SuperException { >> ... >> >> public class SubExceptionB extends SuperException { >> ... >> >> For the above example, the API Documentation for `Subtype.m` will contain documentation for all three exceptions; the page fragment for `Subtype.m` in "Method Details" will look like this: >> >> public void m() >> throws SuperException >> >> Overrides: >> m in class Supertype >> Throws: >> SuperException - general problem >> SubExceptionA - specific problem A >> SubExceptionB - specific problem B >> >> It works for checked and unchecked exceptions, for methods in classes and interfaces. >> >> >> If it's the first time you hear about that behavior and this change affects your area, it's a good opportunity to reflect on whether the exception documentation this change adds is really needed or you were simply unaware of the fact that it was automatically added before. If exception documentation is not needed, please comment on this PR. Otherwise, consider approving it. >> >> Keep in mind that if some exception documentation is not needed, **leaving it out** from this PR might require a CSR. >> >> >> [^1]: Aside from insignificant changes on class-use and index pages. There's one relatively significant change. This change is in jdk.jshell, where some methods disappeared from "Methods declared in ..." section and other irregularities. We are aware of this and have filed JDK-8291803 to fix the root cause. >> [^2]: In reality, the feature can be suppressed, but it looks like a bug rather than intentional design. If we legitimize the feature and its suppression behavior, the model for exception documentation inheritance might become much more complicated. > > Pavel Rappo has updated the pull request incrementally with one additional commit since the last revision: > > revert an extraneous change to jdk.javadoc I'm marking the change as approved, in terms of the cumulative efforts of those who have commented. I've browsed all the changes and read the jshell files in more detail, since no once else has so far. I understand the reason for the change, and generally approve of undoing the magic of the old feature in JDK-8287796. That being said, the proposed state is "not great", and feels like the interim stage of "one step back, to take two steps forward", even though I do not have a good sense at this time of what that future direction might be. So, looks OK for now, but I hope we revisit this issue again sometime. Thanks, in general, for taking on this topic. ------------- Marked as reviewed by jjg (Reviewer). PR: https://git.openjdk.org/jdk/pull/10449 From bpb at openjdk.org Thu Oct 6 15:19:27 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 6 Oct 2022 15:19:27 GMT Subject: RFR: 8294717: (bf) DirectByteBuffer constructor will leak if allocating Deallocator or Cleaner fails with OOME [v2] In-Reply-To: <2ee_8Iso3nVSH7BG5pfoyOkBFWzaNYH1g8iK8A-atN4=.0b8437fe-0ebf-43a7-9d24-85061307b10e@github.com> References: <2ee_8Iso3nVSH7BG5pfoyOkBFWzaNYH1g8iK8A-atN4=.0b8437fe-0ebf-43a7-9d24-85061307b10e@github.com> Message-ID: <2UecYZEsQN7Dt5N5OfmSq-ooLkfMUNaa6gtBD-PwbXU=.2b640c1f-7302-4cf2-b3a2-16c1fbc75c9a@github.com> > Catch any `Throwable` from the `Deallocator` constructor or `Cleaner::create` and free memory to prevent a leak. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8294717: Add Bits.unreserveMemory to leak prevention ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10588/files - new: https://git.openjdk.org/jdk/pull/10588/files/03b92348..78e83b4f Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10588&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10588&range=00-01 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/10588.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10588/head:pull/10588 PR: https://git.openjdk.org/jdk/pull/10588 From bpb at openjdk.org Thu Oct 6 15:19:28 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 6 Oct 2022 15:19:28 GMT Subject: RFR: 8294717: (bf) DirectByteBuffer constructor will leak if allocating Deallocator or Cleaner fails with OOME [v2] In-Reply-To: References: <2ee_8Iso3nVSH7BG5pfoyOkBFWzaNYH1g8iK8A-atN4=.0b8437fe-0ebf-43a7-9d24-85061307b10e@github.com> Message-ID: <7adMZUhPXpi97KFGN4sLiOoxxJ3Y0XYOumUms61b__Q=.aa40a39b-6f11-484c-aab7-1f87d25ecb8d@github.com> On Thu, 6 Oct 2022 08:58:50 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8294717: Add Bits.unreserveMemory to leak prevention > > src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template line 148: > >> 146: } catch (Throwable t) { >> 147: // Prevent leak if the Deallocator or Cleaner fail for any reason >> 148: UNSAFE.freeMemory(base); > > Bits.unreserveMemory is also needed here. Oh, right. Fixed in 78e83b4ffd27d4d57a9fa82fd47c197854ac7ac3. ------------- PR: https://git.openjdk.org/jdk/pull/10588 From bpb at openjdk.org Thu Oct 6 15:47:03 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 6 Oct 2022 15:47:03 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v7] In-Reply-To: References: Message-ID: On Fri, 30 Sep 2022 05:17:43 GMT, Maxim Kartashev wrote: >> This is an implementation of `WatchService` based on File System Events API that is capable of generating events whenever a change occurs in an interesting directory or underneath it. Since the API naturally supports "recursive" watch, the `FILE_TREE` is supported by the watch service. >> >> Some things of note: >> * There's one "service" thread per `WatchService` instance that is inactive unless changes occur in the watched directory. The changes are grouped by introducing a time delay between when they occurred and when they are reported, which is controlled by the sensitivity modifier of the watch service. >> * Since FSEvents API reports directories only, the watch service keeps a snapshot (hierarchical if necessary) of the files in the directory being watched. The snapshot gets updated when an event in that directory or underneath it gets delivered. File changes are detected by comparing "last modified" time with a millisecond precision (`BasicFileAttributes.lastModifiedTime()`). >> * There is a slight complication with the move of an entire directory hierarchy: FSEvents API only reports about the containing directory of that move and not about any of the directories actually moved. There's a separate test for that (`Move.java`). >> * The code is careful not to do any I/O (such as reading the contents of a directory or attributes of a file) unless unavoidable. Any deviation from this line should be considered a bug (of, arguably, low priority). >> * The native part consists mostly of API wrappers with one exception of the callback function invoked by the system to report the events that occurred. The sole task of the function is to convert from C strings to Java strings and pass the array of affected directories to Java code. This can be rewritten if desired to make the code more future-proof. >> >> This commit leaves `PollingWatchService` unused. I'm not sure if I should/can do anything about it. Any advice is welcomed. >> >> ### Testing >> >> * Tested by running `test/jdk/java/nio/file` and `test/jdk/jdk/nio` on MacOS 10.15.7 (x64) and `test/jdk/java/nio/` plus `test/jdk/jdk/nio` on MacOS 12.5.1 (aarch64). >> * Also verified that new tests pass on Linux and Windows. >> * This code (albeit in a slightly modified form) has been in use at JetBrains for around half a year and a few bugs have been found and fixed during that time period. > > Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: > > Dropped __unused attributes from used parameters On Oct 5, 2022, at 11:42 PM, Maxim Kartash?v ***@***.******@***.***>> wrote: Could the upcall be replaced for example with a socketpair Yes, this should be possible; I think I saw something of that sort implemented in the Linux watch service. Yes, it simultaneously polls the inotify descriptor and one end of a socketpair, the latter being used for wakeup. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From bpb at openjdk.org Thu Oct 6 20:05:24 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 6 Oct 2022 20:05:24 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v18] In-Reply-To: <5kWk4eK9PtFqPgTJjyymqxyiTDRC-UKdte3vxdhwA30=.49440d78-db24-4333-9c59-18ee9e82fef7@github.com> References: <5kWk4eK9PtFqPgTJjyymqxyiTDRC-UKdte3vxdhwA30=.49440d78-db24-4333-9c59-18ee9e82fef7@github.com> Message-ID: On Mon, 3 Oct 2022 20:05:13 GMT, Mark Reinhold wrote: > Is that because those systems are interpreting the `txt` suffix, or is it because they always open files of unknown type with the platform?s default text editor? Looks like a bit of each. Files named `.xy74j0` and `...xy74j0` with identical text content are opened by the default text editor on Linux. On macOS, `.xy74j0` is also opened by the default text editor but clicking on `...xy74j0` results in a popup which states `There is no application set to open the document ?...xy74j0?.` On Windows, a popup appears in both cases asking that an appropriate app be selected to open the file. So it looks like on Linux a file of unknown type might always be opened by the default text editor, but based on this example and the previous `...txt` example, on macOS and Windows there is some inference of the file type from the file name or extension, however the latter is defined. ------------- PR: https://git.openjdk.org/jdk/pull/8066 From alanb at openjdk.org Fri Oct 7 06:19:32 2022 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 7 Oct 2022 06:19:32 GMT Subject: Integrated: 8293810: Remove granting of RuntimePermission("stopThread") from tests In-Reply-To: References: Message-ID: On Wed, 5 Oct 2022 15:01:15 GMT, Alan Bateman wrote: > This is a test only change to remove the granting of RuntimePermission("stopThread") from the tests. With Thread.stop changed to throw UOE it means there is nothing that requires this permission. This pull request has now been integrated. Changeset: 0ad6803a Author: Alan Bateman URL: https://git.openjdk.org/jdk/commit/0ad6803ac2bba063d15ce8284a09da36b4cced81 Stats: 52 lines in 6 files changed: 0 ins; 51 del; 1 mod 8293810: Remove granting of RuntimePermission("stopThread") from tests Reviewed-by: dfuchs, mullan, mchung, chegar ------------- PR: https://git.openjdk.org/jdk/pull/10577 From alanb at openjdk.org Fri Oct 7 06:20:11 2022 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 7 Oct 2022 06:20:11 GMT Subject: RFR: 8294717: (bf) DirectByteBuffer constructor will leak if allocating Deallocator or Cleaner fails with OOME [v2] In-Reply-To: <2UecYZEsQN7Dt5N5OfmSq-ooLkfMUNaa6gtBD-PwbXU=.2b640c1f-7302-4cf2-b3a2-16c1fbc75c9a@github.com> References: <2ee_8Iso3nVSH7BG5pfoyOkBFWzaNYH1g8iK8A-atN4=.0b8437fe-0ebf-43a7-9d24-85061307b10e@github.com> <2UecYZEsQN7Dt5N5OfmSq-ooLkfMUNaa6gtBD-PwbXU=.2b640c1f-7302-4cf2-b3a2-16c1fbc75c9a@github.com> Message-ID: On Thu, 6 Oct 2022 15:19:27 GMT, Brian Burkhalter wrote: >> Catch any `Throwable` from the `Deallocator` constructor or `Cleaner::create` and free memory to prevent a leak. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8294717: Add Bits.unreserveMemory to leak prevention Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/10588 From mbaesken at openjdk.org Fri Oct 7 07:37:21 2022 From: mbaesken at openjdk.org (Matthias Baesken) Date: Fri, 7 Oct 2022 07:37:21 GMT Subject: RFR: JDK-8294901: remove pre-VS2017 checks in Windows related coding Message-ID: <0kJCOAI9EAmXB8M7Lqe14-a5sU5V65_KmMcH5NX0eYw=.76a92302-e8c8-44e3-b76a-2ca543144ddf@github.com> After "8293162: Drop support for VS2017" limited current support to VS2019 and VS2022 it is most likely safe to remove various checks/workarounds related to older VS compilers like VS2015 or VS2013. ------------- Commit messages: - JDK-8294901 Changes: https://git.openjdk.org/jdk/pull/10600/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10600&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8294901 Stats: 38 lines in 4 files changed: 0 ins; 37 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/10600.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10600/head:pull/10600 PR: https://git.openjdk.org/jdk/pull/10600 From alanb at openjdk.org Fri Oct 7 12:53:30 2022 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 7 Oct 2022 12:53:30 GMT Subject: RFR: 8294321: Fix typos in files under test/jdk/java, test/jdk/jdk, test/jdk/jni [v2] In-Reply-To: References: Message-ID: <-0yo8KceENmJ48YPNoHCUkx_iEWpIE0mPJn_-BkjbWY=.76a8dcb8-f43a-4c8b-8912-43c7225c183d@github.com> On Mon, 26 Sep 2022 16:51:36 GMT, Michael Ernst wrote: >> 8294321: Fix typos in files under test/jdk/java, test/jdk/jdk, test/jdk/jni > > Michael Ernst has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: > > - Reinstate typos in Apache code that is copied into the JDK > - Merge ../jdk-openjdk into typos-typos > - Remove file that was removed upstream > - Fix inconsistency in capitalization > - Undo change in zlip > - Fix typos src/java.se/share/data/jdwp/jdwp.spec line 101: > 99: "platform thread " > 100: "in the target VM. This includes platform threads created with the Thread " > 101: "API and all native threads attached to the target VM with JNI code." The spec for the JDWP AllThreads command was significantly reworded in Java 19 so this is where this typo crept in. We have JDK-8294672 tracking it to fix for Java 20, maybe you should take it? ------------- PR: https://git.openjdk.org/jdk/pull/10029 From dholmes at openjdk.org Fri Oct 7 13:05:36 2022 From: dholmes at openjdk.org (David Holmes) Date: Fri, 7 Oct 2022 13:05:36 GMT Subject: RFR: JDK-8294901: remove pre-VS2017 checks in Windows related coding In-Reply-To: <0kJCOAI9EAmXB8M7Lqe14-a5sU5V65_KmMcH5NX0eYw=.76a92302-e8c8-44e3-b76a-2ca543144ddf@github.com> References: <0kJCOAI9EAmXB8M7Lqe14-a5sU5V65_KmMcH5NX0eYw=.76a92302-e8c8-44e3-b76a-2ca543144ddf@github.com> Message-ID: On Fri, 7 Oct 2022 07:29:16 GMT, Matthias Baesken wrote: > After "8293162: Drop support for VS2017" limited current support to VS2019 and VS2022 it is most likely safe to remove various checks/workarounds related to older VS compilers like VS2015 or VS2013. Seems reasonable. Thanks. ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.org/jdk/pull/10600 From alanb at openjdk.org Fri Oct 7 13:19:47 2022 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 7 Oct 2022 13:19:47 GMT Subject: RFR: JDK-8294901: remove pre-VS2017 checks in Windows related coding In-Reply-To: <0kJCOAI9EAmXB8M7Lqe14-a5sU5V65_KmMcH5NX0eYw=.76a92302-e8c8-44e3-b76a-2ca543144ddf@github.com> References: <0kJCOAI9EAmXB8M7Lqe14-a5sU5V65_KmMcH5NX0eYw=.76a92302-e8c8-44e3-b76a-2ca543144ddf@github.com> Message-ID: <9wD8YwAnnj-KWVbiz81wfwgzhNi6UsUqA206T416Pgk=.88b21a0a-dc67-4384-aff7-754d6b978bd2@github.com> On Fri, 7 Oct 2022 07:29:16 GMT, Matthias Baesken wrote: > After "8293162: Drop support for VS2017" limited current support to VS2019 and VS2022 it is most likely safe to remove various checks/workarounds related to older VS compilers like VS2015 or VS2013. src/java.base/windows/native/libnio/ch/wepoll.c line 865: > 863: #define inline __inline > 864: #endif > 865: This is 3rd party code so best to leave it out of this change. ------------- PR: https://git.openjdk.org/jdk/pull/10600 From mbaesken at openjdk.org Fri Oct 7 13:29:48 2022 From: mbaesken at openjdk.org (Matthias Baesken) Date: Fri, 7 Oct 2022 13:29:48 GMT Subject: RFR: JDK-8294901: remove pre-VS2017 checks in Windows related coding [v2] In-Reply-To: <0kJCOAI9EAmXB8M7Lqe14-a5sU5V65_KmMcH5NX0eYw=.76a92302-e8c8-44e3-b76a-2ca543144ddf@github.com> References: <0kJCOAI9EAmXB8M7Lqe14-a5sU5V65_KmMcH5NX0eYw=.76a92302-e8c8-44e3-b76a-2ca543144ddf@github.com> Message-ID: > After "8293162: Drop support for VS2017" limited current support to VS2019 and VS2022 it is most likely safe to remove various checks/workarounds related to older VS compilers like VS2015 or VS2013. Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: wepoll.c is 3rd party code, do not change it ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10600/files - new: https://git.openjdk.org/jdk/pull/10600/files/75d999ec..4dcdd911 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10600&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10600&range=00-01 Stats: 5 lines in 1 file changed: 5 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/10600.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10600/head:pull/10600 PR: https://git.openjdk.org/jdk/pull/10600 From mbaesken at openjdk.org Fri Oct 7 13:29:49 2022 From: mbaesken at openjdk.org (Matthias Baesken) Date: Fri, 7 Oct 2022 13:29:49 GMT Subject: RFR: JDK-8294901: remove pre-VS2017 checks in Windows related coding [v2] In-Reply-To: <9wD8YwAnnj-KWVbiz81wfwgzhNi6UsUqA206T416Pgk=.88b21a0a-dc67-4384-aff7-754d6b978bd2@github.com> References: <0kJCOAI9EAmXB8M7Lqe14-a5sU5V65_KmMcH5NX0eYw=.76a92302-e8c8-44e3-b76a-2ca543144ddf@github.com> <9wD8YwAnnj-KWVbiz81wfwgzhNi6UsUqA206T416Pgk=.88b21a0a-dc67-4384-aff7-754d6b978bd2@github.com> Message-ID: <2iqhUZ9LdbbteDx4y8RXCwxoeJPVB5aqiaaQAX2227M=.28830a9b-30b3-4e1d-866b-761440371e98@github.com> On Fri, 7 Oct 2022 13:16:27 GMT, Alan Bateman wrote: >> Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: >> >> wepoll.c is 3rd party code, do not change it > > src/java.base/windows/native/libnio/ch/wepoll.c line 865: > >> 863: #define inline __inline >> 864: #endif >> 865: > > This is 3rd party code so best to leave it out of this change. I agree, let's leave 3rd party code alone. I adjusted wepoll.c and kept the _MSC_VER check. ------------- PR: https://git.openjdk.org/jdk/pull/10600 From mbaesken at openjdk.org Fri Oct 7 14:04:51 2022 From: mbaesken at openjdk.org (Matthias Baesken) Date: Fri, 7 Oct 2022 14:04:51 GMT Subject: RFR: JDK-8294901: remove pre-VS2017 checks in Windows related coding [v2] In-Reply-To: References: <0kJCOAI9EAmXB8M7Lqe14-a5sU5V65_KmMcH5NX0eYw=.76a92302-e8c8-44e3-b76a-2ca543144ddf@github.com> Message-ID: On Fri, 7 Oct 2022 13:29:48 GMT, Matthias Baesken wrote: >> After "8293162: Drop support for VS2017" limited current support to VS2019 and VS2022 it is most likely safe to remove various checks/workarounds related to older VS compilers like VS2015 or VS2013. > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > wepoll.c is 3rd party code, do not change it btw while looking at VS-related references, the build docu seems to be a little outdated too https://github.com/openjdk/jdk/blob/master/doc/building.md states "Windows XP is not a supported platform, but all newer Windows should be able to build the JDK." That's most likely not true any more , the minimum requirements for VS2017/VS2019 are https://learn.microsoft.com/en-us/visualstudio/releases/2019/system-requirements https://learn.microsoft.com/en-us/visualstudio/releases/2017/vs2017-system-requirements-vs so it is Windows7 SP1/Windows Server 2016 minimum . (releases older than that but newer than XP like Vista or Server 2008 are not supported any more) ------------- PR: https://git.openjdk.org/jdk/pull/10600 From mdoerr at openjdk.org Fri Oct 7 14:43:19 2022 From: mdoerr at openjdk.org (Martin Doerr) Date: Fri, 7 Oct 2022 14:43:19 GMT Subject: RFR: JDK-8294901: remove pre-VS2017 checks in Windows related coding [v2] In-Reply-To: References: <0kJCOAI9EAmXB8M7Lqe14-a5sU5V65_KmMcH5NX0eYw=.76a92302-e8c8-44e3-b76a-2ca543144ddf@github.com> Message-ID: On Fri, 7 Oct 2022 13:29:48 GMT, Matthias Baesken wrote: >> After "8293162: Drop support for VS2017" limited current support to VS2019 and VS2022 it is most likely safe to remove various checks/workarounds related to older VS compilers like VS2015 or VS2013. > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > wepoll.c is 3rd party code, do not change it This version LGTM. ------------- Marked as reviewed by mdoerr (Reviewer). PR: https://git.openjdk.org/jdk/pull/10600 From bpb at openjdk.org Fri Oct 7 17:46:58 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 7 Oct 2022 17:46:58 GMT Subject: Integrated: 8294717: (bf) DirectByteBuffer constructor will leak if allocating Deallocator or Cleaner fails with OOME In-Reply-To: <2ee_8Iso3nVSH7BG5pfoyOkBFWzaNYH1g8iK8A-atN4=.0b8437fe-0ebf-43a7-9d24-85061307b10e@github.com> References: <2ee_8Iso3nVSH7BG5pfoyOkBFWzaNYH1g8iK8A-atN4=.0b8437fe-0ebf-43a7-9d24-85061307b10e@github.com> Message-ID: On Wed, 5 Oct 2022 23:54:47 GMT, Brian Burkhalter wrote: > Catch any `Throwable` from the `Deallocator` constructor or `Cleaner::create` and free memory to prevent a leak. This pull request has now been integrated. Changeset: 4cbac40d Author: Brian Burkhalter URL: https://git.openjdk.org/jdk/commit/4cbac40de956974760cf54183b3ba29f0b5ec331 Stats: 8 lines in 1 file changed: 7 ins; 0 del; 1 mod 8294717: (bf) DirectByteBuffer constructor will leak if allocating Deallocator or Cleaner fails with OOME Reviewed-by: alanb ------------- PR: https://git.openjdk.org/jdk/pull/10588 From kbarrett at openjdk.org Fri Oct 7 17:53:20 2022 From: kbarrett at openjdk.org (Kim Barrett) Date: Fri, 7 Oct 2022 17:53:20 GMT Subject: RFR: JDK-8294901: remove pre-VS2017 checks in Windows related coding [v2] In-Reply-To: References: <0kJCOAI9EAmXB8M7Lqe14-a5sU5V65_KmMcH5NX0eYw=.76a92302-e8c8-44e3-b76a-2ca543144ddf@github.com> Message-ID: On Fri, 7 Oct 2022 13:29:48 GMT, Matthias Baesken wrote: >> After "8293162: Drop support for VS2017" limited current support to VS2019 and VS2022 it is most likely safe to remove various checks/workarounds related to older VS compilers like VS2015 or VS2013. > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > wepoll.c is 3rd party code, do not change it Looks good. ------------- Marked as reviewed by kbarrett (Reviewer). PR: https://git.openjdk.org/jdk/pull/10600 From alanb at openjdk.org Sun Oct 9 14:11:57 2022 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 9 Oct 2022 14:11:57 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v7] In-Reply-To: References: Message-ID: On Fri, 30 Sep 2022 05:17:43 GMT, Maxim Kartashev wrote: >> This is an implementation of `WatchService` based on File System Events API that is capable of generating events whenever a change occurs in an interesting directory or underneath it. Since the API naturally supports "recursive" watch, the `FILE_TREE` is supported by the watch service. >> >> Some things of note: >> * There's one "service" thread per `WatchService` instance that is inactive unless changes occur in the watched directory. The changes are grouped by introducing a time delay between when they occurred and when they are reported, which is controlled by the sensitivity modifier of the watch service. >> * Since FSEvents API reports directories only, the watch service keeps a snapshot (hierarchical if necessary) of the files in the directory being watched. The snapshot gets updated when an event in that directory or underneath it gets delivered. File changes are detected by comparing "last modified" time with a millisecond precision (`BasicFileAttributes.lastModifiedTime()`). >> * There is a slight complication with the move of an entire directory hierarchy: FSEvents API only reports about the containing directory of that move and not about any of the directories actually moved. There's a separate test for that (`Move.java`). >> * The code is careful not to do any I/O (such as reading the contents of a directory or attributes of a file) unless unavoidable. Any deviation from this line should be considered a bug (of, arguably, low priority). >> * The native part consists mostly of API wrappers with one exception of the callback function invoked by the system to report the events that occurred. The sole task of the function is to convert from C strings to Java strings and pass the array of affected directories to Java code. This can be rewritten if desired to make the code more future-proof. >> >> This commit leaves `PollingWatchService` unused. I'm not sure if I should/can do anything about it. Any advice is welcomed. >> >> ### Testing >> >> * Tested by running `test/jdk/java/nio/file` and `test/jdk/jdk/nio` on MacOS 10.15.7 (x64) and `test/jdk/java/nio/` plus `test/jdk/jdk/nio` on MacOS 12.5.1 (aarch64). >> * Also verified that new tests pass on Linux and Windows. >> * This code (albeit in a slightly modified form) has been in use at JetBrains for around half a year and a few bugs have been found and fixed during that time period. > > Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: > > Dropped __unused attributes from used parameters It's a real shame that it requires use of a CFRunLoop and callbacks but if that is the only facility that it provides then I suppose we have to use it. I see the discussion about using a socketpair but I don't think that is sensible thing to do here. The socketpair that is used elsewhere is for wakeup only, similar to what we do in the Selector implementations. So I think we should move forward with the current approach but probably will need a few rounds of cleanup. src/java.base/share/native/libnio/nio_util.c line 37: > 35: { > 36: JNIEnv *env; > 37: jvm = vm; Would it be possible for MacOSXWatchService to use GetJavaVM is its initializer so that the changes to nio_util.* can be reverted? test/jdk/jdk/nio/zipfs/test.policy line 7: > 5: permission java.util.PropertyPermission "user.dir","read"; > 6: permission java.lang.RuntimePermission "accessUserInformation"; > 7: permission java.lang.RuntimePermission "loadLibrary.nio"; Why it this needed? It suggests there is a problem elsewhere so would be good to see the exception. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From john_platts at hotmail.com Mon Oct 3 12:59:42 2022 From: john_platts at hotmail.com (John Platts) Date: Mon, 3 Oct 2022 12:59:42 +0000 Subject: Bug in the Direct$Type$Buffer$RW$(int cap) constructor in src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template if the allocation of the cleaner fails Message-ID: I have discovered a bug in the Direct$Type$Buffer$RW$(int cap) constructor if the allocation of the cleaner fails as memory will be leaked if an Exception or Error is thrown by the allocation of the Deallocator or the creation of the cleaner. Here is how the allocation bug in the Direct$Type$Buffer$RW$(int cap) constructor can be fixed: private static class Deallocator implements Runnable { private final long address; private final long size; private final int capacity; private int deallocationPerformed; private static final long DEALLOCATION_PERFORMED = UNSAFE.objectFieldOffset(Deallocator.class, "deallocationPerformed"); private Deallocator(long address, long size, int capacity) { assert (address != 0); this.address = address; this.size = size; this.capacity = capacity; } public void run() { if(UNSAFE.getAndSetIntAcquire(this, DEALLOCATION_PERFORMED, 1) == 0) { UNSAFE.freeMemory(address); Bits.unreserveMemory(size, capacity); } } } // Primary constructor // Direct$Type$Buffer$RW$(int cap) { // package-private #if[rw] super(-1, 0, cap, cap); boolean pa = VM.isDirectMemoryPageAligned(); int ps = Bits.pageSize(); long size = Math.max(1L, (long)cap + (pa ? ps : 0)); Bits.reserveMemory(size, cap); long base = 0; Deallocator deallocator = null; try { base = UNSAFE.allocateMemory(size); UNSAFE.setMemory(base, size, (byte) 0); if (pa && (base % ps != 0)) { // Round up to page boundary address = base + ps - (base & (ps - 1)); } else { address = base; } deallocator = new Deallocator(base, size, cap); cleaner = Cleaner.create(this, deallocator); } catch(Throwable ex) { if(deallocator != null) { deallocator.run(); } else { if(base != 0) UNSAFE.freeMemory(base); Bits.unreserveMemory(size, capacity); } throw ex; } att = null; #else[rw] super(cap); this.isReadOnly = true; #end[rw] } The updated version of the Direct$Type$Buffer$RW$(int cap) above will ensure that the memory that was allocated by UNSAFE.allocateMemory(size) will get freed if the creation of the cleaner fails. A deallocationPerformed field is also added to the Deallocator class to make sure that the memory that was allocated by the Direct$Type$Buffer$RW$(int cap) is freed exactly once. A UNSAFE.getAndSetIntAcquire is used to set deallocationPerformed to 1 in the Deallocator.run() method to make sure that the deallocation of memory by Deallocator.run is performed only once. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Sun Oct 9 16:10:46 2022 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 9 Oct 2022 17:10:46 +0100 Subject: Wrong information in the java.nio.ByteBuffer document In-Reply-To: References: Message-ID: On 28/06/2022 20:35, Kerem Saliho?lu wrote: > In the java.nio.ByteBuffer document, there is an example about floats > > ?float ?getFloat() > ?float ?getFloat(int index) > ? void ?putFloat(float f) > ? void ?putFloat(int index, float f) > > which says putFloat methods are void but this is wrong, they > return?ByteBuffer itself. > This mail was stuck, awaiting moderation as it was sent from a mail account that is not subscribed to the list. In any case, thank you for the report, this is an error in the docs. I've created JDK-8295025 [1] to track? it. -Alan [1] https://bugs.openjdk.org/browse/JDK-8295025 From Alan.Bateman at oracle.com Sun Oct 9 16:13:13 2022 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 9 Oct 2022 17:13:13 +0100 Subject: Bug in the Direct$Type$Buffer$RW$(int cap) constructor in src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template if the allocation of the cleaner fails In-Reply-To: References: Message-ID: <1a6f39dc-32d8-38c3-e7f2-dd2be4094a27@oracle.com> On 03/10/2022 13:59, John Platts wrote: > I have discovered a bug in the?Direct$Type$Buffer$RW$(int cap) > constructor if the allocation of the cleaner fails as memory will be > leaked if an Exception or Error is thrown by the allocation of the > Deallocator or the creation of the cleaner. > > This mail was stuck, awaiting moderation as it was sent from a mail account that is not subscribed to the list. Thank you for the report. If allocating of the Cleaner were to fail with OOME or something else then it would leak.? It is tracked by JDK-8294717 [1] and has been fixed in the main line for JDK 20. -Alan [1] https://bugs.openjdk.org/browse/JDK-8294717 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkartashev at openjdk.org Mon Oct 10 09:20:57 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Mon, 10 Oct 2022 09:20:57 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v7] In-Reply-To: References: Message-ID: On Sun, 9 Oct 2022 13:51:51 GMT, Alan Bateman wrote: >> Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: >> >> Dropped __unused attributes from used parameters > > test/jdk/jdk/nio/zipfs/test.policy line 7: > >> 5: permission java.util.PropertyPermission "user.dir","read"; >> 6: permission java.lang.RuntimePermission "accessUserInformation"; >> 7: permission java.lang.RuntimePermission "loadLibrary.nio"; > > Why it this needed? It suggests there is a problem elsewhere so would be good to see the exception. I understand that previously using a `WatchService` didn't require a native library to be loaded, but now it does. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From mkartashev at openjdk.org Mon Oct 10 10:33:38 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Mon, 10 Oct 2022 10:33:38 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v8] In-Reply-To: References: Message-ID: > This is an implementation of `WatchService` based on File System Events API that is capable of generating events whenever a change occurs in an interesting directory or underneath it. Since the API naturally supports "recursive" watch, the `FILE_TREE` is supported by the watch service. > > Some things of note: > * There's one "service" thread per `WatchService` instance that is inactive unless changes occur in the watched directory. The changes are grouped by introducing a time delay between when they occurred and when they are reported, which is controlled by the sensitivity modifier of the watch service. > * Since FSEvents API reports directories only, the watch service keeps a snapshot (hierarchical if necessary) of the files in the directory being watched. The snapshot gets updated when an event in that directory or underneath it gets delivered. File changes are detected by comparing "last modified" time with a millisecond precision (`BasicFileAttributes.lastModifiedTime()`). > * There is a slight complication with the move of an entire directory hierarchy: FSEvents API only reports about the containing directory of that move and not about any of the directories actually moved. There's a separate test for that (`Move.java`). > * The code is careful not to do any I/O (such as reading the contents of a directory or attributes of a file) unless unavoidable. Any deviation from this line should be considered a bug (of, arguably, low priority). > * The native part consists mostly of API wrappers with one exception of the callback function invoked by the system to report the events that occurred. The sole task of the function is to convert from C strings to Java strings and pass the array of affected directories to Java code. This can be rewritten if desired to make the code more future-proof. > > This commit leaves `PollingWatchService` unused. I'm not sure if I should/can do anything about it. Any advice is welcomed. > > ### Testing > > * Tested by running `test/jdk/java/nio/file` and `test/jdk/jdk/nio` on MacOS 10.15.7 (x64) and `test/jdk/java/nio/` plus `test/jdk/jdk/nio` on MacOS 12.5.1 (aarch64). > * Also verified that new tests pass on Linux and Windows. > * This code (albeit in a slightly modified form) has been in use at JetBrains for around half a year and a few bugs have been found and fixed during that time period. Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: Don't use JavaVM exported from nio_util.c ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10140/files - new: https://git.openjdk.org/jdk/pull/10140/files/8d5c1962..b3d1604b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10140&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10140&range=06-07 Stats: 15 lines in 3 files changed: 6 ins; 6 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/10140.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10140/head:pull/10140 PR: https://git.openjdk.org/jdk/pull/10140 From mkartashev at openjdk.org Mon Oct 10 10:33:43 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Mon, 10 Oct 2022 10:33:43 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v7] In-Reply-To: References: Message-ID: On Sun, 9 Oct 2022 13:50:53 GMT, Alan Bateman wrote: >> Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: >> >> Dropped __unused attributes from used parameters > > src/java.base/share/native/libnio/nio_util.c line 37: > >> 35: { >> 36: JNIEnv *env; >> 37: jvm = vm; > > Would it be possible for MacOSXWatchService to use GetJavaVM is its initializer so that the changes to nio_util.* can be reverted? Of course. Fixed. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From prappo at openjdk.org Mon Oct 10 16:57:21 2022 From: prappo at openjdk.org (Pavel Rappo) Date: Mon, 10 Oct 2022 16:57:21 GMT Subject: Integrated: 8294377: Prepare to stop auto-inheriting documentation for subclasses of exceptions whose documentation is inherited In-Reply-To: References: Message-ID: <5x7fG0iSo2qnkGEPeTumleeWMpV15E_qSXdIrdVHoUY=.369c8b00-4a6e-4f50-9ca7-62be5556be8e@github.com> On Tue, 27 Sep 2022 11:43:08 GMT, Pavel Rappo wrote: > This adds exception documentation to JDK methods that would otherwise lose that documentation once JDK-8287796 is integrated. While adding this exception documentation now does not change [^1] the JDK API Documentation, it will automatically counterbalance the effect that JDK-8287796 will have. > > JDK-8287796 seeks to remove an old, undocumented, and esoteric javadoc feature that cannot be suppressed [^2]. The feature is so little known about that IDEs either do not implement it correctly or do not implement it at all, thus rendering documentation differently from how the javadoc tool renders it. > > That feature was introduced in JDK-4947455 and manifests as follows. If a method inherits documentation for an exception, along with that documentation the method automatically inherits documentation for all exceptions that are subclasses of that former exception and are documented in an overridden method. > > To illustrate that behavior, consider the following example. A method `Subtype.m` inherits documentation for `SuperException`, while the overridden method `Supertype.m` documents `SuperException`, `SubExceptionA` and `SubExceptionB`, where the latter two exceptions are subclasses of the former exception: > > public class Subtype extends Supertype { > > @Override > public void m() throws SuperException { > ... > > public class Supertype { > > /** > * @throws SuperException general problem > * @throws SubExceptionA specific problem A > * @throws SubExceptionB specific problem B > */ > public void m() throws SuperException { > ... > > public class SuperException extends Exception { > ... > > public class SubExceptionA extends SuperException { > ... > > public class SubExceptionB extends SuperException { > ... > > For the above example, the API Documentation for `Subtype.m` will contain documentation for all three exceptions; the page fragment for `Subtype.m` in "Method Details" will look like this: > > public void m() > throws SuperException > > Overrides: > m in class Supertype > Throws: > SuperException - general problem > SubExceptionA - specific problem A > SubExceptionB - specific problem B > > It works for checked and unchecked exceptions, for methods in classes and interfaces. > > > If it's the first time you hear about that behavior and this change affects your area, it's a good opportunity to reflect on whether the exception documentation this change adds is really needed or you were simply unaware of the fact that it was automatically added before. If exception documentation is not needed, please comment on this PR. Otherwise, consider approving it. > > Keep in mind that if some exception documentation is not needed, **leaving it out** from this PR might require a CSR. > > > [^1]: Aside from insignificant changes on class-use and index pages. There's one relatively significant change. This change is in jdk.jshell, where some methods disappeared from "Methods declared in ..." section and other irregularities. We are aware of this and have filed JDK-8291803 to fix the root cause. > [^2]: In reality, the feature can be suppressed, but it looks like a bug rather than intentional design. If we legitimize the feature and its suppression behavior, the model for exception documentation inheritance might become much more complicated. This pull request has now been integrated. Changeset: eb90c4fc Author: Pavel Rappo URL: https://git.openjdk.org/jdk/commit/eb90c4fc0479379c8c1452afca8f37746c762e18 Stats: 320 lines in 14 files changed: 310 ins; 0 del; 10 mod 8294377: Prepare to stop auto-inheriting documentation for subclasses of exceptions whose documentation is inherited Reviewed-by: jjg ------------- PR: https://git.openjdk.org/jdk/pull/10449 From bpb at openjdk.org Mon Oct 10 22:25:26 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 10 Oct 2022 22:25:26 GMT Subject: RFR: 8295025: (bf) ByteBuffer "Access to binary data" section suggests putFloat is void Message-ID: In the "Access to binary data" section of the class specification of `java.nio.ByteBuffer`, change `void putFloat` to `ByteBuffer putFloat`. ------------- Commit messages: - 8295025: (bf) ByteBuffer "Access to binary data" section suggests putFloat is void Changes: https://git.openjdk.org/jdk/pull/10638/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10638&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8295025 Stats: 5 lines in 1 file changed: 1 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/10638.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10638/head:pull/10638 PR: https://git.openjdk.org/jdk/pull/10638 From aturbanov at openjdk.org Tue Oct 11 08:10:03 2022 From: aturbanov at openjdk.org (Andrey Turbanov) Date: Tue, 11 Oct 2022 08:10:03 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v8] In-Reply-To: References: Message-ID: On Mon, 10 Oct 2022 10:33:38 GMT, Maxim Kartashev wrote: >> This is an implementation of `WatchService` based on File System Events API that is capable of generating events whenever a change occurs in an interesting directory or underneath it. Since the API naturally supports "recursive" watch, the `FILE_TREE` is supported by the watch service. >> >> Some things of note: >> * There's one "service" thread per `WatchService` instance that is inactive unless changes occur in the watched directory. The changes are grouped by introducing a time delay between when they occurred and when they are reported, which is controlled by the sensitivity modifier of the watch service. >> * Since FSEvents API reports directories only, the watch service keeps a snapshot (hierarchical if necessary) of the files in the directory being watched. The snapshot gets updated when an event in that directory or underneath it gets delivered. File changes are detected by comparing "last modified" time with a millisecond precision (`BasicFileAttributes.lastModifiedTime()`). >> * There is a slight complication with the move of an entire directory hierarchy: FSEvents API only reports about the containing directory of that move and not about any of the directories actually moved. There's a separate test for that (`Move.java`). >> * The code is careful not to do any I/O (such as reading the contents of a directory or attributes of a file) unless unavoidable. Any deviation from this line should be considered a bug (of, arguably, low priority). >> * The native part consists mostly of API wrappers with one exception of the callback function invoked by the system to report the events that occurred. The sole task of the function is to convert from C strings to Java strings and pass the array of affected directories to Java code. This can be rewritten if desired to make the code more future-proof. >> >> This commit leaves `PollingWatchService` unused. I'm not sure if I should/can do anything about it. Any advice is welcomed. >> >> ### Testing >> >> * Tested by running `test/jdk/java/nio/file` and `test/jdk/jdk/nio` on MacOS 10.15.7 (x64) and `test/jdk/java/nio/` plus `test/jdk/jdk/nio` on MacOS 12.5.1 (aarch64). >> * Also verified that new tests pass on Linux and Windows. >> * This code (albeit in a slightly modified form) has been in use at JetBrains for around half a year and a few bugs have been found and fixed during that time period. > > Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: > > Don't use JavaVM exported from nio_util.c src/java.base/macosx/classes/sun/nio/fs/MacOSXWatchService.java line 348: > 346: > 347: synchronized (eventStreamRefLock) { > 348: final int kFSEventStreamCreateFlagWatchRoot = 0x00000004; Suggestion: final int kFSEventStreamCreateFlagWatchRoot = 0x00000004; test/jdk/java/nio/file/WatchService/Move.java line 193: > 191: try { > 192: testMoveSubtree(dir); > 193: } catch(UnsupportedOperationException e) { Suggestion: } catch (UnsupportedOperationException e) { test/jdk/java/nio/file/WatchService/Move.java line 202: > 200: try { > 201: testMoveFileToDirectory(dir); > 202: } catch(UnsupportedOperationException e) { Suggestion: } catch (UnsupportedOperationException e) { ------------- PR: https://git.openjdk.org/jdk/pull/10140 From alanb at openjdk.org Tue Oct 11 08:47:27 2022 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 11 Oct 2022 08:47:27 GMT Subject: RFR: 8295025: (bf) ByteBuffer "Access to binary data" section suggests putFloat is void In-Reply-To: References: Message-ID: <1owaDW6ll6GjBQWd2vgFb-ibJvZ_fMB1G-V4Xjf97vc=.3eccba39-7fae-48e2-82dd-35e64b58587c@github.com> On Mon, 10 Oct 2022 22:05:58 GMT, Brian Burkhalter wrote: > In the "Access to binary data" section of the class specification of `java.nio.ByteBuffer`, change `void putFloat` to `ByteBuffer putFloat`. No normative changes so I don't think you need a CSR for this one. ------------- Marked as reviewed by alanb (Reviewer). PR: https://git.openjdk.org/jdk/pull/10638 From mkartashev at openjdk.org Tue Oct 11 12:15:27 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Tue, 11 Oct 2022 12:15:27 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v8] In-Reply-To: References: Message-ID: On Tue, 11 Oct 2022 08:07:51 GMT, Andrey Turbanov wrote: >> Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: >> >> Don't use JavaVM exported from nio_util.c > > src/java.base/macosx/classes/sun/nio/fs/MacOSXWatchService.java line 348: > >> 346: >> 347: synchronized (eventStreamRefLock) { >> 348: final int kFSEventStreamCreateFlagWatchRoot = 0x00000004; > > Suggestion: > > final int kFSEventStreamCreateFlagWatchRoot = 0x00000004; Thanks! I'll wait for more formatting-related input from @AlanBateman and others and then do all the changes in bulk. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From bpb at openjdk.org Thu Oct 13 18:43:30 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 13 Oct 2022 18:43:30 GMT Subject: Integrated: 8295025: (bf) ByteBuffer "Access to binary data" section suggests putFloat is void In-Reply-To: References: Message-ID: On Mon, 10 Oct 2022 22:05:58 GMT, Brian Burkhalter wrote: > In the "Access to binary data" section of the class specification of `java.nio.ByteBuffer`, change `void putFloat` to `ByteBuffer putFloat`. This pull request has now been integrated. Changeset: 18dd9ee7 Author: Brian Burkhalter URL: https://git.openjdk.org/jdk/commit/18dd9ee7010cc4c2ded24042049e94d3f775a785 Stats: 5 lines in 1 file changed: 1 ins; 0 del; 4 mod 8295025: (bf) ByteBuffer "Access to binary data" section suggests putFloat is void Reviewed-by: alanb ------------- PR: https://git.openjdk.org/jdk/pull/10638 From bpb at openjdk.org Thu Oct 13 23:54:12 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 13 Oct 2022 23:54:12 GMT Subject: RFR: 6924219: (fc spec) FileChannel.write(ByteBuffer, position) behavior when file opened for append not specified Message-ID: Add some verbiage stating that the outcome of invoking the absolute write method `java.nio.channels.write(ByteBuffer,long)` is unspecified when the channel was opened with the `APPEND` option present. ------------- Commit messages: - 6924219: (fc spec) FileChannel.write(ByteBuffer, position) behavior when file opened for append not specified Changes: https://git.openjdk.org/jdk/pull/10707/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10707&range=00 Issue: https://bugs.openjdk.org/browse/JDK-6924219 Stats: 9 lines in 1 file changed: 7 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/10707.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10707/head:pull/10707 PR: https://git.openjdk.org/jdk/pull/10707 From bpb at openjdk.org Fri Oct 14 16:38:54 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 14 Oct 2022 16:38:54 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v19] In-Reply-To: References: Message-ID: > Resurrection of the proposal to add a method to obtain the filename extension originated in PR [2319](https://github.com/openjdk/jdk/pull/2319). Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 24 additional commits since the last revision: - 8057113: Revise specification and align default implementation and test with it - Merge - 8057113: Remove API note - 8057113: Change specification as suggested in review - 8057113: Update specification, implementation, and test - 8057113: Correct length 2 case; update test cases - 8057113: Revert return type back to a nullable-String; update test - Merge - 8057113: Remove extensions() test method; add threthree new test cases - 8057113: Revert excision of wild cards from imports - ... and 14 more: https://git.openjdk.org/jdk/compare/9fe1ec51...104a7116 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/8066/files - new: https://git.openjdk.org/jdk/pull/8066/files/198ab9a8..104a7116 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=8066&range=18 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=8066&range=17-18 Stats: 195188 lines in 3394 files changed: 95414 ins; 79062 del; 20712 mod Patch: https://git.openjdk.org/jdk/pull/8066.diff Fetch: git fetch https://git.openjdk.org/jdk pull/8066/head:pull/8066 PR: https://git.openjdk.org/jdk/pull/8066 From bpb at openjdk.org Fri Oct 14 16:38:56 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 14 Oct 2022 16:38:56 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v18] In-Reply-To: References: Message-ID: On Mon, 1 Aug 2022 21:54:22 GMT, Brian Burkhalter wrote: >> Resurrection of the proposal to add a method to obtain the filename extension originated in PR [2319](https://github.com/openjdk/jdk/pull/2319). > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8057113: Remove API note The specification paragraph is rewritten in 104a7116288d80ea9cf384b078bbfbeb0f1eece7 but the `@implSpec` content is unchanged. ------------- PR: https://git.openjdk.org/jdk/pull/8066 From alanb at openjdk.org Fri Oct 14 17:08:19 2022 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 14 Oct 2022 17:08:19 GMT Subject: RFR: 6924219: (fc spec) FileChannel.write(ByteBuffer, position) behavior when file opened for append not specified In-Reply-To: References: Message-ID: On Thu, 13 Oct 2022 23:47:54 GMT, Brian Burkhalter wrote: > Add some verbiage stating that the outcome of invoking the absolute write method `java.nio.channels.write(ByteBuffer,long)` is unspecified when the channel was opened with the `APPEND` option present. src/java.base/share/classes/java/nio/channels/FileChannel.java line 151: > 149: * therefore unspecified. In this mode an invocation of the {@linkplain > 150: * #write(ByteBuffer,long) absolute write} operation leads to unspecified > 151: * behavior. I'm not sure about the phrase "absolute write" as we haven't used that before. I think it might be better to say something like "In this mode, the behavior of the method to write at a given position is also system-dependent". ------------- PR: https://git.openjdk.org/jdk/pull/10707 From bpb at openjdk.org Fri Oct 14 17:08:19 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 14 Oct 2022 17:08:19 GMT Subject: RFR: 6924219: (fc spec) FileChannel.write(ByteBuffer, position) behavior when file opened for append not specified In-Reply-To: References: Message-ID: On Fri, 14 Oct 2022 17:02:57 GMT, Alan Bateman wrote: >> Add some verbiage stating that the outcome of invoking the absolute write method `java.nio.channels.write(ByteBuffer,long)` is unspecified when the channel was opened with the `APPEND` option present. > > src/java.base/share/classes/java/nio/channels/FileChannel.java line 151: > >> 149: * therefore unspecified. In this mode an invocation of the {@linkplain >> 150: * #write(ByteBuffer,long) absolute write} operation leads to unspecified >> 151: * behavior. > > I'm not sure about the phrase "absolute write" as we haven't used that before. I think it might be better to say something like "In this mode, the behavior of the method to write at a given position is also system-dependent". I was keying off the expression "In this mode each invocation of a relative write operation" earlier in the same paragraph and also in analogy to absolute and relative operations in `Buffer`. That said, I don't disagree with the comment. ------------- PR: https://git.openjdk.org/jdk/pull/10707 From alanb at openjdk.org Fri Oct 14 17:33:42 2022 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 14 Oct 2022 17:33:42 GMT Subject: RFR: 6924219: (fc spec) FileChannel.write(ByteBuffer, position) behavior when file opened for append not specified In-Reply-To: References: Message-ID: On Thu, 13 Oct 2022 23:47:54 GMT, Brian Burkhalter wrote: > Add some verbiage stating that the outcome of invoking the absolute write method `java.nio.channels.write(ByteBuffer,long)` is unspecified when the channel was opened with the `APPEND` option present. If something has a file opened for append then it's a bug to attempt to write and specify a position. It would be possible to detect this, throw an exception, and thus avoid bugs. However it would be a behavior change. The conservative approach is to specify that it leads to unspecified behavior as you have done. ------------- PR: https://git.openjdk.org/jdk/pull/10707 From bpb at openjdk.org Fri Oct 14 18:13:06 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 14 Oct 2022 18:13:06 GMT Subject: RFR: 6924219: (fc spec) FileChannel.write(ByteBuffer, position) behavior when file opened for append not specified [v2] In-Reply-To: References: Message-ID: > Add some verbiage stating that the outcome of invoking the absolute write method `java.nio.channels.write(ByteBuffer,long)` is unspecified when the channel was opened with the `APPEND` option present. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 6924219: Replaced "absolute write" verbiage ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10707/files - new: https://git.openjdk.org/jdk/pull/10707/files/ac9ba09e..16383bf6 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10707&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10707&range=00-01 Stats: 5 lines in 1 file changed: 0 ins; 0 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/10707.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10707/head:pull/10707 PR: https://git.openjdk.org/jdk/pull/10707 From rriggs at openjdk.org Fri Oct 14 21:18:15 2022 From: rriggs at openjdk.org (Roger Riggs) Date: Fri, 14 Oct 2022 21:18:15 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v19] In-Reply-To: References: Message-ID: On Fri, 14 Oct 2022 16:38:54 GMT, Brian Burkhalter wrote: >> Resurrection of the proposal to add a method to obtain the filename extension originated in PR [2319](https://github.com/openjdk/jdk/pull/2319). > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 24 additional commits since the last revision: > > - 8057113: Revise specification and align default implementation and test with it > - Merge > - 8057113: Remove API note > - 8057113: Change specification as suggested in review > - 8057113: Update specification, implementation, and test > - 8057113: Correct length 2 case; update test cases > - 8057113: Revert return type back to a nullable-String; update test > - Merge > - 8057113: Remove extensions() test method; add threthree new test cases > - 8057113: Revert excision of wild cards from imports > - ... and 14 more: https://git.openjdk.org/jdk/compare/2398d46b...104a7116 The new spec for determining the extension looks fine. ------------- Marked as reviewed by rriggs (Reviewer). PR: https://git.openjdk.org/jdk/pull/8066 From jwaters at openjdk.org Sun Oct 16 08:45:54 2022 From: jwaters at openjdk.org (Julian Waters) Date: Sun, 16 Oct 2022 08:45:54 GMT Subject: RFR: 8295231: Move all linking of native libraries to make Message-ID: Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. ------------- Commit messages: - Commit remaining libraries - Merge branch 'openjdk:master' into patch-2 - Comment change, force link failures to determine libraries to include in make - Update WinSysInfo.cpp - Update WinFileUtils.cpp - Update WinApp.cpp - Update MsiUtils.cpp - Update MsiDb.cpp - Update MsiCA.cpp - Update Guid.cpp - ... and 12 more: https://git.openjdk.org/jdk/compare/0043d58c...cfa80528 Changes: https://git.openjdk.org/jdk/pull/10633/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10633&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8295231 Stats: 35 lines in 14 files changed: 3 ins; 22 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/10633.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10633/head:pull/10633 PR: https://git.openjdk.org/jdk/pull/10633 From alanb at openjdk.org Sun Oct 16 13:21:53 2022 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 16 Oct 2022 13:21:53 GMT Subject: RFR: 8295231: Move all linking of native libraries to make In-Reply-To: References: Message-ID: On Mon, 10 Oct 2022 14:15:37 GMT, Julian Waters wrote: > Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. src/java.base/windows/native/libnio/ch/FileDispatcherImpl.c line 38: > 36: > 37: #include > 38: #pragma comment(lib, "Mswsock.lib") I think this came about with one of the early Microsoft contributions to have transferTo optionally use TransmitFile on Windows. This create the dependency on Mswsock. ------------- PR: https://git.openjdk.org/jdk/pull/10633 From alanb at openjdk.org Sun Oct 16 13:30:10 2022 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 16 Oct 2022 13:30:10 GMT Subject: RFR: 6924219: (fc spec) FileChannel.write(ByteBuffer, position) behavior when file opened for append not specified [v2] In-Reply-To: References: Message-ID: <13zJPPStCGZ2G5E5mw_p-jbfbv0RUftQa0aB2qBydbU=.ee4c1a5a-54d1-4674-bcbe-b18eec7999a7@github.com> On Fri, 14 Oct 2022 18:13:06 GMT, Brian Burkhalter wrote: >> Add some verbiage stating that the outcome of invoking the absolute write method `java.nio.channels.write(ByteBuffer,long)` is unspecified when the channel was opened with the `APPEND` option present. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 6924219: Replaced "absolute write" verbiage Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/10707 From alanb at openjdk.org Sun Oct 16 14:50:08 2022 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 16 Oct 2022 14:50:08 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v19] In-Reply-To: References: Message-ID: <2VGoWiGegfX8kXxEDM5GR9DVxbZuOc-1FjZnxlmAECg=.7d8f1355-b8dc-4fab-bb32-90a1814aedf9@github.com> On Fri, 14 Oct 2022 16:38:54 GMT, Brian Burkhalter wrote: >> Resurrection of the proposal to add a method to obtain the filename extension originated in PR [2319](https://github.com/openjdk/jdk/pull/2319). > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 24 additional commits since the last revision: > > - 8057113: Revise specification and align default implementation and test with it > - Merge > - 8057113: Remove API note > - 8057113: Change specification as suggested in review > - 8057113: Update specification, implementation, and test > - 8057113: Correct length 2 case; update test cases > - 8057113: Revert return type back to a nullable-String; update test > - Merge > - 8057113: Remove extensions() test method; add threthree new test cases > - 8057113: Revert excision of wild cards from imports > - ... and 14 more: https://git.openjdk.org/jdk/compare/8fca5b62...104a7116 src/java.base/share/classes/java/nio/file/Path.java line 253: > 251: > 252: /** > 253: * Returns the file name extension of this path as a {@code String}. The latest proposal looks good but I wonder if we can improve on "Returns the file name extension of this path". What would you think of "Returns the file extension of this path's file name" ? test/jdk/java/nio/file/Path/Extensions.java line 25: > 23: > 24: import java.nio.file.Path; > 25: import java.util.Arrays; I think the import of Arrays is left over from a previous iteration. ------------- PR: https://git.openjdk.org/jdk/pull/8066 From dholmes at openjdk.org Mon Oct 17 02:42:48 2022 From: dholmes at openjdk.org (David Holmes) Date: Mon, 17 Oct 2022 02:42:48 GMT Subject: RFR: 8295231: Move all linking of native libraries to make In-Reply-To: References: Message-ID: On Mon, 10 Oct 2022 14:15:37 GMT, Julian Waters wrote: > Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. I don't agree with the justification here. This seems very windows specific but I like the idea that the source code tracks its own library dependencies. If I am writing some windows code that uses a particular Windows API which in turn requires a specific windows library, then these pragma comments seem an ideal way to express that dependency. This has the advantage that (a) the developer doesn't require detailed knowledge of the build system to make things work; and (b) there is more chance that if the code is later removed then removing the linking of the library will not get overlooked. YMMV. ------------- PR: https://git.openjdk.org/jdk/pull/10633 From stuefe at openjdk.org Mon Oct 17 06:08:04 2022 From: stuefe at openjdk.org (Thomas Stuefe) Date: Mon, 17 Oct 2022 06:08:04 GMT Subject: RFR: 8295231: Move all linking of native libraries to make In-Reply-To: References: Message-ID: On Mon, 10 Oct 2022 14:15:37 GMT, Julian Waters wrote: > Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. I like this change. I am pretty sure not many know about this feature at all, we don't have that many knowledgeable Windows developers. If the methods are equivalent, I prefer linking via make file. Pinging @magicus, maybe he can chime in. Cheers, Thomas ------------- PR: https://git.openjdk.org/jdk/pull/10633 From ihse at openjdk.org Mon Oct 17 09:27:10 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 17 Oct 2022 09:27:10 GMT Subject: RFR: 8295231: Move all linking of native libraries to make In-Reply-To: References: Message-ID: On Mon, 10 Oct 2022 14:15:37 GMT, Julian Waters wrote: > Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. Marked as reviewed by ihse (Reviewer). Wow. I did not even know this was possible. Thank you for fixing this! I would have been mighty surprised if I were to learn that a library has more dependencies than the one in the makefile. @dholmes-ora The point is that we need to be consistent. If we would go that route, then *all* libraries should be loaded by pragmas. That could of course be an alternative, but I really see no upside to it. To do it like we currently do, load 99% of the libraries via make files, and then have few scattered pragmas, that's just bad. @TheShermanTanker Question: is this a Windows-specific thing, or are there pragma-loaded libraries for other compilers as well? ------------- PR: https://git.openjdk.org/jdk/pull/10633 From jwaters at openjdk.org Mon Oct 17 14:21:32 2022 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 17 Oct 2022 14:21:32 GMT Subject: RFR: 8295231: Move all linking of native libraries to make In-Reply-To: References: Message-ID: On Mon, 17 Oct 2022 09:24:58 GMT, Magnus Ihse Bursie wrote: > @TheShermanTanker Question: is this a Windows-specific thing, or are there pragma-loaded libraries for other compilers as well? To my knowledge only Visual C++ has the ability to perform linking through pragmas, the comment pragma works by leaving a linker comment in the object file (hence the name), meaning this only works with the Visual C++ linker, so while clang does support this pragma with clang-cl, for the use cases in the JDK it wouldn't matter. gcc does not have an equivalent pragma (or provide support for linking from inside source files at all, for that matter) > If the methods are equivalent, I prefer linking via make file. There isn't any difference between the 2 unless the library is tampered with by `-nodefaultlib`, but I can only find that specified in https://github.com/openjdk/jdk/blob/a033aa5a3d9c63d72d11af218b9896b037fbd8de/make/autoconf/flags-other.m4#L38 and https://github.com/openjdk/jdk/blob/392f35df4be1a9a8d7a67a25ae01230c7dd060ac/make/autoconf/lib-hsdis.m4#L45, neither of which have an effect on the libraries in this changeset ------------- PR: https://git.openjdk.org/jdk/pull/10633 From jwaters at openjdk.org Mon Oct 17 14:25:08 2022 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 17 Oct 2022 14:25:08 GMT Subject: RFR: 8295231: Move all linking of native libraries to make [v2] In-Reply-To: References: Message-ID: <3fTguMGwo3QBguzYDzhuTVTiqTzn8ZVPiGM5xYuqZbw=.96637af9-ed38-49aa-aa99-71c656a11320@github.com> > Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. Julian Waters has updated the pull request incrementally with two additional commits since the last revision: - user32 - Mswsock.lib requirement ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10633/files - new: https://git.openjdk.org/jdk/pull/10633/files/cfa80528..54e6ec10 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10633&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10633&range=00-01 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/10633.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10633/head:pull/10633 PR: https://git.openjdk.org/jdk/pull/10633 From jwaters at openjdk.org Mon Oct 17 14:27:32 2022 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 17 Oct 2022 14:27:32 GMT Subject: RFR: 8295231: Move all linking of native libraries to make [v3] In-Reply-To: References: Message-ID: <2ETlbmDLPJuS-GkWsIceJISmHts7aROCmrbPKS3lsag=.1fbf54ad-a174-4865-87b1-66c20004cb12@github.com> > Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. Julian Waters has updated the pull request incrementally with one additional commit since the last revision: Update Guid.cpp ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10633/files - new: https://git.openjdk.org/jdk/pull/10633/files/54e6ec10..fe15f907 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10633&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10633&range=01-02 Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/10633.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10633/head:pull/10633 PR: https://git.openjdk.org/jdk/pull/10633 From jwaters at openjdk.org Mon Oct 17 14:27:32 2022 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 17 Oct 2022 14:27:32 GMT Subject: RFR: 8295231: Move all linking of native libraries to make [v3] In-Reply-To: References: Message-ID: On Sun, 16 Oct 2022 13:18:14 GMT, Alan Bateman wrote: >> Julian Waters has updated the pull request incrementally with one additional commit since the last revision: >> >> Update Guid.cpp > > src/java.base/windows/native/libnio/ch/FileDispatcherImpl.c line 38: > >> 36: >> 37: #include >> 38: #pragma comment(lib, "Mswsock.lib") > > I think this came about with one of the early Microsoft contributions to have transferTo optionally use TransmitFile on Windows. This created the dependency on Mswsock. It's not clear why a pragma was used though. I believe it may have had to do with explicitly showing the dependency as David suggests, I added a comment explaining this just in case ------------- PR: https://git.openjdk.org/jdk/pull/10633 From jwaters at openjdk.org Mon Oct 17 14:31:46 2022 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 17 Oct 2022 14:31:46 GMT Subject: RFR: 8295231: Move all linking of native libraries to make [v4] In-Reply-To: References: Message-ID: > Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. Julian Waters has updated the pull request incrementally with five additional commits since the last revision: - Update WinSysInfo.cpp - Update WinApp.cpp - Update MsiUtils.cpp - Update MsiDb.cpp - Update MsiCA.cpp ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10633/files - new: https://git.openjdk.org/jdk/pull/10633/files/fe15f907..8afaf85d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10633&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10633&range=02-03 Stats: 12 lines in 5 files changed: 11 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/10633.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10633/head:pull/10633 PR: https://git.openjdk.org/jdk/pull/10633 From jwaters at openjdk.org Mon Oct 17 14:35:32 2022 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 17 Oct 2022 14:35:32 GMT Subject: RFR: 8295231: Move all linking of native libraries to make [v5] In-Reply-To: References: Message-ID: <1FdeByT381QCHgVhV7RfaietWwt_xj2jmQuZBFklSJM=.0cfa6d5d-3ccd-45cf-84c4-66d3e8b595e4@github.com> > Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. Julian Waters has updated the pull request incrementally with one additional commit since the last revision: Update WindowsRegistry.cpp ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10633/files - new: https://git.openjdk.org/jdk/pull/10633/files/8afaf85d..0779d1fa Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10633&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10633&range=03-04 Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/10633.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10633/head:pull/10633 PR: https://git.openjdk.org/jdk/pull/10633 From jwaters at openjdk.org Mon Oct 17 14:35:32 2022 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 17 Oct 2022 14:35:32 GMT Subject: RFR: 8295231: Move all linking of native libraries to make [v4] In-Reply-To: References: Message-ID: On Mon, 17 Oct 2022 14:31:46 GMT, Julian Waters wrote: >> Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. > > Julian Waters has updated the pull request incrementally with five additional commits since the last revision: > > - Update WinSysInfo.cpp > - Update WinApp.cpp > - Update MsiUtils.cpp > - Update MsiDb.cpp > - Update MsiCA.cpp Thanks all for the reviews, I placed comments where the pragmas used to be detailing the library required by the source file to address David's concerns ------------- PR: https://git.openjdk.org/jdk/pull/10633 From jwaters at openjdk.org Mon Oct 17 14:41:06 2022 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 17 Oct 2022 14:41:06 GMT Subject: RFR: 8295231: Move all linking of native libraries to make [v6] In-Reply-To: References: Message-ID: > Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. Julian Waters has updated the pull request incrementally with one additional commit since the last revision: Formatting ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10633/files - new: https://git.openjdk.org/jdk/pull/10633/files/0779d1fa..4eb2eb7b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10633&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10633&range=04-05 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/10633.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10633/head:pull/10633 PR: https://git.openjdk.org/jdk/pull/10633 From bpb at openjdk.org Mon Oct 17 16:24:22 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 17 Oct 2022 16:24:22 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v19] In-Reply-To: <2VGoWiGegfX8kXxEDM5GR9DVxbZuOc-1FjZnxlmAECg=.7d8f1355-b8dc-4fab-bb32-90a1814aedf9@github.com> References: <2VGoWiGegfX8kXxEDM5GR9DVxbZuOc-1FjZnxlmAECg=.7d8f1355-b8dc-4fab-bb32-90a1814aedf9@github.com> Message-ID: <3mL04PxDvhJSqRjqUbw7Ab_AUIp3f5GsHntzD9451ys=.6abe8a91-a662-4265-979a-72edc0b9ecea@github.com> On Sun, 16 Oct 2022 14:46:09 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 24 additional commits since the last revision: >> >> - 8057113: Revise specification and align default implementation and test with it >> - Merge >> - 8057113: Remove API note >> - 8057113: Change specification as suggested in review >> - 8057113: Update specification, implementation, and test >> - 8057113: Correct length 2 case; update test cases >> - 8057113: Revert return type back to a nullable-String; update test >> - Merge >> - 8057113: Remove extensions() test method; add threthree new test cases >> - 8057113: Revert excision of wild cards from imports >> - ... and 14 more: https://git.openjdk.org/jdk/compare/2a2cfaa7...104a7116 > > src/java.base/share/classes/java/nio/file/Path.java line 253: > >> 251: >> 252: /** >> 253: * Returns the file name extension of this path as a {@code String}. > > The latest proposal looks good but I wonder if we can improve on "Returns the file name extension of this path". What would you think of "Returns the file extension of this path's file name" ? That sounds reasonable. > test/jdk/java/nio/file/Path/Extensions.java line 25: > >> 23: >> 24: import java.nio.file.Path; >> 25: import java.util.Arrays; > > I think the import of Arrays is left over from a previous iteration. It is. ------------- PR: https://git.openjdk.org/jdk/pull/8066 From bpb at openjdk.org Mon Oct 17 17:12:08 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 17 Oct 2022 17:12:08 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v20] In-Reply-To: References: Message-ID: > Resurrection of the proposal to add a method to obtain the filename extension originated in PR [2319](https://github.com/openjdk/jdk/pull/2319). Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8057113: Modify topic sentence of spec; remove obsolete import from test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/8066/files - new: https://git.openjdk.org/jdk/pull/8066/files/104a7116..66cb4a42 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=8066&range=19 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=8066&range=18-19 Stats: 2 lines in 2 files changed: 0 ins; 1 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/8066.diff Fetch: git fetch https://git.openjdk.org/jdk pull/8066/head:pull/8066 PR: https://git.openjdk.org/jdk/pull/8066 From bpb at openjdk.org Mon Oct 17 17:12:10 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 17 Oct 2022 17:12:10 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v19] In-Reply-To: <3mL04PxDvhJSqRjqUbw7Ab_AUIp3f5GsHntzD9451ys=.6abe8a91-a662-4265-979a-72edc0b9ecea@github.com> References: <2VGoWiGegfX8kXxEDM5GR9DVxbZuOc-1FjZnxlmAECg=.7d8f1355-b8dc-4fab-bb32-90a1814aedf9@github.com> <3mL04PxDvhJSqRjqUbw7Ab_AUIp3f5GsHntzD9451ys=.6abe8a91-a662-4265-979a-72edc0b9ecea@github.com> Message-ID: On Mon, 17 Oct 2022 16:21:50 GMT, Brian Burkhalter wrote: >> src/java.base/share/classes/java/nio/file/Path.java line 253: >> >>> 251: >>> 252: /** >>> 253: * Returns the file name extension of this path as a {@code String}. >> >> The latest proposal looks good but I wonder if we can improve on "Returns the file name extension of this path". What would you think of "Returns the file extension of this path's file name" ? > > That sounds reasonable. So changed in 66cb4a42b15b2db0427125e404684060b59b6f39. >> test/jdk/java/nio/file/Path/Extensions.java line 25: >> >>> 23: >>> 24: import java.nio.file.Path; >>> 25: import java.util.Arrays; >> >> I think the import of Arrays is left over from a previous iteration. > > It is. Removed in 66cb4a42b15b2db0427125e404684060b59b6f39. ------------- PR: https://git.openjdk.org/jdk/pull/8066 From bpb at openjdk.org Mon Oct 17 17:12:12 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 17 Oct 2022 17:12:12 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v18] In-Reply-To: References: Message-ID: On Wed, 3 Aug 2022 22:30:53 GMT, Stuart Marks wrote: >> src/java.base/share/classes/java/nio/file/Path.java line 297: >> >>> 295: if (lastPeriodIndex == length - 1) { >>> 296: // null if all period characters, otherwise empty >>> 297: return fileNameString.matches("\\.{" + length + "}") ? >> >> Using a regular expression here is probably okay for the initial PR but I could imagine this show up in profiles and needing to be re-implemented. > > Yeah I don't think we want to use regex here. I'd suggest something like this: > > int lastDot = fileNameString.lastIndexOf('.'); > for (int j = lastDot - 1; j >= 0; j--) { > if (fileNameString.charAt(j) != '.') { > return fileNameString.substring(lastDot + 1); > } > } > return null; No longer relevant. ------------- PR: https://git.openjdk.org/jdk/pull/8066 From asemenyuk at openjdk.org Mon Oct 17 17:19:59 2022 From: asemenyuk at openjdk.org (Alexey Semenyuk) Date: Mon, 17 Oct 2022 17:19:59 GMT Subject: RFR: 8295231: Move all linking of native libraries to make [v6] In-Reply-To: References: Message-ID: On Mon, 17 Oct 2022 14:41:06 GMT, Julian Waters wrote: >> Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. > > Julian Waters has updated the pull request incrementally with one additional commit since the last revision: > > Formatting The change looks harmless. Howevere I don't understand how searching for the standard Windows libs can then become frustrating. ------------- PR: https://git.openjdk.org/jdk/pull/10633 From erikj at openjdk.org Mon Oct 17 17:35:59 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 17 Oct 2022 17:35:59 GMT Subject: RFR: 8295231: Move all linking of native libraries to make [v6] In-Reply-To: References: Message-ID: On Mon, 17 Oct 2022 14:41:06 GMT, Julian Waters wrote: >> Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. > > Julian Waters has updated the pull request incrementally with one additional commit since the last revision: > > Formatting Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/10633 From erikj at openjdk.org Mon Oct 17 17:41:07 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 17 Oct 2022 17:41:07 GMT Subject: RFR: 8295231: Move all linking of native libraries to make [v6] In-Reply-To: References: Message-ID: On Mon, 17 Oct 2022 17:16:21 GMT, Alexey Semenyuk wrote: > The change looks harmless. Howevere I don't understand how searching for the standard Windows libs can then become frustrating. I believe this is part of the effort for https://bugs.openjdk.org/browse/JDK-8288293. ------------- PR: https://git.openjdk.org/jdk/pull/10633 From lancea at openjdk.org Mon Oct 17 17:43:10 2022 From: lancea at openjdk.org (Lance Andersen) Date: Mon, 17 Oct 2022 17:43:10 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v20] In-Reply-To: References: Message-ID: <4HHboagP6n8zcda7LfcAz7-Nh4kBBrmp8lL2T7LTaQg=.5390c9f2-b875-4959-94ee-2ee98a8ad563@github.com> On Mon, 17 Oct 2022 17:12:08 GMT, Brian Burkhalter wrote: >> Resurrection of the proposal to add a method to obtain the filename extension originated in PR [2319](https://github.com/openjdk/jdk/pull/2319). > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8057113: Modify topic sentence of spec; remove obsolete import from test looks good to me ------------- Marked as reviewed by lancea (Reviewer). PR: https://git.openjdk.org/jdk/pull/8066 From asemenyuk at openjdk.org Mon Oct 17 17:46:04 2022 From: asemenyuk at openjdk.org (Alexey Semenyuk) Date: Mon, 17 Oct 2022 17:46:04 GMT Subject: RFR: 8295231: Move all linking of native libraries to make [v6] In-Reply-To: References: Message-ID: On Mon, 17 Oct 2022 17:37:11 GMT, Erik Joelsson wrote: > I believe this is part of the effort for https://bugs.openjdk.org/browse/JDK-8288293. Agree. I'd prefer to have a different description of the bug though to make it clear that this is necessary for decoupling a compiler and an OS. ------------- PR: https://git.openjdk.org/jdk/pull/10633 From jwaters at openjdk.org Mon Oct 17 21:00:10 2022 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 17 Oct 2022 21:00:10 GMT Subject: Integrated: 8295231: Move all linking of native libraries to make In-Reply-To: References: Message-ID: On Mon, 10 Oct 2022 14:15:37 GMT, Julian Waters wrote: > Some external libraries required by native code are linked via linker comments embedded in pragmas. Searching for which libraries are linked can then become frustrating and confusing since they may be included in an obscure place, and for all relevant compilers there is no difference between specifying them from make and in a source file. The easiest solution is to just always link them from make and remove any source level linkage. This pull request has now been integrated. Changeset: 8d751de3 Author: Julian Waters Committer: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/8d751de3198675b22704cdccafaff2fc0fdd3f59 Stats: 26 lines in 14 files changed: 3 ins; 6 del; 17 mod 8295231: Move all linking of native libraries to make Reviewed-by: ihse, erikj ------------- PR: https://git.openjdk.org/jdk/pull/10633 From bpb at openjdk.org Tue Oct 18 17:03:04 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 18 Oct 2022 17:03:04 GMT Subject: Integrated: 6924219: (fc spec) FileChannel.write(ByteBuffer, position) behavior when file opened for append not specified In-Reply-To: References: Message-ID: On Thu, 13 Oct 2022 23:47:54 GMT, Brian Burkhalter wrote: > Add some verbiage stating that the outcome of invoking the absolute write method `java.nio.channels.write(ByteBuffer,long)` is unspecified when the channel was opened with the `APPEND` option present. This pull request has now been integrated. Changeset: d1f79458 Author: Brian Burkhalter URL: https://git.openjdk.org/jdk/commit/d1f794587cbac221649114b71f2fce5e1f8b7e49 Stats: 9 lines in 1 file changed: 7 ins; 0 del; 2 mod 6924219: (fc spec) FileChannel.write(ByteBuffer, position) behavior when file opened for append not specified Reviewed-by: alanb ------------- PR: https://git.openjdk.org/jdk/pull/10707 From bpb at openjdk.org Tue Oct 18 18:27:02 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 18 Oct 2022 18:27:02 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v8] In-Reply-To: References: Message-ID: On Mon, 10 Oct 2022 10:33:38 GMT, Maxim Kartashev wrote: >> This is an implementation of `WatchService` based on File System Events API that is capable of generating events whenever a change occurs in an interesting directory or underneath it. Since the API naturally supports "recursive" watch, the `FILE_TREE` is supported by the watch service. >> >> Some things of note: >> * There's one "service" thread per `WatchService` instance that is inactive unless changes occur in the watched directory. The changes are grouped by introducing a time delay between when they occurred and when they are reported, which is controlled by the sensitivity modifier of the watch service. >> * Since FSEvents API reports directories only, the watch service keeps a snapshot (hierarchical if necessary) of the files in the directory being watched. The snapshot gets updated when an event in that directory or underneath it gets delivered. File changes are detected by comparing "last modified" time with a millisecond precision (`BasicFileAttributes.lastModifiedTime()`). >> * There is a slight complication with the move of an entire directory hierarchy: FSEvents API only reports about the containing directory of that move and not about any of the directories actually moved. There's a separate test for that (`Move.java`). >> * The code is careful not to do any I/O (such as reading the contents of a directory or attributes of a file) unless unavoidable. Any deviation from this line should be considered a bug (of, arguably, low priority). >> * The native part consists mostly of API wrappers with one exception of the callback function invoked by the system to report the events that occurred. The sole task of the function is to convert from C strings to Java strings and pass the array of affected directories to Java code. This can be rewritten if desired to make the code more future-proof. >> >> This commit leaves `PollingWatchService` unused. I'm not sure if I should/can do anything about it. Any advice is welcomed. >> >> ### Testing >> >> * Tested by running `test/jdk/java/nio/file` and `test/jdk/jdk/nio` on MacOS 10.15.7 (x64) and `test/jdk/java/nio/` plus `test/jdk/jdk/nio` on MacOS 12.5.1 (aarch64). >> * Also verified that new tests pass on Linux and Windows. >> * This code (albeit in a slightly modified form) has been in use at JetBrains for around half a year and a few bugs have been found and fixed during that time period. > > Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: > > Don't use JavaVM exported from nio_util.c src/java.base/macosx/classes/sun/nio/fs/BsdFileSystem.java line 55: > 53: throws IOException > 54: { > 55: return new MacOSXWatchService(); Is it still intended that there be a system property to enable `MacOSXWatchService` which would be off by default? ------------- PR: https://git.openjdk.org/jdk/pull/10140 From bpb at openjdk.org Tue Oct 18 18:36:08 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 18 Oct 2022 18:36:08 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v8] In-Reply-To: References: Message-ID: On Mon, 10 Oct 2022 10:33:38 GMT, Maxim Kartashev wrote: >> This is an implementation of `WatchService` based on File System Events API that is capable of generating events whenever a change occurs in an interesting directory or underneath it. Since the API naturally supports "recursive" watch, the `FILE_TREE` is supported by the watch service. >> >> Some things of note: >> * There's one "service" thread per `WatchService` instance that is inactive unless changes occur in the watched directory. The changes are grouped by introducing a time delay between when they occurred and when they are reported, which is controlled by the sensitivity modifier of the watch service. >> * Since FSEvents API reports directories only, the watch service keeps a snapshot (hierarchical if necessary) of the files in the directory being watched. The snapshot gets updated when an event in that directory or underneath it gets delivered. File changes are detected by comparing "last modified" time with a millisecond precision (`BasicFileAttributes.lastModifiedTime()`). >> * There is a slight complication with the move of an entire directory hierarchy: FSEvents API only reports about the containing directory of that move and not about any of the directories actually moved. There's a separate test for that (`Move.java`). >> * The code is careful not to do any I/O (such as reading the contents of a directory or attributes of a file) unless unavoidable. Any deviation from this line should be considered a bug (of, arguably, low priority). >> * The native part consists mostly of API wrappers with one exception of the callback function invoked by the system to report the events that occurred. The sole task of the function is to convert from C strings to Java strings and pass the array of affected directories to Java code. This can be rewritten if desired to make the code more future-proof. >> >> This commit leaves `PollingWatchService` unused. I'm not sure if I should/can do anything about it. Any advice is welcomed. >> >> ### Testing >> >> * Tested by running `test/jdk/java/nio/file` and `test/jdk/jdk/nio` on MacOS 10.15.7 (x64) and `test/jdk/java/nio/` plus `test/jdk/jdk/nio` on MacOS 12.5.1 (aarch64). >> * Also verified that new tests pass on Linux and Windows. >> * This code (albeit in a slightly modified form) has been in use at JetBrains for around half a year and a few bugs have been found and fixed during that time period. > > Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: > > Don't use JavaVM exported from nio_util.c src/java.base/macosx/classes/sun/nio/fs/MacOSXWatchService.java line 52: > 50: class MacOSXWatchService extends AbstractWatchService { > 51: private static final MacOSXFileSystemProvider theFSProvider = DefaultFileSystemProvider.instance(); > 52: private static final MacOSXFileSystem theFS = (MacOSXFileSystem) theFSProvider.theFileSystem(); Typically `static final` constants are presented in upper case with underscores as for example `THE_FS_PROVIDER`. src/java.base/macosx/classes/sun/nio/fs/MacOSXWatchService.java line 66: > 64: t.start(); > 65: > 66: // In order to be able to schedule any FSEventStream's, a reference to a run loop is required. Nit: I think it should be "FSEventStreams", i.e., no apostrophe. src/java.base/macosx/classes/sun/nio/fs/MacOSXWatchService.java line 313: > 311: private static final long kFSEventStreamEventFlagRootChanged = 0x00000020; > 312: > 313: private static final Path relativeRootPath = theFS.getPath(""); Same comment as for lines 51-52. Unsure of line 311 however as that refers to a specific constant. Should that be perhaps in `UnixConstants`? src/java.base/macosx/classes/sun/nio/fs/MacOSXWatchService.java line 379: > 377: > 378: private Path toRelativePath(final String absPath) { > 379: return (absPath.length() > realRootPathLength) Nit: Are there several spaces after `return`? ------------- PR: https://git.openjdk.org/jdk/pull/10140 From bpb at openjdk.org Tue Oct 18 23:31:13 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 18 Oct 2022 23:31:13 GMT Subject: RFR: 8295456: (ch) sun.nio.ch.Util::checkBufferPositionAligned gives misleading/incorrect error Message-ID: Improve `IOException` message in `sun.nio.ch.Util::checkBufferPositionAligned`. ------------- Commit messages: - 8295456: (ch) sun.nio.ch.Util::checkBufferPositionAligned gives misleading/incorrect error Changes: https://git.openjdk.org/jdk/pull/10753/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10753&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8295456 Stats: 10 lines in 1 file changed: 1 ins; 3 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/10753.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10753/head:pull/10753 PR: https://git.openjdk.org/jdk/pull/10753 From bpb at openjdk.org Tue Oct 18 23:49:57 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 18 Oct 2022 23:49:57 GMT Subject: RFR: 8295456: (ch) sun.nio.ch.Util::checkBufferPositionAligned gives misleading/incorrect error [v2] In-Reply-To: References: Message-ID: <7jpfRfuYGtBX6qfR3gMjeSeKSFc_ir-_KNUy8HTTJhY=.029af887-806b-4f8d-b6e0-c18a493c0682@github.com> > Improve `IOException` message in `sun.nio.ch.Util::checkBufferPositionAligned`. Brian Burkhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: 8295456: (ch) sun.nio.ch.Util::checkBufferPositionAligned gives misleading/incorrect error ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10753/files - new: https://git.openjdk.org/jdk/pull/10753/files/b073ad68..a09caf96 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10753&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10753&range=00-01 Stats: 4 lines in 2 files changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/10753.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10753/head:pull/10753 PR: https://git.openjdk.org/jdk/pull/10753 From mkartashev at openjdk.org Wed Oct 19 05:57:00 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Wed, 19 Oct 2022 05:57:00 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v8] In-Reply-To: References: Message-ID: On Tue, 18 Oct 2022 18:24:44 GMT, Brian Burkhalter wrote: >> Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: >> >> Don't use JavaVM exported from nio_util.c > > src/java.base/macosx/classes/sun/nio/fs/BsdFileSystem.java line 55: > >> 53: throws IOException >> 54: { >> 55: return new MacOSXWatchService(); > > Is it still intended that there be a system property to enable `MacOSXWatchService` which would be off by default? There was such a suggestion (not sure about "off by default", though). I guess this should go through CSR review, in which case I'm going to need some guidance. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From mkartashev at openjdk.org Wed Oct 19 06:14:09 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Wed, 19 Oct 2022 06:14:09 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v8] In-Reply-To: References: Message-ID: On Tue, 18 Oct 2022 18:31:59 GMT, Brian Burkhalter wrote: >> Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: >> >> Don't use JavaVM exported from nio_util.c > > src/java.base/macosx/classes/sun/nio/fs/MacOSXWatchService.java line 379: > >> 377: >> 378: private Path toRelativePath(final String absPath) { >> 379: return (absPath.length() > realRootPathLength) > > Nit: Are there several spaces after `return`? Yes, to align the beginning of the returned expression with its continuations on the subsequent two lines. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From mkartashev at openjdk.org Wed Oct 19 06:18:42 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Wed, 19 Oct 2022 06:18:42 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v8] In-Reply-To: References: Message-ID: <0TX1LHrwcqFB8uWK1K23a-4evh1jYT_ZnRF8CttweBk=.47ea26d9-b3a3-4550-8c64-474218b859e4@github.com> On Tue, 18 Oct 2022 18:30:46 GMT, Brian Burkhalter wrote: >> Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: >> >> Don't use JavaVM exported from nio_util.c > > src/java.base/macosx/classes/sun/nio/fs/MacOSXWatchService.java line 313: > >> 311: private static final long kFSEventStreamEventFlagRootChanged = 0x00000020; >> 312: >> 313: private static final Path relativeRootPath = theFS.getPath(""); > > Same comment as for lines 51-52. Unsure of line 311 however as that refers to a specific constant. Should that be perhaps in `UnixConstants`? I'll change the spelling of all but `kFSEventStreamEventFlagRootChanged`, which I'd rather keep as is. The constant being spelled exactly the same as in the corresponding C header file makes it easier to check its correctness (https://developer.apple.com/documentation/coreservices/1455361-fseventstreameventflags/kfseventstreameventflagrootchanged?language=objc). As for moving it to `UnixConstants`, I'm also not so sure as it is hardly a "Unix" constant, more like a macOS constant. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From mkartashev at openjdk.org Wed Oct 19 06:26:00 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Wed, 19 Oct 2022 06:26:00 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v8] In-Reply-To: <0TX1LHrwcqFB8uWK1K23a-4evh1jYT_ZnRF8CttweBk=.47ea26d9-b3a3-4550-8c64-474218b859e4@github.com> References: <0TX1LHrwcqFB8uWK1K23a-4evh1jYT_ZnRF8CttweBk=.47ea26d9-b3a3-4550-8c64-474218b859e4@github.com> Message-ID: On Wed, 19 Oct 2022 06:15:17 GMT, Maxim Kartashev wrote: >> src/java.base/macosx/classes/sun/nio/fs/MacOSXWatchService.java line 313: >> >>> 311: private static final long kFSEventStreamEventFlagRootChanged = 0x00000020; >>> 312: >>> 313: private static final Path relativeRootPath = theFS.getPath(""); >> >> Same comment as for lines 51-52. Unsure of line 311 however as that refers to a specific constant. Should that be perhaps in `UnixConstants`? > > I'll change the spelling of all but `kFSEventStreamEventFlagRootChanged`, which I'd rather keep as is. The constant being spelled exactly the same as in the corresponding C header file makes it easier to check its correctness (https://developer.apple.com/documentation/coreservices/1455361-fseventstreameventflags/kfseventstreameventflagrootchanged?language=objc). > As for moving it to `UnixConstants`, I'm also not so sure as it is hardly a "Unix" constant, more like a macOS constant. How about I make `kFSEventStreamEventFlagRootChanged` into a local variable? It's only used in one place and doesn't have to be a static field really. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From mkartashev at openjdk.org Wed Oct 19 06:48:05 2022 From: mkartashev at openjdk.org (Maxim Kartashev) Date: Wed, 19 Oct 2022 06:48:05 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v9] In-Reply-To: References: Message-ID: > This is an implementation of `WatchService` based on File System Events API that is capable of generating events whenever a change occurs in an interesting directory or underneath it. Since the API naturally supports "recursive" watch, the `FILE_TREE` is supported by the watch service. > > Some things of note: > * There's one "service" thread per `WatchService` instance that is inactive unless changes occur in the watched directory. The changes are grouped by introducing a time delay between when they occurred and when they are reported, which is controlled by the sensitivity modifier of the watch service. > * Since FSEvents API reports directories only, the watch service keeps a snapshot (hierarchical if necessary) of the files in the directory being watched. The snapshot gets updated when an event in that directory or underneath it gets delivered. File changes are detected by comparing "last modified" time with a millisecond precision (`BasicFileAttributes.lastModifiedTime()`). > * There is a slight complication with the move of an entire directory hierarchy: FSEvents API only reports about the containing directory of that move and not about any of the directories actually moved. There's a separate test for that (`Move.java`). > * The code is careful not to do any I/O (such as reading the contents of a directory or attributes of a file) unless unavoidable. Any deviation from this line should be considered a bug (of, arguably, low priority). > * The native part consists mostly of API wrappers with one exception of the callback function invoked by the system to report the events that occurred. The sole task of the function is to convert from C strings to Java strings and pass the array of affected directories to Java code. This can be rewritten if desired to make the code more future-proof. > > This commit leaves `PollingWatchService` unused. I'm not sure if I should/can do anything about it. Any advice is welcomed. > > ### Testing > > * Tested by running `test/jdk/java/nio/file` and `test/jdk/jdk/nio` on MacOS 10.15.7 (x64) and `test/jdk/java/nio/` plus `test/jdk/jdk/nio` on MacOS 12.5.1 (aarch64). > * Also verified that new tests pass on Linux and Windows. > * This code (albeit in a slightly modified form) has been in use at JetBrains for around half a year and a few bugs have been found and fixed during that time period. Maxim Kartashev has updated the pull request incrementally with one additional commit since the last revision: Corrected coding style ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10140/files - new: https://git.openjdk.org/jdk/pull/10140/files/b3d1604b..805c94f3 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10140&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10140&range=07-08 Stats: 25 lines in 2 files changed: 2 ins; 3 del; 20 mod Patch: https://git.openjdk.org/jdk/pull/10140.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10140/head:pull/10140 PR: https://git.openjdk.org/jdk/pull/10140 From alanb at openjdk.org Wed Oct 19 08:31:09 2022 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 19 Oct 2022 08:31:09 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v20] In-Reply-To: References: Message-ID: On Mon, 17 Oct 2022 17:12:08 GMT, Brian Burkhalter wrote: >> Resurrection of the proposal to add a method to obtain the filename extension originated in PR [2319](https://github.com/openjdk/jdk/pull/2319). > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8057113: Modify topic sentence of spec; remove obsolete import from test Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/8066 From alanb at openjdk.org Wed Oct 19 09:00:54 2022 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 19 Oct 2022 09:00:54 GMT Subject: RFR: 8295456: (ch) sun.nio.ch.Util::checkBufferPositionAligned gives misleading/incorrect error [v2] In-Reply-To: <7jpfRfuYGtBX6qfR3gMjeSeKSFc_ir-_KNUy8HTTJhY=.029af887-806b-4f8d-b6e0-c18a493c0682@github.com> References: <7jpfRfuYGtBX6qfR3gMjeSeKSFc_ir-_KNUy8HTTJhY=.029af887-806b-4f8d-b6e0-c18a493c0682@github.com> Message-ID: On Tue, 18 Oct 2022 23:49:57 GMT, Brian Burkhalter wrote: >> Improve `IOException` message in `sun.nio.ch.Util::checkBufferPositionAligned`. > > Brian Burkhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: > > 8295456: (ch) sun.nio.ch.Util::checkBufferPositionAligned gives misleading/incorrect error Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/10753 From bpb at openjdk.org Wed Oct 19 15:56:32 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 19 Oct 2022 15:56:32 GMT Subject: Integrated: 8295456: (ch) sun.nio.ch.Util::checkBufferPositionAligned gives misleading/incorrect error In-Reply-To: References: Message-ID: <0HjAZLy-Vdci-EGEp6m4gbprNCq1n8yuTOllW52x2c0=.833a7b39-7b8d-40c1-9893-e9cbf9386632@github.com> On Tue, 18 Oct 2022 23:22:28 GMT, Brian Burkhalter wrote: > Improve `IOException` message in `sun.nio.ch.Util::checkBufferPositionAligned`. This pull request has now been integrated. Changeset: a5f6e31c Author: Brian Burkhalter URL: https://git.openjdk.org/jdk/commit/a5f6e31ccbef27f4f363a80d41d5c9c1cae75151 Stats: 14 lines in 3 files changed: 1 ins; 3 del; 10 mod 8295456: (ch) sun.nio.ch.Util::checkBufferPositionAligned gives misleading/incorrect error Reviewed-by: alanb ------------- PR: https://git.openjdk.org/jdk/pull/10753 From bpb at openjdk.org Wed Oct 19 16:59:25 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 19 Oct 2022 16:59:25 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v8] In-Reply-To: References: Message-ID: <8WHg8QPgBC2O2sYDXXHnL3Y2nS20dlzyUxRjr5r4Ctg=.2b07af8d-cc32-4f4f-bc6b-e043e6df66f7@github.com> On Wed, 19 Oct 2022 05:54:56 GMT, Maxim Kartashev wrote: >> src/java.base/macosx/classes/sun/nio/fs/BsdFileSystem.java line 55: >> >>> 53: throws IOException >>> 54: { >>> 55: return new MacOSXWatchService(); >> >> Is it still intended that there be a system property to enable `MacOSXWatchService` which would be off by default? > > There was such a suggestion (not sure about "off by default", though). I guess this should go through CSR review, in which case I'm going to need some guidance. If you file a CSR then I can help with editing. >> src/java.base/macosx/classes/sun/nio/fs/MacOSXWatchService.java line 379: >> >>> 377: >>> 378: private Path toRelativePath(final String absPath) { >>> 379: return (absPath.length() > realRootPathLength) >> >> Nit: Are there several spaces after `return`? > > Yes, to align the beginning of the returned expression with its continuations on the subsequent two lines. OK ------------- PR: https://git.openjdk.org/jdk/pull/10140 From bpb at openjdk.org Wed Oct 19 16:59:25 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 19 Oct 2022 16:59:25 GMT Subject: RFR: 8293067: (fs) Implement WatchService using system library (macOS) [v8] In-Reply-To: References: <0TX1LHrwcqFB8uWK1K23a-4evh1jYT_ZnRF8CttweBk=.47ea26d9-b3a3-4550-8c64-474218b859e4@github.com> Message-ID: On Wed, 19 Oct 2022 06:23:50 GMT, Maxim Kartashev wrote: >> I'll change the spelling of all but `kFSEventStreamEventFlagRootChanged`, which I'd rather keep as is. The constant being spelled exactly the same as in the corresponding C header file makes it easier to check its correctness (https://developer.apple.com/documentation/coreservices/1455361-fseventstreameventflags/kfseventstreameventflagrootchanged?language=objc). >> As for moving it to `UnixConstants`, I'm also not so sure as it is hardly a "Unix" constant, more like a macOS constant. > > How about I make `kFSEventStreamEventFlagRootChanged` into a local variable? It's only used in one place and doesn't have to be a static field really. Sounds good. ------------- PR: https://git.openjdk.org/jdk/pull/10140 From djelinski at openjdk.org Mon Oct 24 12:44:35 2022 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 24 Oct 2022 12:44:35 GMT Subject: RFR: 8295798: (ch) Test java/nio/channels/Channels/ReadXBytes.java is very slow on Windows Message-ID: Please review this PR that reduces the number of bytes written by ReadXBytes test on machines supporting sparse files. Before this change the `createFileWithRandomContent` function would usually write either the last byte only, or the entire file (`long p` was usually set to length - 1). Now the function only writes the first and the last 32kB, and the remaining bytes are initialized in platform-defined manner. Before this change the test took between 1-10 minutes to complete on our Windows CI machines. After this change the test finishes in less than 1 minute. ------------- Commit messages: - Speed up file creation Changes: https://git.openjdk.org/jdk/pull/10834/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10834&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8295798 Stats: 22 lines in 1 file changed: 5 ins; 0 del; 17 mod Patch: https://git.openjdk.org/jdk/pull/10834.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10834/head:pull/10834 PR: https://git.openjdk.org/jdk/pull/10834 From alanb at openjdk.org Mon Oct 24 13:13:53 2022 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 24 Oct 2022 13:13:53 GMT Subject: RFR: 8295798: (ch) Test java/nio/channels/Channels/ReadXBytes.java is very slow on Windows In-Reply-To: References: Message-ID: On Mon, 24 Oct 2022 12:27:51 GMT, Daniel Jeli?ski wrote: > Please review this PR that reduces the number of bytes written by ReadXBytes test on machines supporting sparse files. > > Before this change the `createFileWithRandomContent` function would usually write either the last byte only, or the entire file (`long p` was usually set to length - 1). Now the function only writes the first and the last 32kB, and the remaining bytes are initialized in platform-defined manner. > > Before this change the test took between 1-10 minutes to complete on our Windows CI machines. > After this change the test finishes in less than 1 minute. This looks good, I assume you'll update the end date in the header before you integrate. ------------- Marked as reviewed by alanb (Reviewer). PR: https://git.openjdk.org/jdk/pull/10834 From djelinski at openjdk.org Mon Oct 24 15:27:47 2022 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 24 Oct 2022 15:27:47 GMT Subject: RFR: 8295798: (ch) Test java/nio/channels/Channels/ReadXBytes.java is very slow on Windows [v2] In-Reply-To: References: Message-ID: > Please review this PR that reduces the number of bytes written by ReadXBytes test on machines supporting sparse files. > > Before this change the `createFileWithRandomContent` function would usually write either the last byte only, or the entire file (`long p` was usually set to length - 1). Now the function only writes the first and the last 32kB, and the remaining bytes are initialized in platform-defined manner. > > Before this change the test took between 1-10 minutes to complete on our Windows CI machines. > After this change the test finishes in less than 1 minute. Daniel Jeli?ski has updated the pull request incrementally with one additional commit since the last revision: Update copyright ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10834/files - new: https://git.openjdk.org/jdk/pull/10834/files/70f4b4bf..f5229267 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10834&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10834&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/10834.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10834/head:pull/10834 PR: https://git.openjdk.org/jdk/pull/10834 From bpb at openjdk.org Mon Oct 24 16:40:52 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 24 Oct 2022 16:40:52 GMT Subject: RFR: 8295798: (ch) Test java/nio/channels/Channels/ReadXBytes.java is very slow on Windows [v2] In-Reply-To: References: Message-ID: On Mon, 24 Oct 2022 15:27:47 GMT, Daniel Jeli?ski wrote: >> Please review this PR that reduces the number of bytes written by ReadXBytes test on machines supporting sparse files. >> >> Before this change the `createFileWithRandomContent` function would usually write either the last byte only, or the entire file (`long p` was usually set to length - 1). Now the function only writes the first and the last 32kB, and the remaining bytes are initialized in platform-defined manner. >> >> Before this change the test took between 1-10 minutes to complete on our Windows CI machines. >> After this change the test finishes in less than 1 minute. > > Daniel Jeli?ski has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright Looks good, thanks. ------------- Marked as reviewed by bpb (Reviewer). PR: https://git.openjdk.org/jdk/pull/10834 From bpb at openjdk.org Mon Oct 24 23:08:13 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 24 Oct 2022 23:08:13 GMT Subject: RFR: 8295753: (fs) UnixPath::toRealPath does not return correct case when links not followed Message-ID: <6WGVJ1L_0aM7tp0LgFMbZy_L4bJ7Gr80mlg0ZKk9piI=.6fa46e68-3ae7-4582-a651-8b0b1d259aa1@github.com> On macOS, ensure that the components of the `Path` returned by `UnixPath::toRealPath` have the case retained by the file system as opposed to that of the representation present in the `UnixPath` instance. ------------- Commit messages: - 8295753: (fs) UnixPath::toRealPath does not return correct case when links not followed Changes: https://git.openjdk.org/jdk/pull/10843/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10843&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8295753 Stats: 108 lines in 4 files changed: 93 ins; 2 del; 13 mod Patch: https://git.openjdk.org/jdk/pull/10843.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10843/head:pull/10843 PR: https://git.openjdk.org/jdk/pull/10843 From prappo at openjdk.org Mon Oct 24 23:35:54 2022 From: prappo at openjdk.org (Pavel Rappo) Date: Mon, 24 Oct 2022 23:35:54 GMT Subject: RFR: 8295753: (fs) UnixPath::toRealPath does not return correct case when links not followed In-Reply-To: <6WGVJ1L_0aM7tp0LgFMbZy_L4bJ7Gr80mlg0ZKk9piI=.6fa46e68-3ae7-4582-a651-8b0b1d259aa1@github.com> References: <6WGVJ1L_0aM7tp0LgFMbZy_L4bJ7Gr80mlg0ZKk9piI=.6fa46e68-3ae7-4582-a651-8b0b1d259aa1@github.com> Message-ID: On Mon, 24 Oct 2022 22:58:10 GMT, Brian Burkhalter wrote: > On macOS, ensure that the components of the `Path` returned by `UnixPath::toRealPath` have the case retained by the file system as opposed to that of the representation present in the `UnixPath` instance. Not a review, just thanks for addressing that issue, and double thanks for addressing it promptly. That issue was recently implicated in incorrect element lookup in jdk.javadoc. I can see that you also linked that javadoc issue to this PR's issue in JBS. So thanks again! ------------- PR: https://git.openjdk.org/jdk/pull/10843 From alanb at openjdk.org Tue Oct 25 06:17:37 2022 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 25 Oct 2022 06:17:37 GMT Subject: RFR: 8295753: (fs) UnixPath::toRealPath does not return correct case when links not followed In-Reply-To: <6WGVJ1L_0aM7tp0LgFMbZy_L4bJ7Gr80mlg0ZKk9piI=.6fa46e68-3ae7-4582-a651-8b0b1d259aa1@github.com> References: <6WGVJ1L_0aM7tp0LgFMbZy_L4bJ7Gr80mlg0ZKk9piI=.6fa46e68-3ae7-4582-a651-8b0b1d259aa1@github.com> Message-ID: On Mon, 24 Oct 2022 22:58:10 GMT, Brian Burkhalter wrote: > On macOS, ensure that the components of the `Path` returned by `UnixPath::toRealPath` have the case retained by the file system as opposed to that of the representation present in the `UnixPath` instance. src/java.base/unix/classes/sun/nio/fs/UnixPath.java line 893: > 891: // to the path constructed thus far, and extract the entry whose > 892: // name is equal ignoring case to the name of the current element > 893: try (Stream entries = Files.list(path)) { One initial comment on this is that you can't use Files.list in the provider implementation because the default provider can be configured to be something else. Also the security manager is still a supported execution mode so we'll need to work through the issues too. ------------- PR: https://git.openjdk.org/jdk/pull/10843 From djelinski at openjdk.org Tue Oct 25 07:26:42 2022 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Tue, 25 Oct 2022 07:26:42 GMT Subject: RFR: 8295798: (ch) Test java/nio/channels/Channels/ReadXBytes.java is very slow on Windows [v2] In-Reply-To: References: Message-ID: <5z44tZLwkXswwu8pv7bbqSWzQQfmgxElltscbNqXCyA=.43707482-5542-4c62-a89e-2da4c6c081bc@github.com> On Mon, 24 Oct 2022 15:27:47 GMT, Daniel Jeli?ski wrote: >> Please review this PR that reduces the number of bytes written by ReadXBytes test on machines supporting sparse files. >> >> Before this change the `createFileWithRandomContent` function would usually write either the last byte only, or the entire file (`long p` was usually set to length - 1). Now the function only writes the first and the last 32kB, and the remaining bytes are initialized in platform-defined manner. >> >> Before this change the test took between 1-10 minutes to complete on our Windows CI machines. >> After this change the test finishes in less than 1 minute. > > Daniel Jeli?ski has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright Thanks for the reviews! ------------- PR: https://git.openjdk.org/jdk/pull/10834 From djelinski at openjdk.org Tue Oct 25 07:28:11 2022 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Tue, 25 Oct 2022 07:28:11 GMT Subject: Integrated: 8295798: (ch) Test java/nio/channels/Channels/ReadXBytes.java is very slow on Windows In-Reply-To: References: Message-ID: On Mon, 24 Oct 2022 12:27:51 GMT, Daniel Jeli?ski wrote: > Please review this PR that reduces the number of bytes written by ReadXBytes test on machines supporting sparse files. > > Before this change the `createFileWithRandomContent` function would usually write either the last byte only, or the entire file (`long p` was usually set to length - 1). Now the function only writes the first and the last 32kB, and the remaining bytes are initialized in platform-defined manner. > > Before this change the test took between 1-10 minutes to complete on our Windows CI machines. > After this change the test finishes in less than 1 minute. This pull request has now been integrated. Changeset: 706d1b75 Author: Daniel Jeli?ski URL: https://git.openjdk.org/jdk/commit/706d1b7508af58f7ed669bde4820ec9f80f58607 Stats: 23 lines in 1 file changed: 5 ins; 0 del; 18 mod 8295798: (ch) Test java/nio/channels/Channels/ReadXBytes.java is very slow on Windows Reviewed-by: alanb, bpb ------------- PR: https://git.openjdk.org/jdk/pull/10834 From duke at openjdk.org Tue Oct 25 12:43:12 2022 From: duke at openjdk.org (Darragh Clarke) Date: Tue, 25 Oct 2022 12:43:12 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" Message-ID: Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. ------------- Commit messages: - cleanup - added extra logging incase of test failures Changes: https://git.openjdk.org/jdk/pull/10851/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10851&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8293696 Stats: 28 lines in 1 file changed: 25 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/10851.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10851/head:pull/10851 PR: https://git.openjdk.org/jdk/pull/10851 From msheppar at openjdk.org Tue Oct 25 13:03:19 2022 From: msheppar at openjdk.org (Mark Sheppard) Date: Tue, 25 Oct 2022 13:03:19 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" In-Reply-To: References: Message-ID: On Tue, 25 Oct 2022 12:35:15 GMT, Darragh Clarke wrote: > Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. > > I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 25: > 23: > 24: /* @test > 25: * @bug 6935563 7044870 8293696 I think we can omit the bug number in this instance as it is not at this point in time a product bug -- as you are adding diagnostics to the test, maybe a separate subtask or sub-bug could have been used ------------- PR: https://git.openjdk.org/jdk/pull/10851 From alanb at openjdk.org Tue Oct 25 13:06:10 2022 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 25 Oct 2022 13:06:10 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" In-Reply-To: References: Message-ID: On Tue, 25 Oct 2022 12:35:15 GMT, Darragh Clarke wrote: > Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. > > I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 114: > 112: if (!key.isValid() || !key.isReadable()) { > 113: System.out.println("Invalid or unreadable key: " + key); > 114: return; The loop over the selection keys and checking if valid or readable doesn't make sense here. Maybe that can be just removed. If you really want to print out the selection key then save it at L52 so that you have the SelectionKey to print. test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 119: > 117: System.out.println("Attempting to read datagram from key: " + key); > 118: ByteBuffer buf = ByteBuffer.allocate(100); > 119: SocketAddress sa = dc.receive(buf); The datagram channel is configured non-blocking and there may not be a datagram to receive, therefore sa may be null and has to be checked here. ------------- PR: https://git.openjdk.org/jdk/pull/10851 From msheppar at openjdk.org Tue Oct 25 13:23:49 2022 From: msheppar at openjdk.org (Mark Sheppard) Date: Tue, 25 Oct 2022 13:23:49 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" In-Reply-To: References: Message-ID: <4jpQXWVh0-X3bSHOgUpn6DCuc4hIpRhGd379g2MuZXA=.b260d26a-7cf3-473c-a01b-4e597424c4c8@github.com> On Tue, 25 Oct 2022 12:35:15 GMT, Darragh Clarke wrote: > Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. > > I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 57: > 55: sendDatagram(dc, refuser); > 56: int n = sel.select(2000); > 57: if (n > 0) { This has been seen as the location of the intermittent failures, so maybe adding a receive here also to determine the src of the unexpected datagram would be informative test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 77: > 75: n = dc.read(buf); > 76: String message = new String(buf.array()); > 77: System.out.format("received %s at %s%n", message, dc.getLocalAddress()); The intermittent failure is seen on the previous test scenario, so some diagnostics relating to that are of interest I think we are interested in the sender detail here ... where did the stray datagram come from, which would necessitate invoking receive on the datagram channel ... but as of yet there has been no failures observed here afaik Also prior to each DC send it could be worth invoking selectNow on the selector to disperse any stray pending events on the low level OS "select" ------------- PR: https://git.openjdk.org/jdk/pull/10851 From msheppar at openjdk.org Tue Oct 25 13:26:38 2022 From: msheppar at openjdk.org (Mark Sheppard) Date: Tue, 25 Oct 2022 13:26:38 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" In-Reply-To: References: Message-ID: On Tue, 25 Oct 2022 12:35:15 GMT, Darragh Clarke wrote: > Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. > > I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. i see the logUnexpectedWakeup now ... missed that in my first pass :-( ------------- PR: https://git.openjdk.org/jdk/pull/10851 From dfuchs at openjdk.org Tue Oct 25 14:19:39 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Tue, 25 Oct 2022 14:19:39 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" In-Reply-To: References: Message-ID: <7yqVFHvgnrFp_FYADWkStUCNQgroIhOTopK1dssTjk4=.5cb62195-5638-4516-be04-e7a6aab5d849@github.com> On Tue, 25 Oct 2022 12:35:15 GMT, Darragh Clarke wrote: > Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. > > I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. One improvement that could be done is put the whole code from line 55 to line 94 into a loop and respin if we get a datagram that doesn't correspond to what was sent to the refuser. It could be that we are receiving a datagram from something/somewhere else in which case we could just ignore it, clear all keys, and go back to line 55. Otherwise throw if we detected an error or break the loop if everything occurred as expected. ------------- PR: https://git.openjdk.org/jdk/pull/10851 From dfuchs at openjdk.org Tue Oct 25 14:19:42 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Tue, 25 Oct 2022 14:19:42 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" In-Reply-To: References: Message-ID: On Tue, 25 Oct 2022 13:01:11 GMT, Alan Bateman wrote: >> Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. >> >> I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. > > test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 114: > >> 112: if (!key.isValid() || !key.isReadable()) { >> 113: System.out.println("Invalid or unreadable key: " + key); >> 114: return; > > The loop over the selection keys and checking if valid or readable doesn't make sense here. Maybe that can be just removed. If you really want to print out the selection key then save it at L52 so that you have the SelectionKey to print. Since we have no idea of what is going on it's fine to loop over the set of selected keys - even if it should contain at most one entry. But if it contains a key that is either not valid or not readable we should continue after printing it, not return. > test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 119: > >> 117: System.out.println("Attempting to read datagram from key: " + key); >> 118: ByteBuffer buf = ByteBuffer.allocate(100); >> 119: SocketAddress sa = dc.receive(buf); > > The datagram channel is configured non-blocking and there may not be a datagram to receive, therefore sa may be null and has to be checked here. @AlanBateman If we reach here the key is both valid and readable so shouldn't there be something to receive, even if the channel is non-blocking? ------------- PR: https://git.openjdk.org/jdk/pull/10851 From alanb at openjdk.org Tue Oct 25 14:36:49 2022 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 25 Oct 2022 14:36:49 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" In-Reply-To: References: Message-ID: On Tue, 25 Oct 2022 12:35:15 GMT, Darragh Clarke wrote: > Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. > > I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. > @AlanBateman If we reach here the key is both valid and readable so shouldn't there be something to receive, even if the channel is non-blocking? It's okay to print the selection key (singular). The useful part of the change is to do the non-blocking receive and print the sender's socket address when not null. If it trips then it may help identify where the datagram is coming from. That is, I assume the theory here is that some other test is using the "refuser" port (the port that dc1.closer released). The alternative is to just retry when there is interference, which I think you were suggesting in one of the comments. ------------- PR: https://git.openjdk.org/jdk/pull/10851 From bpb at openjdk.org Tue Oct 25 15:42:55 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 25 Oct 2022 15:42:55 GMT Subject: RFR: 8295753: (fs) UnixPath::toRealPath does not return correct case when links not followed In-Reply-To: References: <6WGVJ1L_0aM7tp0LgFMbZy_L4bJ7Gr80mlg0ZKk9piI=.6fa46e68-3ae7-4582-a651-8b0b1d259aa1@github.com> Message-ID: On Tue, 25 Oct 2022 06:13:45 GMT, Alan Bateman wrote: > One initial comment on this is that you can't use Files.list in the provider implementation because the default provider can be configured to be something else. Yes, I knew that, but there is no equivalent in the provider. I guess `java.io.File::list` would work. ------------- PR: https://git.openjdk.org/jdk/pull/10843 From alanb at openjdk.org Tue Oct 25 15:54:33 2022 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 25 Oct 2022 15:54:33 GMT Subject: RFR: 8295753: (fs) UnixPath::toRealPath does not return correct case when links not followed In-Reply-To: References: <6WGVJ1L_0aM7tp0LgFMbZy_L4bJ7Gr80mlg0ZKk9piI=.6fa46e68-3ae7-4582-a651-8b0b1d259aa1@github.com> Message-ID: On Tue, 25 Oct 2022 15:39:20 GMT, Brian Burkhalter wrote: >> src/java.base/unix/classes/sun/nio/fs/UnixPath.java line 893: >> >>> 891: // to the path constructed thus far, and extract the entry whose >>> 892: // name is equal ignoring case to the name of the current element >>> 893: try (Stream entries = Files.list(path)) { >> >> One initial comment on this is that you can't use Files.list in the provider implementation because the default provider can be configured to be something else. Also the security manager is still a supported execution mode so we'll need to work through the issues too. > >> One initial comment on this is that you can't use Files.list in the provider implementation because the default provider can be configured to be something else. > > Yes, I knew that, but there is no equivalent in the provider. I guess `java.io.File::list` would work. I think you'll have to change this to use UnixDirectoryStream here, it shouldn't be too hard. We'll also need to study the case check as I suspect that will need macOS specific code. ------------- PR: https://git.openjdk.org/jdk/pull/10843 From bpb at openjdk.org Tue Oct 25 16:00:36 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 25 Oct 2022 16:00:36 GMT Subject: RFR: 8294399: (ch) Refactor some methods out of sun.nio.ch.UnixFileDispatcherImpl [v3] In-Reply-To: References: Message-ID: > Interpose a new class `sun.nio.ch.UnixDispatcher` between `sun.nio.ch.NativeDispatcher` and its current subclasses `sun.nio.ch.DatagramDispatcher` and `sun.nio.ch.SocketDispatcher`; refactor some methods out of `sun.nio.ch.UnixFileDispatcherImpl` into this updated hierarchy. Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - Merge - 8294399: Correct copyright years - 8294399: (ch) Refactor some methods out of sun.nio.ch.UnixFileDispatcherImpl ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10434/files - new: https://git.openjdk.org/jdk/pull/10434/files/a16bc10a..b66ff40e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10434&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10434&range=01-02 Stats: 172362 lines in 2322 files changed: 112715 ins; 22013 del; 37634 mod Patch: https://git.openjdk.org/jdk/pull/10434.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10434/head:pull/10434 PR: https://git.openjdk.org/jdk/pull/10434 From bpb at openjdk.org Tue Oct 25 16:00:39 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 25 Oct 2022 16:00:39 GMT Subject: RFR: 8294399: (ch) Refactor some methods out of sun.nio.ch.UnixFileDispatcherImpl [v2] In-Reply-To: References: Message-ID: <7NK3E_tstcj_vRbwZwuYwyHQZXlm085u8D4nnil5CU8=.44f88601-2d07-4f5e-a658-337a4e0718ff@github.com> On Mon, 26 Sep 2022 22:17:23 GMT, Brian Burkhalter wrote: >> Interpose a new class `sun.nio.ch.UnixDispatcher` between `sun.nio.ch.NativeDispatcher` and its current subclasses `sun.nio.ch.DatagramDispatcher` and `sun.nio.ch.SocketDispatcher`; refactor some methods out of `sun.nio.ch.UnixFileDispatcherImpl` into this updated hierarchy. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8294399: Correct copyright years Ping. ------------- PR: https://git.openjdk.org/jdk/pull/10434 From bpb at openjdk.org Tue Oct 25 23:36:27 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Tue, 25 Oct 2022 23:36:27 GMT Subject: RFR: 8295753: (fs) UnixPath::toRealPath does not return correct case when links not followed [v2] In-Reply-To: <6WGVJ1L_0aM7tp0LgFMbZy_L4bJ7Gr80mlg0ZKk9piI=.6fa46e68-3ae7-4582-a651-8b0b1d259aa1@github.com> References: <6WGVJ1L_0aM7tp0LgFMbZy_L4bJ7Gr80mlg0ZKk9piI=.6fa46e68-3ae7-4582-a651-8b0b1d259aa1@github.com> Message-ID: > On macOS, ensure that the components of the `Path` returned by `UnixPath::toRealPath` have the case retained by the file system as opposed to that of the representation present in the `UnixPath` instance. Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - Merge - 8295753: Use UnixDirectoryStream; compare UnixFileKeys; add SecurityManager - 8295753: (fs) UnixPath::toRealPath does not return correct case when links not followed ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10843/files - new: https://git.openjdk.org/jdk/pull/10843/files/1eb3912f..96c0257a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10843&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10843&range=00-01 Stats: 142136 lines in 855 files changed: 82369 ins; 29444 del; 30323 mod Patch: https://git.openjdk.org/jdk/pull/10843.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10843/head:pull/10843 PR: https://git.openjdk.org/jdk/pull/10843 From alanb at openjdk.org Wed Oct 26 19:25:31 2022 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 26 Oct 2022 19:25:31 GMT Subject: RFR: 8294399: (ch) Refactor some methods out of sun.nio.ch.UnixFileDispatcherImpl [v3] In-Reply-To: References: Message-ID: On Tue, 25 Oct 2022 16:00:36 GMT, Brian Burkhalter wrote: >> Interpose a new class `sun.nio.ch.UnixDispatcher` between `sun.nio.ch.NativeDispatcher` and its current subclasses `sun.nio.ch.DatagramDispatcher` and `sun.nio.ch.SocketDispatcher`; refactor some methods out of `sun.nio.ch.UnixFileDispatcherImpl` into this updated hierarchy. > > Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - Merge > - 8294399: Correct copyright years > - 8294399: (ch) Refactor some methods out of sun.nio.ch.UnixFileDispatcherImpl Marked as reviewed by alanb (Reviewer). src/java.base/unix/native/libnio/ch/UnixDispatcher.c line 68: > 66: } > 67: preCloseFD = sp[0]; > 68: close(sp[1]); This looks okay, just surprising to see the init methods at the end of the .c files when the previous convention had them before the other JNI methods. ------------- PR: https://git.openjdk.org/jdk/pull/10434 From bpb at openjdk.org Wed Oct 26 19:42:43 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 26 Oct 2022 19:42:43 GMT Subject: RFR: 8294399: (ch) Refactor some methods out of sun.nio.ch.UnixFileDispatcherImpl [v4] In-Reply-To: References: Message-ID: > Interpose a new class `sun.nio.ch.UnixDispatcher` between `sun.nio.ch.NativeDispatcher` and its current subclasses `sun.nio.ch.DatagramDispatcher` and `sun.nio.ch.SocketDispatcher`; refactor some methods out of `sun.nio.ch.UnixFileDispatcherImpl` into this updated hierarchy. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8294399: Move UnixDispatcher_init() before the other JNI methods ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10434/files - new: https://git.openjdk.org/jdk/pull/10434/files/b66ff40e..ab83200d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10434&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10434&range=02-03 Stats: 24 lines in 1 file changed: 12 ins; 12 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/10434.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10434/head:pull/10434 PR: https://git.openjdk.org/jdk/pull/10434 From bpb at openjdk.org Wed Oct 26 19:42:47 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Wed, 26 Oct 2022 19:42:47 GMT Subject: RFR: 8294399: (ch) Refactor some methods out of sun.nio.ch.UnixFileDispatcherImpl [v3] In-Reply-To: References: Message-ID: On Wed, 26 Oct 2022 19:22:57 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: >> >> - Merge >> - 8294399: Correct copyright years >> - 8294399: (ch) Refactor some methods out of sun.nio.ch.UnixFileDispatcherImpl > > src/java.base/unix/native/libnio/ch/UnixDispatcher.c line 68: > >> 66: } >> 67: preCloseFD = sp[0]; >> 68: close(sp[1]); > > This looks okay, just surprising to see the init methods at the end of the .c files when the previous convention had them before the other JNI methods. Fixed in ab83200d83ce853e45ee6b6addb7f4b1d400d53f. ------------- PR: https://git.openjdk.org/jdk/pull/10434 From alanb at openjdk.org Wed Oct 26 19:58:27 2022 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 26 Oct 2022 19:58:27 GMT Subject: RFR: 8294399: (ch) Refactor some methods out of sun.nio.ch.UnixFileDispatcherImpl [v4] In-Reply-To: References: Message-ID: <-k1-x6aFZN9o0AT4I6fleC-Zmudb-3JDHOn2WWj5ofY=.edc82154-e36d-4b26-bdbb-9975d7a6be80@github.com> On Wed, 26 Oct 2022 19:42:43 GMT, Brian Burkhalter wrote: >> Interpose a new class `sun.nio.ch.UnixDispatcher` between `sun.nio.ch.NativeDispatcher` and its current subclasses `sun.nio.ch.DatagramDispatcher` and `sun.nio.ch.SocketDispatcher`; refactor some methods out of `sun.nio.ch.UnixFileDispatcherImpl` into this updated hierarchy. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8294399: Move UnixDispatcher_init() before the other JNI methods Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/10434 From duke at openjdk.org Thu Oct 27 09:34:24 2022 From: duke at openjdk.org (Darragh Clarke) Date: Thu, 27 Oct 2022 09:34:24 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" In-Reply-To: <7yqVFHvgnrFp_FYADWkStUCNQgroIhOTopK1dssTjk4=.5cb62195-5638-4516-be04-e7a6aab5d849@github.com> References: <7yqVFHvgnrFp_FYADWkStUCNQgroIhOTopK1dssTjk4=.5cb62195-5638-4516-be04-e7a6aab5d849@github.com> Message-ID: On Tue, 25 Oct 2022 14:14:05 GMT, Daniel Fuchs wrote: > One improvement that could be done is put the whole code from line 55 to line 94 into a loop and respin if we get a datagram that doesn't correspond to what was sent to the refuser. It could be that we are receiving a datagram from something/somewhere else in which case we could just ignore it, clear all keys, and go back to line 55. Otherwise throw if we detected an error or break the loop if everything occurred as expected. I think that this would be good, but because of the nature of the refuser, would we be reliably able to read datagrams sent, since they should be getting refused and throwing an exception before they can be read? If they can be read, I could change the logUnexpectedWakeup method to return a boolean for whether its local or not since it attempts to read it already ------------- PR: https://git.openjdk.org/jdk/pull/10851 From dfuchs at openjdk.org Thu Oct 27 10:44:28 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 27 Oct 2022 10:44:28 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" In-Reply-To: References: <7yqVFHvgnrFp_FYADWkStUCNQgroIhOTopK1dssTjk4=.5cb62195-5638-4516-be04-e7a6aab5d849@github.com> Message-ID: On Thu, 27 Oct 2022 09:32:18 GMT, Darragh Clarke wrote: > I think that this would be good, but because of the nature of the refuser, would we be reliably able to read datagrams sent, since they should be getting refused and throwing an exception before they can be read? > > If they can be read, I could change the logUnexpectedWakeup method to return a boolean for whether its local or not since it attempts to read it already Yes - that's a good idea - and maybe change the method name to "checkUnexpectedWakeup" if you change it to return a boolean. ------------- PR: https://git.openjdk.org/jdk/pull/10851 From bpb at openjdk.org Thu Oct 27 15:39:38 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 27 Oct 2022 15:39:38 GMT Subject: Integrated: 8294399: (ch) Refactor some methods out of sun.nio.ch.UnixFileDispatcherImpl In-Reply-To: References: Message-ID: On Mon, 26 Sep 2022 21:48:57 GMT, Brian Burkhalter wrote: > Interpose a new class `sun.nio.ch.UnixDispatcher` between `sun.nio.ch.NativeDispatcher` and its current subclasses `sun.nio.ch.DatagramDispatcher` and `sun.nio.ch.SocketDispatcher`; refactor some methods out of `sun.nio.ch.UnixFileDispatcherImpl` into this updated hierarchy. This pull request has now been integrated. Changeset: d6678952 Author: Brian Burkhalter URL: https://git.openjdk.org/jdk/commit/d6678952a6de4e5435dab65e7029021832454857 Stats: 231 lines in 9 files changed: 158 ins; 60 del; 13 mod 8294399: (ch) Refactor some methods out of sun.nio.ch.UnixFileDispatcherImpl Reviewed-by: alanb ------------- PR: https://git.openjdk.org/jdk/pull/10434 From bpb at openjdk.org Thu Oct 27 15:59:20 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 27 Oct 2022 15:59:20 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) Message-ID: On macOS, perform file name normalization if and only if a specific system property is set. ------------- Commit messages: - 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) Changes: https://git.openjdk.org/jdk/pull/10885/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10885&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8289689 Stats: 41 lines in 3 files changed: 26 ins; 0 del; 15 mod Patch: https://git.openjdk.org/jdk/pull/10885.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10885/head:pull/10885 PR: https://git.openjdk.org/jdk/pull/10885 From bpb at openjdk.org Thu Oct 27 15:59:21 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Thu, 27 Oct 2022 15:59:21 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 15:50:56 GMT, Brian Burkhalter wrote: > On macOS, perform file name normalization if and only if a specific system property is set. If the need for the new property is accepted, then a CSR will be filed. ------------- PR: https://git.openjdk.org/jdk/pull/10885 From duke at openjdk.org Thu Oct 27 16:28:29 2022 From: duke at openjdk.org (Darragh Clarke) Date: Thu, 27 Oct 2022 16:28:29 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: > Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. > > I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. Darragh Clarke has updated the pull request incrementally with one additional commit since the last revision: added loop to retry in event of interruption from other tests ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10851/files - new: https://git.openjdk.org/jdk/pull/10851/files/d5a4fd8a..795b073e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10851&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10851&range=00-01 Stats: 76 lines in 1 file changed: 39 ins; 16 del; 21 mod Patch: https://git.openjdk.org/jdk/pull/10851.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10851/head:pull/10851 PR: https://git.openjdk.org/jdk/pull/10851 From msheppar at openjdk.org Thu Oct 27 16:48:30 2022 From: msheppar at openjdk.org (Mark Sheppard) Date: Thu, 27 Oct 2022 16:48:30 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 16:28:29 GMT, Darragh Clarke wrote: >> Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. >> >> I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. > > Darragh Clarke has updated the pull request incrementally with one additional commit since the last revision: > > added loop to retry in event of interruption from other tests I think before making any majory changes to the test's execution in terms of respin etc, the focus should be to add diagnostics to the test, to determine if there is actualy any data available and the origin of that unexpected data. Additionally, to check the verascity of the returned values from the selector, to esnure that the select has behaved in a consistent and reliable manner and that it has not responded to indeterminate intermittent input (or other) event. The test is open to ta race condition in that the "refuser" port could have been resused by another socket in another concurrent test, after the DC1 has closed -- this condition is tested in the test through a bind to refuser port, which is the failed scenarios has not rasied a BindException. But again timing is of the essence. I would suggest the initial focus for the changes should be for diagnostics of the failed scenario, and determing whether the selector returns actually reflect some input read event and not another spurious "low level event trigger". In the test the SelectionKey from the DC::register is not recorded, and not compared with the SelectionKey set returned by the selector. It would be informative if the number of keys returned by the select could be output. Currently in the test or in the updated logUnexpectedWakeup there is no check that the SelectioKey associated with the registered DC is actually returned. It is assumed that this is the case only a single channel has been registered, which is reasonable, but if consistency is to be deteremined then such a check may be worthwhile. Additionally, the registration is for READ events. Again, it may be worth checking that the select has returned SelectioKey for OP_READ and not something else. ------------- PR: https://git.openjdk.org/jdk/pull/10851 From dfuchs at openjdk.org Thu Oct 27 16:58:50 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 27 Oct 2022 16:58:50 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 16:46:28 GMT, Mark Sheppard wrote: > It would be informative if the number of keys returned by the select could be output. I believe that Alan already implied that this number could only take two values: 0 or 1, because the selector has only one key. ------------- PR: https://git.openjdk.org/jdk/pull/10851 From msheppar at openjdk.org Thu Oct 27 17:11:23 2022 From: msheppar at openjdk.org (Mark Sheppard) Date: Thu, 27 Oct 2022 17:11:23 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 16:28:29 GMT, Darragh Clarke wrote: >> Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. >> >> I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. > > Darragh Clarke has updated the pull request incrementally with one additional commit since the last revision: > > added loop to retry in event of interruption from other tests yes that's fine, but in this instance, something unexpected has ocurred then why not validate that those assumptions are true ------------- PR: https://git.openjdk.org/jdk/pull/10851 From dfuchs at openjdk.org Thu Oct 27 17:32:27 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 27 Oct 2022 17:32:27 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: <3cbufag44OS3PF2iKoHiSwTN4b_RCHGWBLp47m5iNDU=.6d25b683-ff53-44e6-908f-870a45826f08@github.com> On Thu, 27 Oct 2022 16:28:29 GMT, Darragh Clarke wrote: >> Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. >> >> I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. > > Darragh Clarke has updated the pull request incrementally with one additional commit since the last revision: > > added loop to retry in event of interruption from other tests test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 83: > 81: try { > 82: ByteBuffer buf = ByteBuffer.allocate(100); > 83: SocketAddress sa = dc.receive(buf); This changes the test to use receive instead of read. I am not sure we want to do that. It's true that if we use read we won't be able to tell where the message comes from - but we're connected, so in theory it could only come from the refuser address, which would indicate that some other process has reused the port. So it should be enough to call read() and print the content of the received buffer. @AlanBateman can we use receive() here instead of read() without changing the nature of the test? ------------- PR: https://git.openjdk.org/jdk/pull/10851 From dfuchs at openjdk.org Thu Oct 27 18:07:29 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 27 Oct 2022 18:07:29 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 16:46:28 GMT, Mark Sheppard wrote: > I would suggest the initial focus for the changes should be for diagnostics of the failed scenario, and determing whether the selector returns actually reflect some input read event and not another spurious "low level event trigger". Given the intermittent nature of the failure I'd rather do both at the same time. If we do receive a datagram, when we expect none, and we do verify that the datagram doesn't come from this test (it doesn't contain "Greetings!") then I'd argue that it must come from somewhere else (whatever that somewhere is) and it's safe to respin and try again. There might be some subtle issues to fix in the retry logic though. I believe we should respin when getting unexpected wake ups only if we actually manage to read a datagram - for instance, and that we can assert that this datagram is not ours (and it should never be, since we sent it to a refuser that has a different port than our receiver). We should fail in all other cases. ------------- PR: https://git.openjdk.org/jdk/pull/10851 From dfuchs at openjdk.org Thu Oct 27 18:07:31 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 27 Oct 2022 18:07:31 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 16:28:29 GMT, Darragh Clarke wrote: >> Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. >> >> I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. > > Darragh Clarke has updated the pull request incrementally with one additional commit since the last revision: > > added loop to retry in event of interruption from other tests It's also interesting in that if other DC/DS tests are running in parallel on the same machine they could be reusing the refuser port, and they could receive the datagram we sent to the refuser, and that may make these other tests fail too! ------------- PR: https://git.openjdk.org/jdk/pull/10851 From msheppar at openjdk.org Thu Oct 27 18:07:32 2022 From: msheppar at openjdk.org (Mark Sheppard) Date: Thu, 27 Oct 2022 18:07:32 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 16:28:29 GMT, Darragh Clarke wrote: >> Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. >> >> I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. > > Darragh Clarke has updated the pull request incrementally with one additional commit since the last revision: > > added loop to retry in event of interruption from other tests test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 58: > 56: dc.register(sel, SelectionKey.OP_READ); > 57: > 58: for (;;) { treat each test scenario individually wrt retry and set an upper bound on the number of potential retries ? test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 65: > 63: ignoreStrayWakeup = checkUnexpectedWakeup(dc, sel.selectedKeys()); > 64: sel.selectedKeys().clear(); > 65: if (ignoreStrayWakeup) { scenario 1 and 3 treat the received message differently to that of scenario 2 -- should they be handled the same ? ------------- PR: https://git.openjdk.org/jdk/pull/10851 From dfuchs at openjdk.org Thu Oct 27 18:22:35 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 27 Oct 2022 18:22:35 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 18:02:16 GMT, Mark Sheppard wrote: >> Darragh Clarke has updated the pull request incrementally with one additional commit since the last revision: >> >> added loop to retry in event of interruption from other tests > > test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 58: > >> 56: dc.register(sel, SelectionKey.OP_READ); >> 57: >> 58: for (;;) { > > treat each test scenario individually wrt retry and set an upper bound on the number of potential retries ? Having an upper bound - e.g. 3 and failing if we reach the upper bound would be prudent yes :-) ------------- PR: https://git.openjdk.org/jdk/pull/10851 From dfuchs at openjdk.org Thu Oct 27 18:26:33 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 27 Oct 2022 18:26:33 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 18:03:50 GMT, Mark Sheppard wrote: >> Darragh Clarke has updated the pull request incrementally with one additional commit since the last revision: >> >> added loop to retry in event of interruption from other tests > > test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 65: > >> 63: ignoreStrayWakeup = checkUnexpectedWakeup(dc, sel.selectedKeys()); >> 64: sel.selectedKeys().clear(); >> 65: if (ignoreStrayWakeup) { > > scenario 1 and 3 treat the received message differently to that of scenario 2 -- should they be handled the same ? I don't think so - scenario 2 is the one that expect the PortUnreachableException - so the wakeup is expected there and the read is expected to throw PortUnreadableException. The other two scenario are expected to timeout of the select with n == 0; ------------- PR: https://git.openjdk.org/jdk/pull/10851 From msheppar at openjdk.org Thu Oct 27 18:55:37 2022 From: msheppar at openjdk.org (Mark Sheppard) Date: Thu, 27 Oct 2022 18:55:37 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 18:24:05 GMT, Daniel Fuchs wrote: >> test/jdk/java/nio/channels/DatagramChannel/SelectWhenRefused.java line 65: >> >>> 63: ignoreStrayWakeup = checkUnexpectedWakeup(dc, sel.selectedKeys()); >>> 64: sel.selectedKeys().clear(); >>> 65: if (ignoreStrayWakeup) { >> >> scenario 1 and 3 treat the received message differently to that of scenario 2 -- should they be handled the same ? > > I don't think so - scenario 2 is the one that expect the PortUnreachableException - so the wakeup is expected there and the read is expected to throw PortUnreadableException. The other two scenario are expected to timeout of the select with n == 0; I mean the actual testing of the received message -- scenario performs the read, and expects an Exception but if it gets a message, which like the other scenarios it shouldn't, then the test of that message should be the same across all 3 scenarios ? scenario 1 and 3 test if they receive what has been sent then retry, while scenario 2 will throw a RuntimeException if it receives what it has sent. This seems inconsistent. Maybe change the send message to be completely unique to this test, there maybe other DatagramSocket or Channel tests that have message containing "Greetings" ------------- PR: https://git.openjdk.org/jdk/pull/10851 From msheppar at openjdk.org Thu Oct 27 20:40:31 2022 From: msheppar at openjdk.org (Mark Sheppard) Date: Thu, 27 Oct 2022 20:40:31 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 17:58:49 GMT, Daniel Fuchs wrote: > Given the intermittent nature of the failure I'd rather do both at the same time. If we do receive a datagram, when we expect none, and we do verify that the datagram doesn't come from this test (it doesn't contain "Greetings!") then I'd argue that it must come from somewhere else (whatever that somewhere is) and it's safe to respin and try again. > > There might be some subtle issues to fix in the retry logic though. I believe we should respin when getting unexpected wake ups only if we actually manage to read a datagram - for instance, and that we can assert that this datagram is not ours (and it should never be, since we sent it to a refuser that has a different port than our receiver). We should fail in all other cases. I don't think it should matter what the source of the unexpected data is. The main issue for the failed scenario is that data may have been reeived. If the test's sending DatagramChannel receives its own data, then that is a more problematic situation than if it is has received some data from an external UDP src. If it receives its own data then the DatagramChannel has had its data looped back, similar to what happens in multicast scenarios when IP_MULTICAST_LOOP is set on a DatagramChannel, and that channel sends to a multicast address. But this is a directed send and that shouldn't happen. That wiuld all suggest anomalous behaviour at the OS level Outputting the socket address of the sender would be helpful to determine if there is a possibility that any datagram originates from itself. ------------- PR: https://git.openjdk.org/jdk/pull/10851 From mcimadamore at openjdk.org Thu Oct 27 21:00:07 2022 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 27 Oct 2022 21:00:07 GMT Subject: RFR: 8295044: Implementation of Foreign Function and Memory API (Second Preview) In-Reply-To: References: Message-ID: On Wed, 26 Oct 2022 13:11:50 GMT, Maurizio Cimadamore wrote: > This PR contains the API and implementation changes for JEP-434 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.org/jeps/434 Here are the main API changes introduced in this round (there are also some JVM changes which will be integrated separately): * The main change is the removal of `MemoryAddress` and `Addressable`. Instead, *zero-length memory segments* are used whenever the API needs to model "raw" addresses coming from native code. This simplifies the API, removing an ambiguous abstraction as well as some duplication in the API (see accessor methods in `MemoryAddress`); * To allow for "unsafe" access of zero-length memory segments, a new method has been added to `ValueLayout.OfAddress`, namely `asUnbounded`. This new restricted method takes an address layout and creates a new unbounded address layout. When using an unbounded layout to dereference memory, or construct downcall method handles, the API will create memory segments with maximal length (i.e. `Long.MAX_VALUE`, rather than zero-length memory segments, which can therefore be accessed; * The `MemoryLayout` hierarchy has been improved in several ways. First, the hierarchy is now defined in terms of sealed interfaces (intermediate abstract classes have been moved into the implementation package). The hierarchy is also exhaustive now, and works much better to pattern matching. More specifically, three new types have been added: `PaddingLayout`, `StructLayout` and `UnionLayout`, the latter two are a subtype of `GroupLayout`. Thanks to this move, several predicate methods (`isPadding`, `isStruct`, `isUnion`) have been dropped from the API; * The `SymbolLookup::lookup` method has been renamed to `SymbolLookup::find` - to avoid using the same word `lookup` in both noun and verb form, which leads to confusion; * A new method, on `ModuleLayer.Controller` has been added to enable native access on a module in a custom layer; * The new interface `Linker.Option` has been introduced. This is a tag interface accepted in `Linker::downcallHandle`. At the moment, only a single option is provided, to specify variadic function calls (because of this, the `FunctionDescriptor` interface has been simplified, and is now a simple carrier of arguments/return layouts). More linker options will follow. Javadoc: http://cr.openjdk.java.net/~mcimadamore/jdk/8295044/v1/javadoc/java.base/java/lang/foreign/package-summary.html ------------- PR: https://git.openjdk.org/jdk/pull/10872 From mcimadamore at openjdk.org Thu Oct 27 21:00:07 2022 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 27 Oct 2022 21:00:07 GMT Subject: RFR: 8295044: Implementation of Foreign Function and Memory API (Second Preview) Message-ID: This PR contains the API and implementation changes for JEP-434 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. [1] - https://openjdk.org/jeps/434 ------------- Commit messages: - Merge branch 'master' into PR_20 - Merge pull request #14 from minborg/small-javadoc - Update some javadocs - Revert some javadoc changes - Merge branch 'master' into PR_20 - Fix benchmark and test failure - Merge pull request #13 from minborg/revert-factories - Update javadocs after comments - Revert MemorySegment factories - Merge pull request #12 from minborg/fix-lookup-find - ... and 6 more: https://git.openjdk.org/jdk/compare/78454b69...ac7733da Changes: https://git.openjdk.org/jdk/pull/10872/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10872&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8295044 Stats: 10527 lines in 200 files changed: 4754 ins; 3539 del; 2234 mod Patch: https://git.openjdk.org/jdk/pull/10872.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10872/head:pull/10872 PR: https://git.openjdk.org/jdk/pull/10872 From cjplummer at openjdk.org Fri Oct 28 00:55:44 2022 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 28 Oct 2022 00:55:44 GMT Subject: RFR: 8294321: Fix typos in files under test/jdk/java, test/jdk/jdk, test/jdk/jni [v2] In-Reply-To: <-0yo8KceENmJ48YPNoHCUkx_iEWpIE0mPJn_-BkjbWY=.76a8dcb8-f43a-4c8b-8912-43c7225c183d@github.com> References: <-0yo8KceENmJ48YPNoHCUkx_iEWpIE0mPJn_-BkjbWY=.76a8dcb8-f43a-4c8b-8912-43c7225c183d@github.com> Message-ID: On Fri, 7 Oct 2022 12:51:26 GMT, Alan Bateman wrote: >> Michael Ernst has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: >> >> - Reinstate typos in Apache code that is copied into the JDK >> - Merge ../jdk-openjdk into typos-typos >> - Remove file that was removed upstream >> - Fix inconsistency in capitalization >> - Undo change in zlip >> - Fix typos > > src/java.se/share/data/jdwp/jdwp.spec line 101: > >> 99: "platform thread " >> 100: "in the target VM. This includes platform threads created with the Thread " >> 101: "API and all native threads attached to the target VM with JNI code." > > The spec for the JDWP AllThreads command was significantly reworded in Java 19 so this is where this typo crept in. We have JDK-8294672 tracking it to fix for Java 20, maybe you should take it? Since this PR has gone stale, I'll be fixing this typo in jdwp.spec via [JDK-8294672](https://bugs.openjdk.org/browse/JDK-8294672). ------------- PR: https://git.openjdk.org/jdk/pull/10029 From dfuchs at openjdk.org Fri Oct 28 08:34:31 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Fri, 28 Oct 2022 08:34:31 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: <6VOqzTWmmDJQvBFidoj_Fbcu8-aQny0KAY3gwKpVBuY=.187bfe91-11f9-48b3-a294-48084824900d@github.com> On Thu, 27 Oct 2022 16:28:29 GMT, Darragh Clarke wrote: >> Added logging to SelectWhenRefused for an intermittent failure caused by unexpected wakeups, this includes trying to receive data if there is any available to identify where it is coming from. >> >> I also set the test to run in othervm mode which should reduce the chances of this failure happening in the first place. > > Darragh Clarke has updated the pull request incrementally with one additional commit since the last revision: > > added loop to retry in event of interruption from other tests Changes requested by dfuchs (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/10851 From dfuchs at openjdk.org Fri Oct 28 08:34:31 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Fri, 28 Oct 2022 08:34:31 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 18:53:19 GMT, Mark Sheppard wrote: >> I don't think so - scenario 2 is the one that expect the PortUnreachableException - so the wakeup is expected there and the read is expected to throw PortUnreadableException. The other two scenario are expected to timeout of the select with n == 0; > > I mean the actual testing of the received message -- scenario performs the read, and expects an Exception but if it gets a message, which like the other scenarios it shouldn't, then the test of that message should be the same across all 3 scenarios ? > scenario 1 and 3 test if they receive what has been sent then retry, while scenario 2 will throw a RuntimeException if it receives what it has sent. This seems inconsistent. > > Maybe change the send message to be completely unique to this test, there maybe other DatagramSocket or Channel tests that have message containing "Greetings" No I believe scenario 2 is different - but you are right - it would be more correct to replace lines 90-91 with this: // BindException will be thrown if another service is using // our expected refuser port, cannot run just exit. DatagramChannel.open().bind(refuser).close(); throw new RuntimeException("PortUnreachableException not raised"); In other words: we are connected - so we could only receive a message from something bound to the refuser's port. So check whether that port is still unbound, and if so, complain that PortUnreachableException should have been received. Note that there still might be a race condition here because something could have transiently bound to the refuser and then released it. But the window of opportunity for the race should be smaller. Whether we should `throw new RuntimeException("PortUnreachableException not raised");` or simply `continue` and try again could be discussed. ------------- PR: https://git.openjdk.org/jdk/pull/10851 From dfuchs at openjdk.org Fri Oct 28 08:34:31 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Fri, 28 Oct 2022 08:34:31 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Fri, 28 Oct 2022 08:28:40 GMT, Daniel Fuchs wrote: >> I mean the actual testing of the received message -- scenario performs the read, and expects an Exception but if it gets a message, which like the other scenarios it shouldn't, then the test of that message should be the same across all 3 scenarios ? >> scenario 1 and 3 test if they receive what has been sent then retry, while scenario 2 will throw a RuntimeException if it receives what it has sent. This seems inconsistent. >> >> Maybe change the send message to be completely unique to this test, there maybe other DatagramSocket or Channel tests that have message containing "Greetings" > > No I believe scenario 2 is different - but you are right - it would be more correct to replace lines 90-91 with this: > > > // BindException will be thrown if another service is using > // our expected refuser port, cannot run just exit. > DatagramChannel.open().bind(refuser).close(); > throw new RuntimeException("PortUnreachableException not raised"); > > > In other words: we are connected - so we could only receive a message from something bound to the refuser's port. So check whether that port is still unbound, and if so, complain that PortUnreachableException should have been received. > Note that there still might be a race condition here because something could have transiently bound to the refuser and then released it. But the window of opportunity for the race should be smaller. > Whether we should `throw new RuntimeException("PortUnreachableException not raised");` or simply `continue` and try again could be discussed. > Maybe change the send message to be completely unique to this test, there maybe other DatagramSocket or Channel tests that have message containing "Greetings" Good point. Although it's the only one I could find that uses Datagrams. We could add a time stamp - or a UUID for extra security. ------------- PR: https://git.openjdk.org/jdk/pull/10851 From duke at openjdk.org Fri Oct 28 13:23:29 2022 From: duke at openjdk.org (Darragh Clarke) Date: Fri, 28 Oct 2022 13:23:29 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Fri, 28 Oct 2022 08:32:06 GMT, Daniel Fuchs wrote: >> No I believe scenario 2 is different - but you are right - it would be more correct to replace lines 90-91 with this: >> >> >> // BindException will be thrown if another service is using >> // our expected refuser port, cannot run just exit. >> DatagramChannel.open().bind(refuser).close(); >> throw new RuntimeException("PortUnreachableException not raised"); >> >> >> In other words: we are connected - so we could only receive a message from something bound to the refuser's port. So check whether that port is still unbound, and if so, complain that PortUnreachableException should have been received. >> Note that there still might be a race condition here because something could have transiently bound to the refuser and then released it. But the window of opportunity for the race should be smaller. >> Whether we should `throw new RuntimeException("PortUnreachableException not raised");` or simply `continue` and try again could be discussed. > >> Maybe change the send message to be completely unique to this test, there maybe other DatagramSocket or Channel tests that have message containing "Greetings" > > Good point. Although it's the only one I could find that uses Datagrams. We could add a time stamp - or a UUID for extra security. I think even something as simple as changing the message to ``Greetings from SelectWhenRefused!`` makes it unique, though I could do a UUID too? ------------- PR: https://git.openjdk.org/jdk/pull/10851 From dfuchs at openjdk.org Fri Oct 28 13:31:00 2022 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Fri, 28 Oct 2022 13:31:00 GMT Subject: RFR: 8293696: java/nio/channels/DatagramChannel/SelectWhenRefused.java fails with "Unexpected wakeup" [v2] In-Reply-To: References: Message-ID: On Fri, 28 Oct 2022 13:21:06 GMT, Darragh Clarke wrote: >>> Maybe change the send message to be completely unique to this test, there maybe other DatagramSocket or Channel tests that have message containing "Greetings" >> >> Good point. Although it's the only one I could find that uses Datagrams. We could add a time stamp - or a UUID for extra security. > > I think even something as simple as changing the message to ``Greetings from SelectWhenRefused!`` makes it unique, though I could do a UUID too? Good point too :-) `Greetings from SelectWhenRefused!` sounds fine to me. Also it will help identify the culprit if another test receives it! That's a good idea. ------------- PR: https://git.openjdk.org/jdk/pull/10851 From alanb at openjdk.org Fri Oct 28 15:45:45 2022 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 28 Oct 2022 15:45:45 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) In-Reply-To: References: Message-ID: On Thu, 27 Oct 2022 15:50:56 GMT, Brian Burkhalter wrote: > On macOS, perform file name normalization if and only if a specific system property is set. src/java.base/macosx/classes/sun/nio/fs/MacOSXFileSystem.java line 42: > 40: > 41: private static final boolean ENCODE_FILE_NAMES = > 42: Boolean.getBoolean(PROPERTY_ENABLE_FILE_NAME_ENCODING); APFS is the default since macOS 10.13 but I agree, it's safer to have a system property for a few releases in case anything is relying on this. I assume you'll need to use GetPropertyAction.privilegedGetProperty so that it works when there is a security manager set. As regards the property name then it will need to be jdk.something because it's JDK specific rather than a standard property. Maybe something like -Djdk.nio.path.useNormalizationFormD ? ------------- PR: https://git.openjdk.org/jdk/pull/10885 From bpb at openjdk.org Fri Oct 28 22:22:27 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 28 Oct 2022 22:22:27 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) [v2] In-Reply-To: References: Message-ID: > On macOS, perform file name normalization if and only if a specific system property is set. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8289689: Use GetPropertyAction; rename property ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10885/files - new: https://git.openjdk.org/jdk/pull/10885/files/7d2d7382..ab2e42db Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10885&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10885&range=00-01 Stats: 15 lines in 3 files changed: 8 ins; 1 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/10885.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10885/head:pull/10885 PR: https://git.openjdk.org/jdk/pull/10885 From bpb at openjdk.org Fri Oct 28 22:22:28 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Fri, 28 Oct 2022 22:22:28 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) [v2] In-Reply-To: References: Message-ID: <-y323zEHrAba_kDHADsNdXj2FMr0d_OqUdIbs6dAsqM=.f14385f7-6d6d-41a8-b522-605f7bda4d52@github.com> On Fri, 28 Oct 2022 15:42:54 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8289689: Use GetPropertyAction; rename property > > src/java.base/macosx/classes/sun/nio/fs/MacOSXFileSystem.java line 42: > >> 40: >> 41: private static final boolean ENCODE_FILE_NAMES = >> 42: Boolean.getBoolean(PROPERTY_ENABLE_FILE_NAME_ENCODING); > > APFS is the default since macOS 10.13 but I agree, it's safer to have a system property for a few releases in case anything is relying on this. > > I assume you'll need to use GetPropertyAction.privilegedGetProperty so that it works when there is a security manager set. > > As regards the property name then it will need to be jdk.something because it's JDK specific rather than a standard property. Maybe something like -Djdk.nio.path.useNormalizationFormD ? Fixed in ab2e42dbfc47ffe9026e769ef5d9329ee9098452. ------------- PR: https://git.openjdk.org/jdk/pull/10885 From duke at openjdk.org Sat Oct 29 06:31:32 2022 From: duke at openjdk.org (duke) Date: Sat, 29 Oct 2022 06:31:32 GMT Subject: Withdrawn: 8265891: (ch) InputStream returned by Channels.newInputStream should override transferTo In-Reply-To: References: Message-ID: On Sun, 30 May 2021 17:30:56 GMT, Markus KARG wrote: > This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review. > > As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are: > * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer. > * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides. > > Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. > > I encourage everybody to discuss this draft: > * Are there valid arguments for *not* doing this change? > * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`? > * How to go on from here: What is missing to get this ready for an actual review? This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/4263 From aturbanov at openjdk.org Sat Oct 29 10:58:15 2022 From: aturbanov at openjdk.org (Andrey Turbanov) Date: Sat, 29 Oct 2022 10:58:15 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) [v2] In-Reply-To: References: Message-ID: On Fri, 28 Oct 2022 22:22:27 GMT, Brian Burkhalter wrote: >> On macOS, perform file name normalization if and only if a specific system property is set. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8289689: Use GetPropertyAction; rename property src/java.base/macosx/classes/sun/nio/fs/MacOSXFileSystem.java line 46: > 44: static { > 45: final String name = PROPERTY_ENABLE_FILE_NAME_ENCODING; > 46: String value = GetPropertyAction.privilegedGetProperty(name); Can we use `sun.security.action.GetBooleanAction#privilegedGetProperty` instead? ------------- PR: https://git.openjdk.org/jdk/pull/10885 From alanb at openjdk.org Sat Oct 29 11:29:26 2022 From: alanb at openjdk.org (Alan Bateman) Date: Sat, 29 Oct 2022 11:29:26 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) [v2] In-Reply-To: References: Message-ID: On Sat, 29 Oct 2022 10:54:54 GMT, Andrey Turbanov wrote: > Can we use `sun.security.action.GetBooleanAction#privilegedGetProperty` instead? That is problematic for the case that a value is empty, as in `-Djdk.nio.path.useNormalizationFormD`. ------------- PR: https://git.openjdk.org/jdk/pull/10885 From jpai at openjdk.org Sat Oct 29 11:40:01 2022 From: jpai at openjdk.org (Jaikiran Pai) Date: Sat, 29 Oct 2022 11:40:01 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) [v2] In-Reply-To: References: Message-ID: <7BTI_K8IjyyAeTnLX3UWck3mVyv56ooJfe6t8y1ndbQ=.42aee522-31fd-404b-beee-e4a038cb4df0@github.com> On Fri, 28 Oct 2022 22:22:27 GMT, Brian Burkhalter wrote: >> On macOS, perform file name normalization if and only if a specific system property is set. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8289689: Use GetPropertyAction; rename property test/jdk/java/nio/file/Path/MacPathTest.java line 51: > 49: ProcessBuilder pb; > 50: if (ENABLE_FILE_NAME_ENCODING) { > 51: String option = ENABLE_FILE_NAME_ENCODING ? Hello Brian, it appears that the additional ternary check here is unecessary since the enclosing `if (ENABLE_FILE_NAME_ENCODING)` already implies `ENABLE_FILE_NAME_ENCODING` is `true`. ------------- PR: https://git.openjdk.org/jdk/pull/10885 From aturbanov at openjdk.org Sat Oct 29 12:08:24 2022 From: aturbanov at openjdk.org (Andrey Turbanov) Date: Sat, 29 Oct 2022 12:08:24 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) [v2] In-Reply-To: References: Message-ID: On Sat, 29 Oct 2022 11:25:49 GMT, Alan Bateman wrote: >> src/java.base/macosx/classes/sun/nio/fs/MacOSXFileSystem.java line 46: >> >>> 44: static { >>> 45: final String name = PROPERTY_ENABLE_FILE_NAME_ENCODING; >>> 46: String value = GetPropertyAction.privilegedGetProperty(name); >> >> Can we use `sun.security.action.GetBooleanAction#privilegedGetProperty` instead? > >> Can we use `sun.security.action.GetBooleanAction#privilegedGetProperty` instead? > > That is problematic for the case that a value is empty, as in `-Djdk.nio.path.useNormalizationFormD`. I'm not sure I understand. What the problem with empty value exactly? ------------- PR: https://git.openjdk.org/jdk/pull/10885 From duke at openjdk.org Mon Oct 31 08:46:31 2022 From: duke at openjdk.org (Markus KARG) Date: Mon, 31 Oct 2022 08:46:31 GMT Subject: RFR: 8265891: (ch) InputStream returned by Channels.newInputStream should override transferTo [v13] In-Reply-To: References: Message-ID: On Sun, 1 Aug 2021 22:01:33 GMT, Markus KARG wrote: >> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review. >> >> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are: >> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer. >> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides. >> >> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. >> >> I encourage everybody to discuss this draft: >> * Are there valid arguments for *not* doing this change? >> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`? >> * How to go on from here: What is missing to get this ready for an actual review? > > Markus KARG has updated the pull request incrementally with two additional commits since the last revision: > > - Draft: Eliminated duplicate code using lambda expressions > - Draft: Use blocking mode also for target channel @laeubi The performance was optimized by several PRs (works well for file based streams now), but there are still cases that could (possibly should) get optimized. These cases are still on my agenda. But before I continue with those cases, I focus on *enabling* that optimization (BufferedInputStream prevents the benefit in *most* cases, but https://github.com/openjdk/jdk/pull/10525 will fix that, so I can step on soon). ------------- PR: https://git.openjdk.org/jdk/pull/4263 From alanb at openjdk.org Mon Oct 31 16:02:11 2022 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 31 Oct 2022 16:02:11 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) [v2] In-Reply-To: References: Message-ID: On Sat, 29 Oct 2022 12:04:48 GMT, Andrey Turbanov wrote: >>> Can we use `sun.security.action.GetBooleanAction#privilegedGetProperty` instead? >> >> That is problematic for the case that a value is empty, as in `-Djdk.nio.path.useNormalizationFormD`. > > I'm not sure I understand. What the problem with empty value exactly? There are 4 cases: jdk.nio.path.useNormalizationFormD not set -Djdk.nio.path.useNormalizationFormD -Djdk.nio.path.useNormalizationFormD=true or a case variant -Djdk.nio.path.useNormalizationFormD=false or a case variant The second and third should enable the normalisation. This will do it NORMALIZE_FILE_PATHS = (value != null) && ("".equals(value) || Boolean.parseBoolean(value)); ------------- PR: https://git.openjdk.org/jdk/pull/10885 From aturbanov at openjdk.org Mon Oct 31 16:15:09 2022 From: aturbanov at openjdk.org (Andrey Turbanov) Date: Mon, 31 Oct 2022 16:15:09 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) [v2] In-Reply-To: References: Message-ID: On Mon, 31 Oct 2022 15:58:46 GMT, Alan Bateman wrote: >The second and third should enable the normalisation. Is it convention for all JDK system properties? Then I don't understand why `privilegedGetProperty` doesn't have exactly the same logic as you propose. Perhaps another helper method could be introduced? ------------- PR: https://git.openjdk.org/jdk/pull/10885 From alanb at openjdk.org Mon Oct 31 16:22:40 2022 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 31 Oct 2022 16:22:40 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) [v2] In-Reply-To: References: Message-ID: On Mon, 31 Oct 2022 16:11:12 GMT, Andrey Turbanov wrote: > Is it convention for all JDK system properties? Then I don't understand why `privilegedGetProperty` doesn't have exactly the same logic as you propose. Perhaps another helper method could be introduced? It comes up quite often. Ideally privilegedGetProperty would have another parameter to indicate the default when the value is empty. ------------- PR: https://git.openjdk.org/jdk/pull/10885 From bpb at openjdk.org Mon Oct 31 16:37:38 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 31 Oct 2022 16:37:38 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) [v3] In-Reply-To: References: Message-ID: > On macOS, perform file name normalization if and only if a specific system property is set. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8289689: Remove unneeded check of ENABLE_FILE_NAME_ENCODING in test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/10885/files - new: https://git.openjdk.org/jdk/pull/10885/files/ab2e42db..b420d2ce Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=10885&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=10885&range=01-02 Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/10885.diff Fetch: git fetch https://git.openjdk.org/jdk pull/10885/head:pull/10885 PR: https://git.openjdk.org/jdk/pull/10885 From bpb at openjdk.org Mon Oct 31 16:37:39 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 31 Oct 2022 16:37:39 GMT Subject: RFR: 8289689: (fs) Re-examine the need for normalization to Unicode Normalization Format D (macOS) [v2] In-Reply-To: <7BTI_K8IjyyAeTnLX3UWck3mVyv56ooJfe6t8y1ndbQ=.42aee522-31fd-404b-beee-e4a038cb4df0@github.com> References: <7BTI_K8IjyyAeTnLX3UWck3mVyv56ooJfe6t8y1ndbQ=.42aee522-31fd-404b-beee-e4a038cb4df0@github.com> Message-ID: On Sat, 29 Oct 2022 11:36:27 GMT, Jaikiran Pai wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8289689: Use GetPropertyAction; rename property > > test/jdk/java/nio/file/Path/MacPathTest.java line 51: > >> 49: ProcessBuilder pb; >> 50: if (ENABLE_FILE_NAME_ENCODING) { >> 51: String option = ENABLE_FILE_NAME_ENCODING ? > > Hello Brian, it appears that the additional ternary check here is unecessary since the enclosing `if (ENABLE_FILE_NAME_ENCODING)` already implies `ENABLE_FILE_NAME_ENCODING` is `true`. Indeed that is lame. Thanks for catching it. Fixed in b420d2cea162ef7a3abde0522d687a290fe154f2. ------------- PR: https://git.openjdk.org/jdk/pull/10885 From bpb at openjdk.org Mon Oct 31 17:43:21 2022 From: bpb at openjdk.org (Brian Burkhalter) Date: Mon, 31 Oct 2022 17:43:21 GMT Subject: RFR: 8295753: (fs) UnixPath::toRealPath does not return correct case when links not followed [v2] In-Reply-To: References: <6WGVJ1L_0aM7tp0LgFMbZy_L4bJ7Gr80mlg0ZKk9piI=.6fa46e68-3ae7-4582-a651-8b0b1d259aa1@github.com> Message-ID: On Tue, 25 Oct 2022 15:50:51 GMT, Alan Bateman wrote: >>> One initial comment on this is that you can't use Files.list in the provider implementation because the default provider can be configured to be something else. >> >> Yes, I knew that, but there is no equivalent in the provider. I guess `java.io.File::list` would work. > > I think you'll have to change this to use UnixDirectoryStream here, it shouldn't be too hard. We'll also need to study the case check as I suspect that will need macOS specific code. Resolved (I think) in b6170562187932311597047524647eec66ad006f. ------------- PR: https://git.openjdk.org/jdk/pull/10843