From cgo at openjdk.java.net Thu Apr 1 07:39:49 2021 From: cgo at openjdk.java.net (Christoph =?UTF-8?B?R8O2dHRzY2hrZXM=?=) Date: Thu, 1 Apr 2021 07:39:49 GMT Subject: RFR: 8264502: (fc) FileDispatcherImpl.setDirect0 might return uninitialized variable on some platforms [v2] In-Reply-To: References: Message-ID: > Hi, > > please review this small patch which fixes a possible return of an uninitialized variable. > > Thanks, > Christoph Christoph G?ttschkes has updated the pull request incrementally with one additional commit since the last revision: Updated copyright dates. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3284/files - new: https://git.openjdk.java.net/jdk/pull/3284/files/e6a4cf9d..b93ba6c8 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3284&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3284&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3284.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3284/head:pull/3284 PR: https://git.openjdk.java.net/jdk/pull/3284 From cgo at openjdk.java.net Thu Apr 1 07:39:51 2021 From: cgo at openjdk.java.net (Christoph =?UTF-8?B?R8O2dHRzY2hrZXM=?=) Date: Thu, 1 Apr 2021 07:39:51 GMT Subject: RFR: 8264502: (fc) FileDispatcherImpl.setDirect0 might return uninitialized variable on some platforms [v2] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 15:46:41 GMT, Brian Burkhalter wrote: >> Christoph G?ttschkes has updated the pull request incrementally with one additional commit since the last revision: >> >> Updated copyright dates. > > Approved modulo updating the copyright year. Thanks for the reviews. I pushed another commit with the updated copyright year. ------------- PR: https://git.openjdk.java.net/jdk/pull/3284 From alanb at openjdk.java.net Thu Apr 1 07:52:18 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 1 Apr 2021 07:52:18 GMT Subject: RFR: 8264502: (fc) FileDispatcherImpl.setDirect0 might return uninitialized variable on some platforms [v2] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 07:39:49 GMT, Christoph G?ttschkes wrote: >> Hi, >> >> please review this small patch which fixes a possible return of an uninitialized variable. >> >> Thanks, >> Christoph > > Christoph G?ttschkes has updated the pull request incrementally with one additional commit since the last revision: > > Updated copyright dates. Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3284 From michaelm at openjdk.java.net Thu Apr 1 10:03:33 2021 From: michaelm at openjdk.java.net (Michael McMahon) Date: Thu, 1 Apr 2021 10:03:33 GMT Subject: RFR: 8264502: (fc) FileDispatcherImpl.setDirect0 might return uninitialized variable on some platforms [v2] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 07:39:49 GMT, Christoph G?ttschkes wrote: >> Hi, >> >> please review this small patch which fixes a possible return of an uninitialized variable. >> >> Thanks, >> Christoph > > Christoph G?ttschkes has updated the pull request incrementally with one additional commit since the last revision: > > Updated copyright dates. Strange there isn't a compiler warning to catch that. ------------- Marked as reviewed by michaelm (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3284 From alanb at openjdk.java.net Thu Apr 1 11:12:26 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 1 Apr 2021 11:12:26 GMT Subject: RFR: 8264539: Improve failure message of java/nio/file/WatchService/SensitivityModifier.java In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 20:22:31 GMT, Brian Burkhalter wrote: > Please consider this request to improve the message printed when no `ENTRY_MODIFY` events are received. In the polling implementation of `WatchService` currently used on macOS, it would be useful to know the sensitivity of polling in effect for the failing case. It might indicate, for example, that low sensitivity should not be used in the test. test/jdk/java/nio/file/WatchService/SensitivityModifier.java line 126: > 124: String msg = String.format("No ENTRY_MODIFY event received for %s (dir: %s, sensitivity: %d)", > 125: file, parent, pathToTime.get(parent)); > 126: throw new RuntimeException(msg); Not clear to me that this helps. Have you looked at changing the 1s at L104. I assume the issue is that the test needs the idle (no events) time increased to deal with cases where the watcher is slow to queue events. ------------- PR: https://git.openjdk.java.net/jdk/pull/3290 From alanb at openjdk.java.net Thu Apr 1 11:17:21 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 1 Apr 2021 11:17:21 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: <4vGA2Y0_MbgbHRsedaxBpyEFx0UqluLvm6zmgSipav4=.27955baf-1195-42d4-becc-34c30bbd20be@github.com> References: <_5Q_jfIcOJWwzGR1RyPYU4Twct04KvrlgH-nNRWrHnw=.33da8810-101f-47f2-8c5a-e5b891e110ac@github.com> <4vGA2Y0_MbgbHRsedaxBpyEFx0UqluLvm6zmgSipav4=.27955baf-1195-42d4-becc-34c30bbd20be@github.com> Message-ID: On Wed, 31 Mar 2021 15:36:03 GMT, Brian Burkhalter wrote: > I was not able to test that case yet. Using the VolumeInformation was intended to limit spurious equality. > > Can you explain abut DRIVE_REMOTE? I think the volume type will be DRIVE_REMOTE when the root is a UNC (the constants for the volume types are defined in WIndowsConstants). I'm not 100% sure if the server and share names should be compared with or without regard to case so I think we have a bit more exploration to do there. The DRIVE_FIXED case is clear, and it would be good to at least get that fixed as there has been C:\ vs. c:\ confusion in a number of bug reports. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Thu Apr 1 16:29:26 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 1 Apr 2021 16:29:26 GMT Subject: RFR: 8264539: Improve failure message of java/nio/file/WatchService/SensitivityModifier.java In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 11:09:06 GMT, Alan Bateman wrote: >> Please consider this request to improve the message printed when no `ENTRY_MODIFY` events are received. In the polling implementation of `WatchService` currently used on macOS, it would be useful to know the sensitivity of polling in effect for the failing case. It might indicate, for example, that low sensitivity should not be used in the test. > > test/jdk/java/nio/file/WatchService/SensitivityModifier.java line 126: > >> 124: String msg = String.format("No ENTRY_MODIFY event received for %s (dir: %s, sensitivity: %d)", >> 125: file, parent, pathToTime.get(parent)); >> 126: throw new RuntimeException(msg); > > Not clear to me that this helps. Have you looked at changing the 1s at L104. I assume the issue is that the test needs the idle (no events) time increased to deal with cases where the watcher is slow to queue events. That's a better idea but I think this change should go in also. I have not in any case been able to reproduce it. There is no record of on which specific machines it was sighted. ------------- PR: https://git.openjdk.java.net/jdk/pull/3290 From bpb at openjdk.java.net Thu Apr 1 16:38:17 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 1 Apr 2021 16:38:17 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: <_5Q_jfIcOJWwzGR1RyPYU4Twct04KvrlgH-nNRWrHnw=.33da8810-101f-47f2-8c5a-e5b891e110ac@github.com> <4vGA2Y0_MbgbHRsedaxBpyEFx0UqluLvm6zmgSipav4=.27955baf-1195-42d4-becc-34c30bbd20be@github.com> Message-ID: On Thu, 1 Apr 2021 11:14:10 GMT, Alan Bateman wrote: >> I was not able to test that case yet. Using the VolumeInformation was intended to limit spurious equality. >> >> Can you explain abut DRIVE_REMOTE? > >> I was not able to test that case yet. Using the VolumeInformation was intended to limit spurious equality. >> >> Can you explain abut DRIVE_REMOTE? > > I think the volume type will be DRIVE_REMOTE when the root is a UNC (the constants for the volume types are defined in WIndowsConstants). I'm not 100% sure if the server and share names should be compared with or without regard to case so I think we have a bit more exploration to do there. The DRIVE_FIXED case is clear, and it would be good to at least get that fixed as there has been C:\ vs. c:\ confusion in a number of bug reports. Based on testing in a Win 10 VM, the volume type is in fact DRIVE_REMOTE. The volume serial number however is zero. From what I can test via testing and reading the server and share names don't care about case. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Thu Apr 1 16:45:11 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 1 Apr 2021 16:45:11 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: <_5Q_jfIcOJWwzGR1RyPYU4Twct04KvrlgH-nNRWrHnw=.33da8810-101f-47f2-8c5a-e5b891e110ac@github.com> <4vGA2Y0_MbgbHRsedaxBpyEFx0UqluLvm6zmgSipav4=.27955baf-1195-42d4-becc-34c30bbd20be@github.com> Message-ID: On Thu, 1 Apr 2021 16:35:10 GMT, Brian Burkhalter wrote: >>> I was not able to test that case yet. Using the VolumeInformation was intended to limit spurious equality. >>> >>> Can you explain abut DRIVE_REMOTE? >> >> I think the volume type will be DRIVE_REMOTE when the root is a UNC (the constants for the volume types are defined in WIndowsConstants). I'm not 100% sure if the server and share names should be compared with or without regard to case so I think we have a bit more exploration to do there. The DRIVE_FIXED case is clear, and it would be good to at least get that fixed as there has been C:\ vs. c:\ confusion in a number of bug reports. > > Based on testing in a Win 10 VM, the volume type is in fact DRIVE_REMOTE. The volume serial number however is zero. From what I can test via testing and reading the server and share names don't care about case. If we are unsure, for UNC, perhaps the server and share name comparisons can remain case sensitive but the file path component comparison be case insensitive? ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From alanb at openjdk.java.net Thu Apr 1 18:01:15 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 1 Apr 2021 18:01:15 GMT Subject: RFR: 8264539: Improve failure message of java/nio/file/WatchService/SensitivityModifier.java In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 16:26:15 GMT, Brian Burkhalter wrote: >> test/jdk/java/nio/file/WatchService/SensitivityModifier.java line 126: >> >>> 124: String msg = String.format("No ENTRY_MODIFY event received for %s (dir: %s, sensitivity: %d)", >>> 125: file, parent, pathToTime.get(parent)); >>> 126: throw new RuntimeException(msg); >> >> Not clear to me that this helps. Have you looked at changing the 1s at L104. I assume the issue is that the test needs the idle (no events) time increased to deal with cases where the watcher is slow to queue events. > > That's a better idea but I think this change should go in also. I have not in any case been able to reproduce it. There is no record of on which specific machines it was sighted. Okay. ------------- PR: https://git.openjdk.java.net/jdk/pull/3290 From alanb at openjdk.java.net Thu Apr 1 18:06:17 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 1 Apr 2021 18:06:17 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: <_5Q_jfIcOJWwzGR1RyPYU4Twct04KvrlgH-nNRWrHnw=.33da8810-101f-47f2-8c5a-e5b891e110ac@github.com> <4vGA2Y0_MbgbHRsedaxBpyEFx0UqluLvm6zmgSipav4=.27955baf-1195-42d4-becc-34c30bbd20be@github.com> Message-ID: On Thu, 1 Apr 2021 16:42:38 GMT, Brian Burkhalter wrote: >> Based on testing in a Win 10 VM, the volume type is in fact DRIVE_REMOTE. The volume serial number however is zero. From what I can test via testing and reading the server and share names don't care about case. > > If we are unsure, for UNC, perhaps the server and share name comparisons can remain case sensitive but the file path component comparison be case insensitive? Maybe this is something that the Microsoft folks can help us with, it might be that it depends on the transport. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Thu Apr 1 19:34:20 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 1 Apr 2021 19:34:20 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: <_5Q_jfIcOJWwzGR1RyPYU4Twct04KvrlgH-nNRWrHnw=.33da8810-101f-47f2-8c5a-e5b891e110ac@github.com> <4vGA2Y0_MbgbHRsedaxBpyEFx0UqluLvm6zmgSipav4=.27955baf-1195-42d4-becc-34c30bbd20be@github.com> Message-ID: On Thu, 1 Apr 2021 18:03:30 GMT, Alan Bateman wrote: >> If we are unsure, for UNC, perhaps the server and share name comparisons can remain case sensitive but the file path component comparison be case insensitive? > > Maybe this is something that the Microsoft folks can help us with, it might be that it depends on the transport. Good idea. I will ask them. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From ecki at zusammenkunft.net Thu Apr 1 20:29:27 2021 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Thu, 1 Apr 2021 20:29:27 +0000 Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: <_5Q_jfIcOJWwzGR1RyPYU4Twct04KvrlgH-nNRWrHnw=.33da8810-101f-47f2-8c5a-e5b891e110ac@github.com> <4vGA2Y0_MbgbHRsedaxBpyEFx0UqluLvm6zmgSipav4=.27955baf-1195-42d4-becc-34c30bbd20be@github.com> , Message-ID: Hello, Is it instead possible to store a normalized version of the store base path (for the purpose of equals but also for display)? This would reduce dependence of file name casing semantics which does not in all cases depend only on the host operating system. Maybe using lpRootPathName if present in GetVolumeInformation? Gruss Bernd -- http://bernd.eckenfels.net ________________________________ Von: nio-dev im Auftrag von Brian Burkhalter Gesendet: Thursday, April 1, 2021 9:34:20 PM An: nio-dev at openjdk.java.net Betreff: Re: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed On Thu, 1 Apr 2021 18:03:30 GMT, Alan Bateman wrote: >> If we are unsure, for UNC, perhaps the server and share name comparisons can remain case sensitive but the file path component comparison be case insensitive? > > Maybe this is something that the Microsoft folks can help us with, it might be that it depends on the transport. Good idea. I will ask them. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Apr 1 21:46:02 2021 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 1 Apr 2021 21:46:02 +0000 Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: <_5Q_jfIcOJWwzGR1RyPYU4Twct04KvrlgH-nNRWrHnw=.33da8810-101f-47f2-8c5a-e5b891e110ac@github.com> <4vGA2Y0_MbgbHRsedaxBpyEFx0UqluLvm6zmgSipav4=.27955baf-1195-42d4-becc-34c30bbd20be@github.com> Message-ID: Hello, On Apr 1, 2021, at 1:29 PM, Bernd Eckenfels > wrote: Is it instead possible to store a normalized version of the store base path (for the purpose of equals but also for display)? This would reduce dependence of file name casing semantics which does not in all cases depend only on the host operating system. It?s not clear what is intended here. WindowsFileStore does not have a base path itself. Its root path is the path supplied as a parameter of Files.getFileStore(). Maybe using lpRootPathName if present in GetVolumeInformation? lpRootPathName is the same as WindowsFileStore.root and is an input to GetVolumeInformationW() [1]. Thanks, Brian [1] https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getvolumeinformationw -------------- next part -------------- An HTML attachment was scrubbed... URL: From bpb at openjdk.java.net Fri Apr 2 01:50:37 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 2 Apr 2021 01:50:37 GMT Subject: RFR: 8264539: Improve failure message of java/nio/file/WatchService/SensitivityModifier.java [v2] In-Reply-To: References: Message-ID: > Please consider this request to improve the message printed when no `ENTRY_MODIFY` events are received. In the polling implementation of `WatchService` currently used on macOS, it would be useful to know the sensitivity of polling in effect for the failing case. It might indicate, for example, that low sensitivity should not be used in the test. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8264539: Change poll timeout on macOS; use RandomFactory ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3290/files - new: https://git.openjdk.java.net/jdk/pull/3290/files/708e9d6a..bc8f9504 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3290&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3290&range=00-01 Stats: 16 lines in 1 file changed: 8 ins; 0 del; 8 mod Patch: https://git.openjdk.java.net/jdk/pull/3290.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3290/head:pull/3290 PR: https://git.openjdk.java.net/jdk/pull/3290 From cgo at openjdk.java.net Fri Apr 2 09:12:26 2021 From: cgo at openjdk.java.net (Christoph =?UTF-8?B?R8O2dHRzY2hrZXM=?=) Date: Fri, 2 Apr 2021 09:12:26 GMT Subject: RFR: 8264502: (fc) FileDispatcherImpl.setDirect0 might return uninitialized variable on some platforms [v2] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 10:00:05 GMT, Michael McMahon wrote: >> Christoph G?ttschkes has updated the pull request incrementally with one additional commit since the last revision: >> >> Updated copyright dates. > > Strange there isn't a compiler warning to catch that. Thanks for the reviews. There is a compiler warning about this. But the code path is never exercised by a compiler for any of the supported target platforms. Only on systems, which neither define O_DIRECT, F_NOCACHE, nor DIRECTIO_ON, this code path is actually compiled. I found this while playing around with a new internal port. ------------- PR: https://git.openjdk.java.net/jdk/pull/3284 From cgo at openjdk.java.net Fri Apr 2 09:24:24 2021 From: cgo at openjdk.java.net (Christoph =?UTF-8?B?R8O2dHRzY2hrZXM=?=) Date: Fri, 2 Apr 2021 09:24:24 GMT Subject: Integrated: 8264502: (fc) FileDispatcherImpl.setDirect0 might return uninitialized variable on some platforms In-Reply-To: References: Message-ID: <0KKpemMo4JyYKTFqhYrH5i6CtxUxKfB4Bs7UhyOmVL8=.5846a6fb-6683-4225-86d7-d2408ed27ed8@github.com> On Wed, 31 Mar 2021 10:50:56 GMT, Christoph G?ttschkes wrote: > Hi, > > please review this small patch which fixes a possible return of an uninitialized variable. > > Thanks, > Christoph This pull request has now been integrated. Changeset: 4793557f Author: Christoph G?ttschkes Committer: Alan Bateman URL: https://git.openjdk.java.net/jdk/commit/4793557f Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod 8264502: (fc) FileDispatcherImpl.setDirect0 might return uninitialized variable on some platforms Reviewed-by: alanb, bpb, michaelm ------------- PR: https://git.openjdk.java.net/jdk/pull/3284 From alanb at openjdk.java.net Fri Apr 2 14:23:25 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 2 Apr 2021 14:23:25 GMT Subject: RFR: 8264539: Improve failure message of java/nio/file/WatchService/SensitivityModifier.java [v2] In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 01:50:37 GMT, Brian Burkhalter wrote: >> Please consider this request to improve the message printed when no `ENTRY_MODIFY` events are received. In the polling implementation of `WatchService` currently used on macOS, it would be useful to know the sensitivity of polling in effect for the failing case. It might indicate, for example, that low sensitivity should not be used in the test. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8264539: Change poll timeout on macOS; use RandomFactory Changes look okay although I don't think changing POLL_TIMEOUT_SECONDS vary by platform is necessary. ------------- Marked as reviewed by alanb (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3290 From Alan.Bateman at oracle.com Fri Apr 2 14:36:54 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 2 Apr 2021 15:36:54 +0100 Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: <_5Q_jfIcOJWwzGR1RyPYU4Twct04KvrlgH-nNRWrHnw=.33da8810-101f-47f2-8c5a-e5b891e110ac@github.com> <4vGA2Y0_MbgbHRsedaxBpyEFx0UqluLvm6zmgSipav4=.27955baf-1195-42d4-becc-34c30bbd20be@github.com> Message-ID: <02d78031-b4af-3861-5beb-45ad782a51ad@oracle.com> On 01/04/2021 22:46, Brian Burkhalter wrote: > Hello, > >> On Apr 1, 2021, at 1:29 PM, Bernd Eckenfels > > wrote: >> >> Is it instead possible to store a normalized version of the store >> base path (for the purpose of equals but also for display)? This >> would reduce dependence of file name casing semantics which does not >> in all cases depend only on the host operating system. > > It?s not clear what is intended here. WindowsFileStore does not have a > base path itself. Its root path is the path supplied as a parameter?of > Files.getFileStore(). > I think it would be useful to looking at the values of the volInfo field for both the DRIVE_FIXED and DRIVE_REMOTE cases. I think we need to see the values of fileSystemName and volumeName to see if we could use them here. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bpb at openjdk.java.net Fri Apr 2 16:53:18 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 2 Apr 2021 16:53:18 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 00:22:20 GMT, Brian Burkhalter wrote: > Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. I already have this information. 1. `C:` - Win10 system drive `Root: C:\ volName: System, fsName: NTFS, volSN: 1620920694, volType: 3` 2. `E:` - CD drive (VirtualBox Guest Additions) `Root: E:\ volName: VBox_GAs_6.1.18, fsName: CDFS, volSN: -1657170319, volType: 5` 3. `Z:` - VirtualBox shared folder `Root: Z:\ volName: VBOX_VirtualBox, fsName: VBoxSharedFolderFS, volSN: 16777221, volType: 4` 4. `\\BPB\Public` - Local share created for `C:\Users\Public` `Root: \\BPB\Public\ volName: System, fsName: NTFS, volSN: 1620920694, volType: 4` 5. `\\192.168.0.111\Shared` - "Remote" share of macOS host `/Users/Shared` `Root: \\192.168.0.111\Shared\ volName: Macintosh HD - Data, fsName: NTFS, volSN: 0, volType: 4` ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Fri Apr 2 19:56:28 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 2 Apr 2021 19:56:28 GMT Subject: Integrated: 8264539: Improve failure message of java/nio/file/WatchService/SensitivityModifier.java In-Reply-To: References: Message-ID: <1noMutwggKySX5fLK_q3iZYj_PHhDDD4jv0glb98gyA=.cfc34b56-d4a9-42c9-8c5b-132d0e49b742@github.com> On Wed, 31 Mar 2021 20:22:31 GMT, Brian Burkhalter wrote: > Please consider this request to improve the message printed when no `ENTRY_MODIFY` events are received. In the polling implementation of `WatchService` currently used on macOS, it would be useful to know the sensitivity of polling in effect for the failing case. It might indicate, for example, that low sensitivity should not be used in the test. This pull request has now been integrated. Changeset: 3991b329 Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/3991b329 Stats: 37 lines in 1 file changed: 23 ins; 0 del; 14 mod 8264539: Improve failure message of java/nio/file/WatchService/SensitivityModifier.java Reviewed-by: alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/3290 From github.com+1204330+overheadhunter at openjdk.java.net Mon Apr 5 05:12:53 2021 From: github.com+1204330+overheadhunter at openjdk.java.net (Sebastian Stenzel) Date: Mon, 5 Apr 2021 05:12:53 GMT Subject: RFR: JDK-8262957 (fs) Fail fast in UnixFileStore.isExtendedAttributesEnabled In-Reply-To: References: Message-ID: On Thu, 4 Mar 2021 02:10:57 GMT, Brian Burkhalter wrote: >> Added new capability flag `UnixNativeDispatcher.SUPPORTS_XATTR`, which gets set `#ifdef _SYS_XATTR_H_`. >> >> Note that `_SYS_XATTR_H_` is defined in `xattr.h` in both [macOS/Darwin](https://github.com/apple/darwin-xnu/blob/main/bsd/sys/xattr.h), [Linux and GNU-based BSD systems using glibc](https://github.com/bminor/glibc/blob/master/misc/sys/xattr.h). It might not be defined for other operating systems that still support xattr. So if OpenJDK eventually adds support for further platforms, this might need to be adjusted as well. >> >> If xattr capabilities are missing, `UnixFileStore.isExtendedAttributesEnabled` will return false immediately, avoiding any I/O. >> >> As a side effect of this change, I redefined some other (private) capabilities. Strictly speaking, this change is not required but keeps the code consistent, as [previously discussed in the mailing list](https://mail.openjdk.java.net/pipermail/nio-dev/2021-February/008133.html). > > I think this look all right. I'll withhold approving until @AlanBateman has commented. @bplb @AlanBateman Bump ?? ------------- PR: https://git.openjdk.java.net/jdk/pull/2816 From Alan.Bateman at oracle.com Mon Apr 5 07:14:39 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 5 Apr 2021 08:14:39 +0100 Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: Message-ID: On 02/04/2021 17:53, Brian Burkhalter wrote: > : > 4. `\\BPB\Public` - Local share created for `C:\Users\Public` > > `Root: \\BPB\Public\ > volName: System, fsName: NTFS, volSN: 1620920694, volType: 4` > > 5. `\\192.168.0.111\Shared` - "Remote" share of macOS host `/Users/Shared` > > `Root: \\192.168.0.111\Shared\ > volName: Macintosh HD - Data, fsName: NTFS, volSN: 0, volType: 4` > It would be useful for the discussion if you could repeat with UNC server name and share name in other cases, e.g. \\bpb\public\. One other thing is that if we do change the equals (and I think we need to for at least DRIVE_FIXED case) then it will require the hashCode method to be changed too. -Alan From bpb at openjdk.java.net Mon Apr 5 17:22:20 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 5 Apr 2021 17:22:20 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 16:49:31 GMT, Brian Burkhalter wrote: >> Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. > > I already have this information. > > 1. `C:` - Win10 system drive > > `Root: C:\ > volName: System, fsName: NTFS, volSN: 1620920694, volType: 3` > > 2. `E:` - CD drive (VirtualBox Guest Additions) > > `Root: E:\ > volName: VBox_GAs_6.1.18, fsName: CDFS, volSN: -1657170319, volType: 5` > > 3. `Z:` - VirtualBox shared folder > > `Root: Z:\ > volName: VBOX_VirtualBox, fsName: VBoxSharedFolderFS, volSN: 16777221, volType: > 4` > > 4. `\\BPB\Public` - Local share created for `C:\Users\Public` > > `Root: \\BPB\Public\ > volName: System, fsName: NTFS, volSN: 1620920694, volType: 4` > > 5. `\\192.168.0.111\Shared` - "Remote" share of macOS host `/Users/Shared` > > `Root: \\192.168.0.111\Shared\ > volName: Macintosh HD - Data, fsName: NTFS, volSN: 0, volType: 4` I already tested with other cases but didn't include it: Root: \\bpb\Public\ volName: System, fsName: NTFS, volSN: 1620920694, volType: 4 Root: \\BPB\pUBLIC\ volName: System, fsName: NTFS, volSN: 1620920694, volType: 4 Root: \\192.168.0.111\Shared\ volName: Macintosh HD - Data, fsName: NTFS, volSN: 0, volType: 4 Root: \\192.168.0.111\shared\ volName: Macintosh HD - Data, fsName: NTFS, volSN: 0, volType: 4 Good point about `hashCode()`. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From ccleary at openjdk.java.net Tue Apr 6 11:10:44 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 11:10:44 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently Message-ID: ### Description SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. - Test is run in AgentVM mode. - Receivers bind to wildcard/localhost which has been a source of instability. - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. ### Fixes To Address these issues, the following was done respective to the order of the Issues above: - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. - Receivers now bind to loopback address instead of localhost. - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). ------------- Commit messages: - 8264200: Set timeout to 20 secs - 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently Changes: https://git.openjdk.java.net/jdk/pull/3354/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3354&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264200 Stats: 149 lines in 1 file changed: 73 ins; 37 del; 39 mod Patch: https://git.openjdk.java.net/jdk/pull/3354.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3354/head:pull/3354 PR: https://git.openjdk.java.net/jdk/pull/3354 From dfuchs at openjdk.java.net Tue Apr 6 11:42:21 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Tue, 6 Apr 2021 11:42:21 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently In-Reply-To: References: Message-ID: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> On Tue, 6 Apr 2021 10:24:50 GMT, Conor Cleary wrote: > ### Description > SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. > > - Test is run in AgentVM mode. > - Receivers bind to wildcard/localhost which has been a source of instability. > - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. > - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. > > ### Fixes > To Address these issues, the following was done respective to the order of the Issues above: > - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. > - Receivers now bind to loopback address instead of localhost. > - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). > - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). Good cleanup overall. test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 155: > 153: public void run() { > 154: try { > 155: DatagramChannel dc = DatagramChannel.open(); This line needs to be removed since `dc` is now created in the constructor. test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 165: > 163: dc.send(bb, isa); > 164: } catch (Exception ex) { > 165: log.println("ClassicWriter [" + dc.socket().getLocalAddress() + "]"); This is not the `ClassicWriter` but the `NioWriter` test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 228: > 226: SocketAddress sa = dc.receive(bb); > 227: bb.flip(); > 228: CharBuffer cb = StandardCharsets.US_ASCII.newDecoder().decode(bb); If a charset is used, then it should be used consistently - in both classic and nio readers and writers. That is, whenever `String::getBytes` is called or `new String(byte[])` is called. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Tue Apr 6 12:27:09 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 12:27:09 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently In-Reply-To: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> References: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> Message-ID: <2UeWIooE7IF36O1VsMLYgRYCKBENCSiucXW6FjGiDp0=.fa39a6fd-c516-405f-9f5f-19533d61e4b8@github.com> On Tue, 6 Apr 2021 11:22:04 GMT, Daniel Fuchs wrote: >> ### Description >> SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. >> >> - Test is run in AgentVM mode. >> - Receivers bind to wildcard/localhost which has been a source of instability. >> - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. >> - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. >> >> ### Fixes >> To Address these issues, the following was done respective to the order of the Issues above: >> - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. >> - Receivers now bind to loopback address instead of localhost. >> - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). >> - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). > > test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 155: > >> 153: public void run() { >> 154: try { >> 155: DatagramChannel dc = DatagramChannel.open(); > > This line needs to be removed since `dc` is now created in the constructor. Dang, that's a bad one. Thank you for spotting! Didn't cause any test failures but of course very wrong. > test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 165: > >> 163: dc.send(bb, isa); >> 164: } catch (Exception ex) { >> 165: log.println("ClassicWriter [" + dc.socket().getLocalAddress() + "]"); > > This is not the `ClassicWriter` but the `NioWriter` Same is true for the other classes, a fix was not included in the previous commit and will be in the next one ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Tue Apr 6 13:11:27 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 13:11:27 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently In-Reply-To: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> References: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> Message-ID: <09VepmgusTE1Rdyg1sumW9M-URJAg3Kcm19OmVXruDk=.997f7190-9e28-4016-8d09-a984cba752ca@github.com> On Tue, 6 Apr 2021 11:38:20 GMT, Daniel Fuchs wrote: >> ### Description >> SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. >> >> - Test is run in AgentVM mode. >> - Receivers bind to wildcard/localhost which has been a source of instability. >> - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. >> - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. >> >> ### Fixes >> To Address these issues, the following was done respective to the order of the Issues above: >> - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. >> - Receivers now bind to loopback address instead of localhost. >> - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). >> - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). > > test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 228: > >> 226: SocketAddress sa = dc.receive(bb); >> 227: bb.flip(); >> 228: CharBuffer cb = StandardCharsets.US_ASCII.newDecoder().decode(bb); > > If a charset is used, then it should be used consistently - in both classic and nio readers and writers. > That is, whenever `String::getBytes` is called or `new String(byte[])` is called. Makes sense, I think I will opt for using `US_ASCII` across all of the classic and nio readers/writers. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From rriggs at openjdk.java.net Tue Apr 6 13:56:25 2021 From: rriggs at openjdk.java.net (Roger Riggs) Date: Tue, 6 Apr 2021 13:56:25 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently In-Reply-To: <09VepmgusTE1Rdyg1sumW9M-URJAg3Kcm19OmVXruDk=.997f7190-9e28-4016-8d09-a984cba752ca@github.com> References: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> <09VepmgusTE1Rdyg1sumW9M-URJAg3Kcm19OmVXruDk=.997f7190-9e28-4016-8d09-a984cba752ca@github.com> Message-ID: <0jTHzHzp8lwRhi5lg8T5sFTcbQ9NaHAm_asZ3VVhysM=.a4a0d8fa-8397-484c-ba1b-5f57527b1cb7@github.com> On Tue, 6 Apr 2021 13:07:58 GMT, Conor Cleary wrote: >> test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 228: >> >>> 226: SocketAddress sa = dc.receive(bb); >>> 227: bb.flip(); >>> 228: CharBuffer cb = StandardCharsets.US_ASCII.newDecoder().decode(bb); >> >> If a charset is used, then it should be used consistently - in both classic and nio readers and writers. >> That is, whenever `String::getBytes` is called or `new String(byte[])` is called. > > Makes sense, I think I will opt for using `US_ASCII` across all of the classic and nio readers/writers. Using an 8-bit charset might be a better choice, US_ASCII is only 7bit and might produce some unmappable characters. ISO_8859_1 would be a better choice. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Tue Apr 6 14:11:24 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 14:11:24 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently In-Reply-To: <0jTHzHzp8lwRhi5lg8T5sFTcbQ9NaHAm_asZ3VVhysM=.a4a0d8fa-8397-484c-ba1b-5f57527b1cb7@github.com> References: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> <09VepmgusTE1Rdyg1sumW9M-URJAg3Kcm19OmVXruDk=.997f7190-9e28-4016-8d09-a984cba752ca@github.com> <0jTHzHzp8lwRhi5lg8T5sFTcbQ9NaHAm_asZ3VVhysM=.a4a0d8fa-8397-484c-ba1b-5f57527b1cb7@github.com> Message-ID: On Tue, 6 Apr 2021 13:53:41 GMT, Roger Riggs wrote: >> Makes sense, I think I will opt for using `US_ASCII` across all of the classic and nio readers/writers. > > Using an 8-bit charset might be a better choice, US_ASCII is only 7bit and might produce some unmappable characters. > ISO_8859_1 would be a better choice. Fair enough. Is there any reason then why UTF-8 would be used over ISO_8859_1 then, given what you said above about character mapping? ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From dfuchs at openjdk.java.net Tue Apr 6 14:27:26 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Tue, 6 Apr 2021 14:27:26 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently In-Reply-To: References: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> <09VepmgusTE1Rdyg1sumW9M-URJAg3Kcm19OmVXruDk=.997f7190-9e28-4016-8d09-a984cba752ca@github.com> <0jTHzHzp8lwRhi5lg8T5sFTcbQ9NaHAm_asZ3VVhysM=.a4a0d8fa-8397-484c-ba1b-5f57527b1cb7@github.com> Message-ID: On Tue, 6 Apr 2021 14:08:01 GMT, Conor Cleary wrote: >> Using an 8-bit charset might be a better choice, US_ASCII is only 7bit and might produce some unmappable characters. >> ISO_8859_1 would be a better choice. > > Fair enough. Is there any reason then why UTF-8 would be used over ISO_8859_1 then, given what you said above about character mapping? Or vice-versa The message sent by the writer is "hello" - it only contains ASCII characters, so any of US-ASCII, UTF-8, or ISO-8859-1, which are all compatible with ASCII (meaning: all these encode ASCII chars in the same way) would work. In fact any charset in which "hello" is mappable would work. We simply need to make sure to use the *same charset* when encoding or decoding. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From rriggs at openjdk.java.net Tue Apr 6 14:31:22 2021 From: rriggs at openjdk.java.net (Roger Riggs) Date: Tue, 6 Apr 2021 14:31:22 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently In-Reply-To: References: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> <09VepmgusTE1Rdyg1sumW9M-URJAg3Kcm19OmVXruDk=.997f7190-9e28-4016-8d09-a984cba752ca@github.com> <0jTHzHzp8lwRhi5lg8T5sFTcbQ9NaHAm_asZ3VVhysM=.a4a0d8fa-8397-484c-ba1b-5f57527b1cb7@github.com> Message-ID: On Tue, 6 Apr 2021 14:24:16 GMT, Daniel Fuchs wrote: >> Fair enough. Is there any reason then why UTF-8 would be used over ISO_8859_1 then, given what you said above about character mapping? Or vice-versa > > The message sent by the writer is "hello" - it only contains ASCII characters, so any of US-ASCII, UTF-8, or ISO-8859-1, which are all compatible with ASCII (meaning: all these encode ASCII chars in the same way) would work. > In fact any charset in which "hello" is mappable would work. We simply need to make sure to use the *same charset* when encoding or decoding. Its case by case, if you don't know how the stream of characters was encoded into a byte stream then you can't reliably decode it back to characters. For SRTest, it doesn't matter, all the strings I see are ascii. But note that String.getBytes() uses a platform dependent encoding. Until the platform encoding is UTF-8, its recommended to use String.getBytes(charset). A lot of times UTF-8 is a good guess for operations that convert strings to bytes and back. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From rriggs at openjdk.java.net Tue Apr 6 14:38:28 2021 From: rriggs at openjdk.java.net (Roger Riggs) Date: Tue, 6 Apr 2021 14:38:28 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently In-Reply-To: References: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> <09VepmgusTE1Rdyg1sumW9M-URJAg3Kcm19OmVXruDk=.997f7190-9e28-4016-8d09-a984cba752ca@github.com> <0jTHzHzp8lwRhi5lg8T5sFTcbQ9NaHAm_asZ3VVhysM=.a4a0d8fa-8397-484c-ba1b-5f57527b1cb7@github.com> Message-ID: On Tue, 6 Apr 2021 14:28:37 GMT, Roger Riggs wrote: >> The message sent by the writer is "hello" - it only contains ASCII characters, so any of US-ASCII, UTF-8, or ISO-8859-1, which are all compatible with ASCII (meaning: all these encode ASCII chars in the same way) would work. >> In fact any charset in which "hello" is mappable would work. We simply need to make sure to use the *same charset* when encoding or decoding. > > Its case by case, if you don't know how the stream of characters was encoded into a byte stream then you can't reliably decode it back to characters. For SRTest, it doesn't matter, all the strings I see are ascii. > But note that String.getBytes() uses a platform dependent encoding. > Until the platform encoding is UTF-8, its recommended to use String.getBytes(charset). > A lot of times UTF-8 is a good guess for operations that convert strings to bytes and back. As the test is for Datagram send and receive all the data is in byte arrays or equivalent. The entire test could be written using only byte arrays (no Strings, Charsets or conversions) and omit that possibility of confusion about what the test is doing. (And there's no need to change anything). ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Tue Apr 6 14:49:27 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 14:49:27 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently In-Reply-To: References: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> <09VepmgusTE1Rdyg1sumW9M-URJAg3Kcm19OmVXruDk=.997f7190-9e28-4016-8d09-a984cba752ca@github.com> <0jTHzHzp8lwRhi5lg8T5sFTcbQ9NaHAm_asZ3VVhysM=.a4a0d8fa-8397-484c-ba1b-5f57527b1cb7@github.com> Message-ID: On Tue, 6 Apr 2021 14:34:57 GMT, Roger Riggs wrote: >> Its case by case, if you don't know how the stream of characters was encoded into a byte stream then you can't reliably decode it back to characters. For SRTest, it doesn't matter, all the strings I see are ascii. >> But note that String.getBytes() uses a platform dependent encoding. >> Until the platform encoding is UTF-8, its recommended to use String.getBytes(charset). >> A lot of times UTF-8 is a good guess for operations that convert strings to bytes and back. > > As the test is for Datagram send and receive all the data is in byte arrays or equivalent. > The entire test could be written using only byte arrays (no Strings, Charsets or conversions) and omit that possibility of confusion about what the test is doing. > (And there's no need to change anything). Alright then so I will continue to to just use `US_ASCII` in this test as "hello" is mappable with it. Encoding and Decoding the string will be done with `US_ASCII`. Appreciate the feedback guys, thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From dfuchs at openjdk.java.net Tue Apr 6 14:49:28 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Tue, 6 Apr 2021 14:49:28 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently In-Reply-To: References: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> <09VepmgusTE1Rdyg1sumW9M-URJAg3Kcm19OmVXruDk=.997f7190-9e28-4016-8d09-a984cba752ca@github.com> <0jTHzHzp8lwRhi5lg8T5sFTcbQ9NaHAm_asZ3VVhysM=.a4a0d8fa-8397-484c-ba1b-5f57527b1cb7@github.com> Message-ID: <6GcNNoJ9aHVuOrX951YEJUn4Lsx1S4c-4t4r-8nCxZs=.fcb2e27c-8648-4d39-b1c0-df0cc2fa04df@github.com> On Tue, 6 Apr 2021 14:43:13 GMT, Conor Cleary wrote: >> As the test is for Datagram send and receive all the data is in byte arrays or equivalent. >> The entire test could be written using only byte arrays (no Strings, Charsets or conversions) and omit that possibility of confusion about what the test is doing. >> (And there's no need to change anything). > > Alright then so I will continue to to just use `US_ASCII` in this test as "hello" is mappable with it. Encoding and Decoding the string will be done with `US_ASCII`. Appreciate the feedback guys, thanks! Thanks Conor. Yes - we could work with byte arrays but strings are easier to print & log & more human friendly... ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Tue Apr 6 14:57:54 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 14:57:54 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v2] In-Reply-To: References: Message-ID: > ### Description > SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. > > - Test is run in AgentVM mode. > - Receivers bind to wildcard/localhost which has been a source of instability. > - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. > - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. > > ### Fixes > To Address these issues, the following was done respective to the order of the Issues above: > - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. > - Receivers now bind to loopback address instead of localhost. > - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). > - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: 8264200: Use charset consistently & correct logs ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3354/files - new: https://git.openjdk.java.net/jdk/pull/3354/files/611e54b9..6ac1b2c3 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3354&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3354&range=00-01 Stats: 12 lines in 1 file changed: 1 ins; 1 del; 10 mod Patch: https://git.openjdk.java.net/jdk/pull/3354.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3354/head:pull/3354 PR: https://git.openjdk.java.net/jdk/pull/3354 From rriggs at openjdk.java.net Tue Apr 6 14:57:55 2021 From: rriggs at openjdk.java.net (Roger Riggs) Date: Tue, 6 Apr 2021 14:57:55 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v2] In-Reply-To: <6GcNNoJ9aHVuOrX951YEJUn4Lsx1S4c-4t4r-8nCxZs=.fcb2e27c-8648-4d39-b1c0-df0cc2fa04df@github.com> References: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> <09VepmgusTE1Rdyg1sumW9M-URJAg3Kcm19OmVXruDk=.997f7190-9e28-4016-8d09-a984cba752ca@github.com> <0jTHzHzp8lwRhi5lg8T5sFTcbQ9NaHAm_asZ3VVhysM=.a4a0d8fa-8397-484c-ba1b-5f57527b1cb7@github.com> <6GcNNoJ9aHVuOrX951YEJUn4Lsx1S4c-4t4r-8nCxZs=.fcb2e27c-8648-4d39-b1c0-df0cc2fa04df@github.com> Message-ID: On Tue, 6 Apr 2021 14:46:26 GMT, Daniel Fuchs wrote: >> Alright then so I will continue to to just use `US_ASCII` in this test as "hello" is mappable with it. Encoding and Decoding the string will be done with `US_ASCII`. Appreciate the feedback guys, thanks! > > Thanks Conor. Yes - we could work with byte arrays but strings are easier to print & log & more human friendly... ok, but please use getBytes(StandardCharsets.US_ASCII) to get the bytes avoiding ambiguity. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Tue Apr 6 14:57:56 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 14:57:56 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v2] In-Reply-To: References: <9bkcQovEZXYHgZIzyEvmJaeO1W18fA8FXOOBetjHXtc=.4c958701-77b6-454d-a4e1-404eb3dfba18@github.com> <09VepmgusTE1Rdyg1sumW9M-URJAg3Kcm19OmVXruDk=.997f7190-9e28-4016-8d09-a984cba752ca@github.com> <0jTHzHzp8lwRhi5lg8T5sFTcbQ9NaHAm_asZ3VVhysM=.a4a0d8fa-8397-484c-ba1b-5f57527b1cb7@github.com> <6GcNNoJ9aHVuOrX951YEJUn4Lsx1S4c-4t4r-8nCxZs=.fcb2e27c-8648-4d39-b1c0-df0cc2fa04df@github.com> Message-ID: On Tue, 6 Apr 2021 14:49:42 GMT, Roger Riggs wrote: >> Thanks Conor. Yes - we could work with byte arrays but strings are easier to print & log & more human friendly... > > ok, but please use getBytes(StandardCharsets.US_ASCII) to get the bytes avoiding ambiguity. The most recent commit performs a static import of the charset with `import static java.nio.charset.StandardCharsets.US_ASCII;`. I feel that it is clear enough and keeps references to the Charset concise. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From rriggs at openjdk.java.net Tue Apr 6 15:00:20 2021 From: rriggs at openjdk.java.net (Roger Riggs) Date: Tue, 6 Apr 2021 15:00:20 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 14:57:54 GMT, Conor Cleary wrote: >> ### Description >> SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. >> >> - Test is run in AgentVM mode. >> - Receivers bind to wildcard/localhost which has been a source of instability. >> - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. >> - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. >> >> ### Fixes >> To Address these issues, the following was done respective to the order of the Issues above: >> - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. >> - Receivers now bind to loopback address instead of localhost. >> - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). >> - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). > > Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: > > 8264200: Use charset consistently & correct logs Marked as reviewed by rriggs (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From dfuchs at openjdk.java.net Tue Apr 6 15:04:15 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Tue, 6 Apr 2021 15:04:15 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 14:57:54 GMT, Conor Cleary wrote: >> ### Description >> SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. >> >> - Test is run in AgentVM mode. >> - Receivers bind to wildcard/localhost which has been a source of instability. >> - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. >> - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. >> >> ### Fixes >> To Address these issues, the following was done respective to the order of the Issues above: >> - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. >> - Receivers now bind to loopback address instead of localhost. >> - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). >> - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). > > Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: > > 8264200: Use charset consistently & correct logs test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 195: > 193: DatagramPacket dp = new DatagramPacket(buf, buf.length); > 194: ds.receive(dp); > 195: String received = new String(dp.getData(), US_ASCII); That's not quite correct, you need to take into account the offset and length of the received data. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Tue Apr 6 16:53:26 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 16:53:26 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 15:00:03 GMT, Daniel Fuchs wrote: >> Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: >> >> 8264200: Use charset consistently & correct logs > > test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 195: > >> 193: DatagramPacket dp = new DatagramPacket(buf, buf.length); >> 194: ds.receive(dp); >> 195: String received = new String(dp.getData(), US_ASCII); > > That's not quite correct, you need to take into account the offset and length of the received data. Ah... good spot, this results in a needlessly long string. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Tue Apr 6 16:53:26 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 16:53:26 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 16:48:23 GMT, Conor Cleary wrote: >> test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 195: >> >>> 193: DatagramPacket dp = new DatagramPacket(buf, buf.length); >>> 194: ds.receive(dp); >>> 195: String received = new String(dp.getData(), US_ASCII); >> >> That's not quite correct, you need to take into account the offset and length of the received data. > > Ah... good spot, this results in a needlessly long string. Changing to `String received = new String(dp.getData(), 0, DATA_STRING.length(), US_ASCII);` seems to correct the issue, using a constant `DATA_STRING` ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From msheppar at openjdk.java.net Tue Apr 6 17:09:30 2021 From: msheppar at openjdk.java.net (Mark Sheppard) Date: Tue, 6 Apr 2021 17:09:30 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 14:57:38 GMT, Roger Riggs wrote: >> Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: >> >> 8264200: Use charset consistently & correct logs > > Marked as reviewed by rriggs (Reviewer). I note in the senders (ClassicWriter and NioWriter) there are two send invocation while in the receivers there is only one receive invocation. That raises a question as to why there are two send invocations, and which of these a receiver processes - no way of telling if the first has been lost? now a bit of conjecture: In theory based on the structure of the test the receiver or reader thread could finish before the writer have executed the second send. So if the receiver has finished, executed close and released their port, making it available for re-allocation in another concurrently executing test, then the second send could be a stray send to another now unrelated UDP end point. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Tue Apr 6 17:24:27 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 17:24:27 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v2] In-Reply-To: References: Message-ID: <5v3Br3l4PVXvk4x8a_xIhv6sfIX-hhh3zfLx_I7O3AY=.85bf14b0-6185-4d24-84c3-5bcf7a7ea233@github.com> On Tue, 6 Apr 2021 17:06:11 GMT, Mark Sheppard wrote: > I note in the senders (ClassicWriter and NioWriter) there are two send invocation while in the receivers there is only one receive invocation. That raises a question as to why there are two send invocations, and which of these a receiver processes - no way of telling if the first has been lost? > > now a bit of conjecture: > In theory based on the structure of the test the receiver or reader thread could finish before the writer have executed the second send. So if the receiver has finished, executed close and released their port, making it available for re-allocation in another concurrently executing test, then the second send could be a stray send to another now unrelated UDP end point. @msheppar it seems to me that the duplicate send is a feature from the old test which may no longer be needed. In the old test, the `invoke()` method contained the following code: static void invoke(Sprintable reader, Sprintable writer) throws Exception { Thread readerThread = new Thread(reader); readerThread.start(); Thread.sleep(50); Thread writerThread = new Thread(writer); writerThread.start(); ... This `thread.sleep(50)` I'm guessing is to ensure the readerThread has fully started and is waiting to receive before starting the writer thread. Following on from this, both writer classes (ClassicWriter & NioWriter) contain something like... dc.send(bb, isa); Thread.sleep(50); dc.send(bb, isa); I would subsequently guess that this serves to be extra sure that a Datagram reaches the reader. Assuming the first packet doesnt make it (probably unlikely), waiting for 50 milliseconds ensures that the reader is at the very least waiting to receive. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Tue Apr 6 17:46:53 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 17:46:53 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v3] In-Reply-To: References: Message-ID: > ### Description > SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. > > - Test is run in AgentVM mode. > - Receivers bind to wildcard/localhost which has been a source of instability. > - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. > - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. > > ### Fixes > To Address these issues, the following was done respective to the order of the Issues above: > - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. > - Receivers now bind to loopback address instead of localhost. > - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). > - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). Conor Cleary has updated the pull request incrementally with two additional commits since the last revision: - 8264200: Remove duplicate send invocations - 8264200: Construct String with data offset/length ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3354/files - new: https://git.openjdk.java.net/jdk/pull/3354/files/6ac1b2c3..2a5acfee Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3354&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3354&range=01-02 Stats: 11 lines in 1 file changed: 2 ins; 6 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3354.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3354/head:pull/3354 PR: https://git.openjdk.java.net/jdk/pull/3354 From msheppar at openjdk.java.net Tue Apr 6 17:50:28 2021 From: msheppar at openjdk.java.net (Mark Sheppard) Date: Tue, 6 Apr 2021 17:50:28 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v2] In-Reply-To: <5v3Br3l4PVXvk4x8a_xIhv6sfIX-hhh3zfLx_I7O3AY=.85bf14b0-6185-4d24-84c3-5bcf7a7ea233@github.com> References: <5v3Br3l4PVXvk4x8a_xIhv6sfIX-hhh3zfLx_I7O3AY=.85bf14b0-6185-4d24-84c3-5bcf7a7ea233@github.com> Message-ID: <8xqe2PBu2BA0lWDEJbCtSrQapENdjBxzjavoYay53EI=.73be4da2-f10a-4e78-810c-d45dc7e014d8@github.com> On Tue, 6 Apr 2021 17:20:56 GMT, Conor Cleary wrote: >> I note in the senders (ClassicWriter and NioWriter) there are two send invocation while in the receivers there is only one receive invocation. That raises a question as to why there are two send invocations, and which of these a receiver processes - no way of telling if the first has been lost? >> >> now a bit of conjecture: >> In theory based on the structure of the test the receiver or reader thread could finish before the writer have executed the second send. So if the receiver has finished, executed close and released their port, making it available for re-allocation in another concurrently executing test, then the second send could be a stray send to another now unrelated UDP end point. > >> I note in the senders (ClassicWriter and NioWriter) there are two send invocation while in the receivers there is only one receive invocation. That raises a question as to why there are two send invocations, and which of these a receiver processes - no way of telling if the first has been lost? >> >> now a bit of conjecture: >> In theory based on the structure of the test the receiver or reader thread could finish before the writer have executed the second send. So if the receiver has finished, executed close and released their port, making it available for re-allocation in another concurrently executing test, then the second send could be a stray send to another now unrelated UDP end point. > > @msheppar it seems to me that the duplicate send is a feature from the old test which may no longer be needed. In the old test, the `invoke()` method contained the following code: > > static void invoke(Sprintable reader, Sprintable writer) throws Exception { > Thread readerThread = new Thread(reader); > readerThread.start(); > Thread.sleep(50); > > Thread writerThread = new Thread(writer); > writerThread.start(); > ... > This `thread.sleep(50)` I'm guessing is to ensure the readerThread has fully started and is waiting to receive before starting the writer thread. Following on from this, both writer classes (ClassicWriter & NioWriter) contain something like... > dc.send(bb, isa); > Thread.sleep(50); > dc.send(bb, isa); > > I would subsequently guess that this serves to be extra sure that a Datagram reaches the reader. Assuming the first packet doesnt make it (probably unlikely), waiting for 50 milliseconds ensures that the reader is at the very least waiting to receive. Yes, the original test had this conundrum. But for the refactored test there still exists the possibility that the second send could be received by another test process, because the test's receiver has completed and released its socket resources before the second send has been invoked. In your refactored test that possibility is diminished as you start the writer thread prior to the reader thread. While in the original test it was reader thread first and writer next. But as with all multithreaded scenarios there is a strong element of non determinism so the possibility still remains. As such there is no synchronicity between the sender and the receiver in the test, other than the receiver may block indefinitely if a datagram is not received, which is now diminished by using the loopback as the receiver's bind address and as the destination address. But the rationale for invoking two sends and one receive is obscure and still remains a potential, if somewhat rare, problem. So I'd proffer some symmetry between the sender and receiver with one send and one receive, or that the receiver should remain extant until the sender has terminated, as such it would wait on "signal" from the sender that it has finished. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From dfuchs at openjdk.java.net Tue Apr 6 19:01:21 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Tue, 6 Apr 2021 19:01:21 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v3] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 17:46:53 GMT, Conor Cleary wrote: >> ### Description >> SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. >> >> - Test is run in AgentVM mode. >> - Receivers bind to wildcard/localhost which has been a source of instability. >> - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. >> - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. >> >> ### Fixes >> To Address these issues, the following was done respective to the order of the Issues above: >> - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. >> - Receivers now bind to loopback address instead of localhost. >> - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). >> - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). > > Conor Cleary has updated the pull request incrementally with two additional commits since the last revision: > > - 8264200: Remove duplicate send invocations > - 8264200: Construct String with data offset/length test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 191: > 189: DatagramPacket dp = new DatagramPacket(buf, buf.length); > 190: ds.receive(dp); > 191: String received = new String(dp.getData(), 0, DATA_STRING.length(), US_ASCII); That's not quite correct either. The `DatagramPacket` should have its length and offset positioned correctly after `receive` - that's the values you should use (`p.getOffset()`, `p.getLength()`). ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Tue Apr 6 19:41:32 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 19:41:32 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v3] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 18:57:37 GMT, Daniel Fuchs wrote: >> Conor Cleary has updated the pull request incrementally with two additional commits since the last revision: >> >> - 8264200: Remove duplicate send invocations >> - 8264200: Construct String with data offset/length > > test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 191: > >> 189: DatagramPacket dp = new DatagramPacket(buf, buf.length); >> 190: ds.receive(dp); >> 191: String received = new String(dp.getData(), 0, DATA_STRING.length(), US_ASCII); > > That's not quite correct either. The `DatagramPacket` should have its length and offset positioned correctly after `receive` - that's the values you should use (`p.getOffset()`, `p.getLength()`). Ah ok understood, I can see that is more correct now. Will change shortly ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Tue Apr 6 20:20:53 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 6 Apr 2021 20:20:53 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v4] In-Reply-To: References: Message-ID: <_ece4mFrjesKXz4bMfwozRp4l7jbrYfxUxboLc1WMik=.418bdb44-9c39-429d-99a0-6ca66984547e@github.com> > ### Description > SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. > > - Test is run in AgentVM mode. > - Receivers bind to wildcard/localhost which has been a source of instability. > - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. > - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. > > ### Fixes > To Address these issues, the following was done respective to the order of the Issues above: > - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. > - Receivers now bind to loopback address instead of localhost. > - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). > - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: 8264200: Use datagram packet for string construction ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3354/files - new: https://git.openjdk.java.net/jdk/pull/3354/files/2a5acfee..a2e25979 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3354&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3354&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3354.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3354/head:pull/3354 PR: https://git.openjdk.java.net/jdk/pull/3354 From bpb at openjdk.java.net Tue Apr 6 21:38:30 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 6 Apr 2021 21:38:30 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 17:19:35 GMT, Brian Burkhalter wrote: >> I already have this information. >> >> 1. `C:` - Win10 system drive >> >> `Root: C:\ >> volName: System, fsName: NTFS, volSN: 1620920694, volType: 3` >> >> 2. `E:` - CD drive (VirtualBox Guest Additions) >> >> `Root: E:\ >> volName: VBox_GAs_6.1.18, fsName: CDFS, volSN: -1657170319, volType: 5` >> >> 3. `Z:` - VirtualBox shared folder >> >> `Root: Z:\ >> volName: VBOX_VirtualBox, fsName: VBoxSharedFolderFS, volSN: 16777221, volType: >> 4` >> >> 4. `\\BPB\Public` - Local share created for `C:\Users\Public` >> >> `Root: \\BPB\Public\ >> volName: System, fsName: NTFS, volSN: 1620920694, volType: 4` >> >> 5. `\\192.168.0.111\Shared` - "Remote" share of macOS host `/Users/Shared` >> >> `Root: \\192.168.0.111\Shared\ >> volName: Macintosh HD - Data, fsName: NTFS, volSN: 0, volType: 4` > > I already tested with other cases but didn't include it: > > Root: \\bpb\Public\ > volName: System, fsName: NTFS, volSN: 1620920694, volType: 4 > > Root: \\BPB\pUBLIC\ > volName: System, fsName: NTFS, volSN: 1620920694, volType: 4 > > Root: \\192.168.0.111\Shared\ > volName: Macintosh HD - Data, fsName: NTFS, volSN: 0, volType: 4 > > Root: \\192.168.0.111\shared\ > volName: Macintosh HD - Data, fsName: NTFS, volSN: 0, volType: 4 > > Good point about `hashCode()`. If we want to constrain the case insensitive comparison to fixed drives, then this might work: @Override public boolean equals(Object ob) { if (ob == this) return true; if (!(ob instanceof WindowsFileStore)) return false; WindowsFileStore other = (WindowsFileStore)ob; if (root.equals(other.root)) return true; if (volType == DRIVE_FIXED && other.volumeType() == DRIVE_FIXED) return root.equalsIgnoreCase(other.root); return false; } @Override public int hashCode() { return volType == DRIVE_FIXED ? root.toLowerCase().hashCode() : root.hashCode(); } ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Wed Apr 7 02:41:56 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 7 Apr 2021 02:41:56 GMT Subject: RFR: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) Message-ID: Apparently `fcntl(fd, F_FULLFSYNC)` can fail with `ENOTTY` in addition to `ENOTSUP` although it is not so documented. The `ioctl()` case is documented as [ENOTTY] fildes is not associated with a character special device. [ENOTTY] The specified request does not apply to the kind of object that the descriptor fildes references. This request proposes to catch `ENOTTY` as well and fall back to `fsync(fd)`. The change was verified manually using a WebDAV server and no failures in CI test tiers 1-3 were observed. An alternative would be to fall back to `fsync(fd)` for **all** errors. ------------- Commit messages: - 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) Changes: https://git.openjdk.java.net/jdk/pull/3366/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3366&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8232861 Stats: 5 lines in 1 file changed: 2 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3366.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3366/head:pull/3366 PR: https://git.openjdk.java.net/jdk/pull/3366 From alanb at openjdk.java.net Wed Apr 7 07:39:21 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 7 Apr 2021 07:39:21 GMT Subject: RFR: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 02:35:18 GMT, Brian Burkhalter wrote: > Apparently `fcntl(fd, F_FULLFSYNC)` can fail with `ENOTTY` in addition to `ENOTSUP` although it is not so documented. The `ioctl()` case is documented as > [ENOTTY] fildes is not associated with a character special > device. > [ENOTTY] The specified request does not apply to the kind of > object that the descriptor fildes references. > > This request proposes to catch `ENOTTY` as well and fall back to `fsync(fd)`. The change was verified manually using a WebDAV server and no failures in CI test tiers 1-3 were observed. An alternative would be to fall back to `fsync(fd)` for **all** errors. fcntl(fd, F_FULLFSYNC) is the semantics we want for local files. If it fails then the force method has to throw. The force method doesn't provide any guarantees when the file is remote so using fsync for the WebDAV case is okay. I can't find any man pages or other documentation that documents ENOTTY. I think I would be more comfortable with an approach that somehow detects the file is local or not, that would allow it to fallback to fsync when not local. ------------- PR: https://git.openjdk.java.net/jdk/pull/3366 From dfuchs at openjdk.java.net Wed Apr 7 08:29:27 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Wed, 7 Apr 2021 08:29:27 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v4] In-Reply-To: <_ece4mFrjesKXz4bMfwozRp4l7jbrYfxUxboLc1WMik=.418bdb44-9c39-429d-99a0-6ca66984547e@github.com> References: <_ece4mFrjesKXz4bMfwozRp4l7jbrYfxUxboLc1WMik=.418bdb44-9c39-429d-99a0-6ca66984547e@github.com> Message-ID: <7LpyXZoyHLqRYOZU25O6HtwiUT4I_NJ_ZFILFvX6Ddk=.c852eb95-d08e-43e1-b86d-91b1c4ce5b57@github.com> On Tue, 6 Apr 2021 20:20:53 GMT, Conor Cleary wrote: >> ### Description >> SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. >> >> - Test is run in AgentVM mode. >> - Receivers bind to wildcard/localhost which has been a source of instability. >> - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. >> - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. >> >> ### Fixes >> To Address these issues, the following was done respective to the order of the Issues above: >> - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. >> - Receivers now bind to loopback address instead of localhost. >> - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). >> - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). > > Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: > > 8264200: Use datagram packet for string construction LGTM! ------------- Marked as reviewed by dfuchs (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3354 From alanb at openjdk.java.net Wed Apr 7 11:32:42 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 7 Apr 2021 11:32:42 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 21:35:14 GMT, Brian Burkhalter wrote: > If we want to constrain the case insensitive comparison to fixed drives, then this might work: Yes, although it would be good to extend this to the DRIVE_REMOTE and other cases too, but that requires being confident that it is correct (and I'm not sure we have enough information to say this yet). An initial change for the DRIVE_FIXED case is okay. It will mean using Locale.ROOT or Locale.US. Also the hash code will need to be cached. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From msheppar at openjdk.java.net Wed Apr 7 11:53:12 2021 From: msheppar at openjdk.java.net (Mark Sheppard) Date: Wed, 7 Apr 2021 11:53:12 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v4] In-Reply-To: <7LpyXZoyHLqRYOZU25O6HtwiUT4I_NJ_ZFILFvX6Ddk=.c852eb95-d08e-43e1-b86d-91b1c4ce5b57@github.com> References: <_ece4mFrjesKXz4bMfwozRp4l7jbrYfxUxboLc1WMik=.418bdb44-9c39-429d-99a0-6ca66984547e@github.com> <7LpyXZoyHLqRYOZU25O6HtwiUT4I_NJ_ZFILFvX6Ddk=.c852eb95-d08e-43e1-b86d-91b1c4ce5b57@github.com> Message-ID: On Wed, 7 Apr 2021 08:26:28 GMT, Daniel Fuchs wrote: >> Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: >> >> 8264200: Use datagram packet for string construction > > LGTM! another observation on the test structure is that each sender is bound to wildcard address, ClassicWriter explicitly through constructor and the NioWriter implicitly when it invokes send - the DatagramChannel is unbound initially, no explicit bind is invoked, and when send is invoked then the underlying socket is bound to wildcard during that call flow. There are some subtleties when sending from a socket bound to a wildcard address bound - in this case this is minimal as the recipient address is the loopback address. One possible consideration is to explicitly bind the sender to the loopback address also? Both ClassicWriter and NioWriter constructors take a port, which is the destination port of the recipient rather than a port to which they'll bind -- maybe a rename of the parameter to dstPort ? ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From dfuchs at openjdk.java.net Wed Apr 7 13:04:24 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Wed, 7 Apr 2021 13:04:24 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v4] In-Reply-To: References: <_ece4mFrjesKXz4bMfwozRp4l7jbrYfxUxboLc1WMik=.418bdb44-9c39-429d-99a0-6ca66984547e@github.com> <7LpyXZoyHLqRYOZU25O6HtwiUT4I_NJ_ZFILFvX6Ddk=.c852eb95-d08e-43e1-b86d-91b1c4ce5b57@github.com> Message-ID: On Wed, 7 Apr 2021 11:50:45 GMT, Mark Sheppard wrote: >> LGTM! > > another observation on the test structure is that each sender is bound to wildcard address, ClassicWriter explicitly through constructor and the NioWriter implicitly when it invokes send - the DatagramChannel is unbound initially, no explicit bind is invoked, and when send is invoked then the underlying socket is bound to wildcard during that call flow. > There are some subtleties when sending from a socket bound to a wildcard address bound - in this case this is minimal as the recipient address is the loopback address. > One possible consideration is to explicitly bind the sender to the loopback address also? > > Both ClassicWriter and NioWriter constructors take a port, which is the destination port of the recipient rather than a port to which they'll bind -- maybe a rename of the parameter to dstPort ? @msheppar I would rather not change the way that the senders are bound, unless it is verified that this is causing problems and we are able to figure out why. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From alanb at openjdk.java.net Wed Apr 7 13:51:26 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 7 Apr 2021 13:51:26 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 13:22:44 GMT, Conor Cleary wrote: > This fix addresses the following warnings which were generated by building JDK API documentation with the `-Xdoclint:all` option enabled: > > ./build/linux-x64/support/gensrc/java.base/java/nio/charset/IllegalCharsetNameException.java:47: warning: no comment > private String charsetName; > ^ > ./open/src/java.base/share/classes/java/nio/charset/MalformedInputException.java:44: warning: no comment > private int inputLength; > ^ > ./open/src/java.base/share/classes/java/nio/charset/UnmappableCharacterException.java:44: warning: no comment > private int inputLength; > ^ > ./build/linux-x64/support/gensrc/java.base/java/nio/charset/UnsupportedCharsetException.java:47: warning: no comment > private String charsetName; > ^ > ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @param for s > private void readObject(ObjectInputStream s) > ^ > ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @throws for java.lang.ClassNotFoundException > private void readObject(ObjectInputStream s) > ^ > ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:43: warning: no comment > private final String file; > ^ > ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:44: warning: no comment > private final String other; > ^ > ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:43: warning: no comment > private int index; > ^ > ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:42: warning: no comment > private String input; > ^ > ./open/src/java.base/share/classes/java/nio/file/attribute/UserPrincipalNotFoundException.java:43: warning: no comment > private final String name; > ^ > Changes to [`genExceptions.sh`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) and the two [`exceptions`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) templates are to address the warnings concerned with `UnsupportedCharsetException.java` and `IllegalCharsetNameException.java` which are generated when `make jdk-image` is run. A CSR will be filed in due course with respect to these changes. src/java.base/share/classes/java/nio/file/InvalidPathException.java line 44: > 42: /** > 43: * The invalid input path string. > 44: */ This should probably be "The input string" to be consistent with the getInput method. src/java.base/share/classes/java/nio/file/attribute/UserPrincipalNotFoundException.java line 44: > 42: > 43: /** > 44: * The name of the {@code UserPrincipal} that does not exist. Probably best to use "user principal name" so that it is consistent with getName. ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From ccleary at openjdk.java.net Wed Apr 7 14:22:29 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Wed, 7 Apr 2021 14:22:29 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 13:48:00 GMT, Alan Bateman wrote: >> This fix addresses the following warnings which were generated by building JDK API documentation with the `-Xdoclint:all` option enabled: >> >> ./build/linux-x64/support/gensrc/java.base/java/nio/charset/IllegalCharsetNameException.java:47: warning: no comment >> private String charsetName; >> ^ >> ./open/src/java.base/share/classes/java/nio/charset/MalformedInputException.java:44: warning: no comment >> private int inputLength; >> ^ >> ./open/src/java.base/share/classes/java/nio/charset/UnmappableCharacterException.java:44: warning: no comment >> private int inputLength; >> ^ >> ./build/linux-x64/support/gensrc/java.base/java/nio/charset/UnsupportedCharsetException.java:47: warning: no comment >> private String charsetName; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @param for s >> private void readObject(ObjectInputStream s) >> ^ >> ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @throws for java.lang.ClassNotFoundException >> private void readObject(ObjectInputStream s) >> ^ >> ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:43: warning: no comment >> private final String file; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:44: warning: no comment >> private final String other; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:43: warning: no comment >> private int index; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:42: warning: no comment >> private String input; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/attribute/UserPrincipalNotFoundException.java:43: warning: no comment >> private final String name; >> ^ >> Changes to [`genExceptions.sh`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) and the two [`exceptions`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) templates are to address the warnings concerned with `UnsupportedCharsetException.java` and `IllegalCharsetNameException.java` which are generated when `make jdk-image` is run. A CSR will be filed in due course with respect to these changes. > > src/java.base/share/classes/java/nio/file/attribute/UserPrincipalNotFoundException.java line 44: > >> 42: >> 43: /** >> 44: * The name of the {@code UserPrincipal} that does not exist. > > Probably best to use "user principal name" so that it is consistent with getName. Thanks Alan, will update as you suggest shortly for both your comments (also thanks for adding the nio label!) ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From ccleary at openjdk.java.net Wed Apr 7 14:40:48 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Wed, 7 Apr 2021 14:40:48 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio [v2] In-Reply-To: References: Message-ID: > This fix addresses the following warnings which were generated by building JDK API documentation with the `-Xdoclint:all` option enabled: > > ./build/linux-x64/support/gensrc/java.base/java/nio/charset/IllegalCharsetNameException.java:47: warning: no comment > private String charsetName; > ^ > ./open/src/java.base/share/classes/java/nio/charset/MalformedInputException.java:44: warning: no comment > private int inputLength; > ^ > ./open/src/java.base/share/classes/java/nio/charset/UnmappableCharacterException.java:44: warning: no comment > private int inputLength; > ^ > ./build/linux-x64/support/gensrc/java.base/java/nio/charset/UnsupportedCharsetException.java:47: warning: no comment > private String charsetName; > ^ > ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @param for s > private void readObject(ObjectInputStream s) > ^ > ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @throws for java.lang.ClassNotFoundException > private void readObject(ObjectInputStream s) > ^ > ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:43: warning: no comment > private final String file; > ^ > ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:44: warning: no comment > private final String other; > ^ > ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:43: warning: no comment > private int index; > ^ > ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:42: warning: no comment > private String input; > ^ > ./open/src/java.base/share/classes/java/nio/file/attribute/UserPrincipalNotFoundException.java:43: warning: no comment > private final String name; > ^ > Changes to [`genExceptions.sh`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) and the two [`exceptions`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) templates are to address the warnings concerned with `UnsupportedCharsetException.java` and `IllegalCharsetNameException.java` which are generated when `make jdk-image` is run. A CSR will be filed in due course with respect to these changes. Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: 8264779: Simplified comments ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3376/files - new: https://git.openjdk.java.net/jdk/pull/3376/files/b729d8ed..0b879f15 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3376&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3376&range=00-01 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3376.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3376/head:pull/3376 PR: https://git.openjdk.java.net/jdk/pull/3376 From iris at openjdk.java.net Wed Apr 7 16:17:24 2021 From: iris at openjdk.java.net (Iris Clark) Date: Wed, 7 Apr 2021 16:17:24 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 14:40:48 GMT, Conor Cleary wrote: >> This fix addresses the following warnings which were generated by building JDK API documentation with the `-Xdoclint:all` option enabled: >> >> ./build/linux-x64/support/gensrc/java.base/java/nio/charset/IllegalCharsetNameException.java:47: warning: no comment >> private String charsetName; >> ^ >> ./open/src/java.base/share/classes/java/nio/charset/MalformedInputException.java:44: warning: no comment >> private int inputLength; >> ^ >> ./open/src/java.base/share/classes/java/nio/charset/UnmappableCharacterException.java:44: warning: no comment >> private int inputLength; >> ^ >> ./build/linux-x64/support/gensrc/java.base/java/nio/charset/UnsupportedCharsetException.java:47: warning: no comment >> private String charsetName; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @param for s >> private void readObject(ObjectInputStream s) >> ^ >> ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @throws for java.lang.ClassNotFoundException >> private void readObject(ObjectInputStream s) >> ^ >> ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:43: warning: no comment >> private final String file; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:44: warning: no comment >> private final String other; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:43: warning: no comment >> private int index; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:42: warning: no comment >> private String input; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/attribute/UserPrincipalNotFoundException.java:43: warning: no comment >> private final String name; >> ^ >> Changes to [`genExceptions.sh`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) and the two [`exceptions`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) templates are to address the warnings concerned with `UnsupportedCharsetException.java` and `IllegalCharsetNameException.java` which are generated when `make jdk-image` is run. A CSR will be filed in due course with respect to these changes. > > Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: > > 8264779: Simplified comments Marked as reviewed by iris (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From naoto at openjdk.java.net Wed Apr 7 16:21:45 2021 From: naoto at openjdk.java.net (Naoto Sato) Date: Wed, 7 Apr 2021 16:21:45 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 14:40:48 GMT, Conor Cleary wrote: >> This fix addresses the following warnings which were generated by building JDK API documentation with the `-Xdoclint:all` option enabled: >> >> ./build/linux-x64/support/gensrc/java.base/java/nio/charset/IllegalCharsetNameException.java:47: warning: no comment >> private String charsetName; >> ^ >> ./open/src/java.base/share/classes/java/nio/charset/MalformedInputException.java:44: warning: no comment >> private int inputLength; >> ^ >> ./open/src/java.base/share/classes/java/nio/charset/UnmappableCharacterException.java:44: warning: no comment >> private int inputLength; >> ^ >> ./build/linux-x64/support/gensrc/java.base/java/nio/charset/UnsupportedCharsetException.java:47: warning: no comment >> private String charsetName; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @param for s >> private void readObject(ObjectInputStream s) >> ^ >> ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @throws for java.lang.ClassNotFoundException >> private void readObject(ObjectInputStream s) >> ^ >> ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:43: warning: no comment >> private final String file; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:44: warning: no comment >> private final String other; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:43: warning: no comment >> private int index; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:42: warning: no comment >> private String input; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/attribute/UserPrincipalNotFoundException.java:43: warning: no comment >> private final String name; >> ^ >> Changes to [`genExceptions.sh`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) and the two [`exceptions`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) templates are to address the warnings concerned with `UnsupportedCharsetException.java` and `IllegalCharsetNameException.java` which are generated when `make jdk-image` is run. A CSR will be filed in due course with respect to these changes. > > Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: > > 8264779: Simplified comments src/java.base/share/classes/java/nio/charset/MalformedInputException.java line 45: > 43: > 44: /** > 45: * The length of the input byte sequence. Should this comment also refer to the character sequence? src/java.base/share/classes/java/nio/charset/exceptions line 31: > 29: PACKAGE=java.nio.charset > 30: # This year should only change if the generated source is modified. > 31: COPYRIGHT_YEARS="2000, 2021," Do those modifications to the exception sources end up in a change in `generated` sources? src/java.base/share/classes/java/nio/exceptions line 31: > 29: PACKAGE=java.nio > 30: # This year should only change if the generated source is modified. > 31: COPYRIGHT_YEARS="2000, 2021," Does not seem necessary, as I don't see any changes in java.nio package. ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From bpb at openjdk.java.net Wed Apr 7 16:37:27 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 7 Apr 2021 16:37:27 GMT Subject: RFR: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) [v2] In-Reply-To: References: Message-ID: > Apparently `fcntl(fd, F_FULLFSYNC)` can fail with `ENOTTY` in addition to `ENOTSUP` although it is not so documented. The `ioctl()` case is documented as > [ENOTTY] fildes is not associated with a character special > device. > [ENOTTY] The specified request does not apply to the kind of > object that the descriptor fildes references. > > This request proposes to catch `ENOTTY` as well and fall back to `fsync(fd)`. The change was verified manually using a WebDAV server and no failures in CI test tiers 1-3 were observed. An alternative would be to fall back to `fsync(fd)` for **all** errors. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8232861: Fallback on F_FULLSYNC if file is remote instead of if errno is ENOTTY ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3366/files - new: https://git.openjdk.java.net/jdk/pull/3366/files/cbb1ba92..cb13e940 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3366&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3366&range=00-01 Stats: 15 lines in 1 file changed: 10 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/3366.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3366/head:pull/3366 PR: https://git.openjdk.java.net/jdk/pull/3366 From alanb at openjdk.java.net Wed Apr 7 16:49:29 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 7 Apr 2021 16:49:29 GMT Subject: RFR: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 16:37:27 GMT, Brian Burkhalter wrote: >> Apparently `fcntl(fd, F_FULLFSYNC)` can fail with `ENOTTY` in addition to `ENOTSUP` although it is not so documented. The `ioctl()` case is documented as >> [ENOTTY] fildes is not associated with a character special >> device. >> [ENOTTY] The specified request does not apply to the kind of >> object that the descriptor fildes references. >> >> This request proposes to catch `ENOTTY` as well and fall back to `fsync(fd)`. The change was verified manually using a WebDAV server and no failures in CI test tiers 1-3 were observed. An alternative would be to fall back to `fsync(fd)` for **all** errors. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8232861: Fallback on F_FULLSYNC if file is remote instead of if errno is ENOTTY src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c line 184: > 182: result = fsync(fd); > 183: } > 184: } Do you need to save/restore errno so that the original error from fcntl(F_FULLSYNC) is used in the error handling? Also what you would think about dropping the check for ENOTSUP. ------------- PR: https://git.openjdk.java.net/jdk/pull/3366 From alanb at openjdk.java.net Wed Apr 7 16:54:45 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 7 Apr 2021 16:54:45 GMT Subject: RFR: JDK-8262957 (fs) Fail fast in UnixFileStore.isExtendedAttributesEnabled In-Reply-To: References: Message-ID: <-FT7LavNSb-NViqSfsISQvOUoic51dQsxuaQPBoirCI=.090bd820-2ef8-49d6-973e-8fba3278cab6@github.com> On Wed, 3 Mar 2021 22:15:38 GMT, Sebastian Stenzel wrote: > Added new capability flag `UnixNativeDispatcher.SUPPORTS_XATTR`, which gets set `#ifdef _SYS_XATTR_H_`. > > Note that `_SYS_XATTR_H_` is defined in `xattr.h` in both [macOS/Darwin](https://github.com/apple/darwin-xnu/blob/main/bsd/sys/xattr.h), [Linux and GNU-based BSD systems using glibc](https://github.com/bminor/glibc/blob/master/misc/sys/xattr.h). It might not be defined for other operating systems that still support xattr. So if OpenJDK eventually adds support for further platforms, this might need to be adjusted as well. > > If xattr capabilities are missing, `UnixFileStore.isExtendedAttributesEnabled` will return false immediately, avoiding any I/O. > > As a side effect of this change, I redefined some other (private) capabilities. Strictly speaking, this change is not required but keeps the code consistent, as [previously discussed in the mailing list](https://mail.openjdk.java.net/pipermail/nio-dev/2021-February/008133.html). Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/2816 From bpb at openjdk.java.net Wed Apr 7 16:54:49 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 7 Apr 2021 16:54:49 GMT Subject: RFR: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 16:46:12 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8232861: Fallback on F_FULLSYNC if file is remote instead of if errno is ENOTTY > > src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c line 184: > >> 182: result = fsync(fd); >> 183: } >> 184: } > > Do you need to save/restore errno so that the original error from fcntl(F_FULLSYNC) is used in the error handling? > > Also what you would think about dropping the check for ENOTSUP. I thought about `errno` but didn't know whether it's common practice to cache it like that. We had `ENOTSUP` I suppose in case the `F_FULLSYNC` is unsupported on a local file system. From the man page of `fcntl()` the doc of `F_FULLSYNC` states This is currently implemented on HFS, MS-DOS (FAT), Universal Disk Format (UDF) and APFS file systems although `ENOTSUP` itself is not actually documented here. ------------- PR: https://git.openjdk.java.net/jdk/pull/3366 From bpb at openjdk.java.net Wed Apr 7 17:10:05 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 7 Apr 2021 17:10:05 GMT Subject: RFR: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) [v3] In-Reply-To: References: Message-ID: > Apparently `fcntl(fd, F_FULLFSYNC)` can fail with `ENOTTY` in addition to `ENOTSUP` although it is not so documented. The `ioctl()` case is documented as > [ENOTTY] fildes is not associated with a character special > device. > [ENOTTY] The specified request does not apply to the kind of > object that the descriptor fildes references. > > This request proposes to catch `ENOTTY` as well and fall back to `fsync(fd)`. The change was verified manually using a WebDAV server and no failures in CI test tiers 1-3 were observed. An alternative would be to fall back to `fsync(fd)` for **all** errors. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8232861: Restore errno from fcntl() if fstatfs() fails ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3366/files - new: https://git.openjdk.java.net/jdk/pull/3366/files/cb13e940..0c01b23d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3366&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3366&range=01-02 Stats: 9 lines in 1 file changed: 6 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3366.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3366/head:pull/3366 PR: https://git.openjdk.java.net/jdk/pull/3366 From ccleary at openjdk.java.net Wed Apr 7 18:04:41 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Wed, 7 Apr 2021 18:04:41 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 16:09:22 GMT, Naoto Sato wrote: >> Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: >> >> 8264779: Simplified comments > > src/java.base/share/classes/java/nio/charset/MalformedInputException.java line 45: > >> 43: >> 44: /** >> 45: * The length of the input byte sequence. > > Should this comment also refer to the character sequence? It could, maybe something like "The length of the input byte (or character) sequence." would work? > src/java.base/share/classes/java/nio/exceptions line 31: > >> 29: PACKAGE=java.nio >> 30: # This year should only change if the generated source is modified. >> 31: COPYRIGHT_YEARS="2000, 2021," > > Does not seem necessary, as I don't see any changes in java.nio package. This makes changes to the copyright headers of of the generated source files `UnsupportedCharsetException.java` and `IllegalCharsetNameException.java`. These can be found under a jdk build in `support/gensrc` and the headers update after running `make jdk-image`. Perhaps the build is out of date? These changes are harder to track as the source is not checked in. ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From alanb at openjdk.java.net Wed Apr 7 18:08:56 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 7 Apr 2021 18:08:56 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio [v2] In-Reply-To: References: Message-ID: <-pYKVTpbmpbgrNurQIiM184IGTYqaOS0LWh5TzziV7E=.37390828-1920-48da-9f73-4fde1d1f0734@github.com> On Wed, 7 Apr 2021 14:40:48 GMT, Conor Cleary wrote: >> This fix addresses the following warnings which were generated by building JDK API documentation with the `-Xdoclint:all` option enabled: >> >> ./build/linux-x64/support/gensrc/java.base/java/nio/charset/IllegalCharsetNameException.java:47: warning: no comment >> private String charsetName; >> ^ >> ./open/src/java.base/share/classes/java/nio/charset/MalformedInputException.java:44: warning: no comment >> private int inputLength; >> ^ >> ./open/src/java.base/share/classes/java/nio/charset/UnmappableCharacterException.java:44: warning: no comment >> private int inputLength; >> ^ >> ./build/linux-x64/support/gensrc/java.base/java/nio/charset/UnsupportedCharsetException.java:47: warning: no comment >> private String charsetName; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @param for s >> private void readObject(ObjectInputStream s) >> ^ >> ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @throws for java.lang.ClassNotFoundException >> private void readObject(ObjectInputStream s) >> ^ >> ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:43: warning: no comment >> private final String file; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:44: warning: no comment >> private final String other; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:43: warning: no comment >> private int index; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:42: warning: no comment >> private String input; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/attribute/UserPrincipalNotFoundException.java:43: warning: no comment >> private final String name; >> ^ >> Changes to [`genExceptions.sh`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) and the two [`exceptions`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) templates are to address the warnings concerned with `UnsupportedCharsetException.java` and `IllegalCharsetNameException.java` which are generated when `make jdk-image` is run. A CSR will be filed in due course with respect to these changes. > > Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: > > 8264779: Simplified comments Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From alanb at openjdk.java.net Wed Apr 7 18:08:57 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 7 Apr 2021 18:08:57 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 18:01:25 GMT, Conor Cleary wrote: >> src/java.base/share/classes/java/nio/charset/MalformedInputException.java line 45: >> >>> 43: >>> 44: /** >>> 45: * The length of the input byte sequence. >> >> Should this comment also refer to the character sequence? > > It could, maybe something like "The length of the input byte (or character) sequence." would work? or just "The length of the input" so that it is consistent with the description of the getInputLength method. ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From naoto at openjdk.java.net Wed Apr 7 18:15:40 2021 From: naoto at openjdk.java.net (Naoto Sato) Date: Wed, 7 Apr 2021 18:15:40 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 18:04:48 GMT, Alan Bateman wrote: >> It could, maybe something like "The length of the input byte (or character) sequence." would work? > > or just "The length of the input" so that it is consistent with the description of the getInputLength method. That sounds good. ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From naoto at openjdk.java.net Wed Apr 7 18:15:41 2021 From: naoto at openjdk.java.net (Naoto Sato) Date: Wed, 7 Apr 2021 18:15:41 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 17:58:58 GMT, Conor Cleary wrote: >> src/java.base/share/classes/java/nio/exceptions line 31: >> >>> 29: PACKAGE=java.nio >>> 30: # This year should only change if the generated source is modified. >>> 31: COPYRIGHT_YEARS="2000, 2021," >> >> Does not seem necessary, as I don't see any changes in java.nio package. > > This makes changes to the copyright headers of of the generated source files `UnsupportedCharsetException.java` and `IllegalCharsetNameException.java`. These can be found under a jdk build in `support/gensrc` and the headers update after running `make jdk-image`. Perhaps the build is out of date? These changes are harder to track as the source is not checked in. That's right. Sorry for the false alarm. ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From alanb at openjdk.java.net Wed Apr 7 18:19:38 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 7 Apr 2021 18:19:38 GMT Subject: RFR: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 16:51:26 GMT, Brian Burkhalter wrote: >> src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c line 184: >> >>> 182: result = fsync(fd); >>> 183: } >>> 184: } >> >> Do you need to save/restore errno so that the original error from fcntl(F_FULLSYNC) is used in the error handling? >> >> Also what you would think about dropping the check for ENOTSUP. > > I thought about `errno` but didn't know whether it's common practice to cache it like that. > > We had `ENOTSUP` I suppose in case the `F_FULLSYNC` is unsupported on a local file system. From the man page of `fcntl()` the doc of `F_FULLSYNC` states > This is currently implemented on HFS, MS-DOS (FAT), Universal Disk Format (UDF) and APFS file systems > although `ENOTSUP` itself is not actually documented here. I think this looks good, I'm just wondering to myself if it is possible to get ENOTSUP from a local device. ------------- PR: https://git.openjdk.java.net/jdk/pull/3366 From bpb at openjdk.java.net Wed Apr 7 18:19:38 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 7 Apr 2021 18:19:38 GMT Subject: RFR: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 18:15:24 GMT, Alan Bateman wrote: >> I thought about `errno` but didn't know whether it's common practice to cache it like that. >> >> We had `ENOTSUP` I suppose in case the `F_FULLSYNC` is unsupported on a local file system. From the man page of `fcntl()` the doc of `F_FULLSYNC` states >> This is currently implemented on HFS, MS-DOS (FAT), Universal Disk Format (UDF) and APFS file systems >> although `ENOTSUP` itself is not actually documented here. > > I think this looks good, I'm just wondering to myself if it is possible to get ENOTSUP from a local device. I am going to try to test this. ------------- PR: https://git.openjdk.java.net/jdk/pull/3366 From bpb at openjdk.java.net Wed Apr 7 18:35:51 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 7 Apr 2021 18:35:51 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 11:29:29 GMT, Alan Bateman wrote: >> If we want to constrain the case insensitive comparison to fixed drives, then this might work: >> @Override >> public boolean equals(Object ob) { >> if (ob == this) >> return true; >> if (!(ob instanceof WindowsFileStore)) >> return false; >> WindowsFileStore other = (WindowsFileStore)ob; >> if (root.equals(other.root)) >> return true; >> if (volType == DRIVE_FIXED && other.volumeType() == DRIVE_FIXED) >> return root.equalsIgnoreCase(other.root); >> return false; >> } >> >> @Override >> public int hashCode() { >> return volType == DRIVE_FIXED ? >> root.toLowerCase().hashCode() : root.hashCode(); >> } > >> If we want to constrain the case insensitive comparison to fixed drives, then this might work: > > Yes, although it would be good to extend this to the DRIVE_REMOTE and other cases too, but that requires being confident that it is correct (and I'm not sure we have enough information to say this yet). > > An initial change for the DRIVE_FIXED case is okay. It will mean using Locale.ROOT or Locale.US. Also the hash code will need to be cached. What do you intend by the `Locale.X` comment? ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From ccleary at openjdk.java.net Wed Apr 7 18:45:20 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Wed, 7 Apr 2021 18:45:20 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 18:12:20 GMT, Naoto Sato wrote: >> This makes changes to the copyright headers of of the generated source files `UnsupportedCharsetException.java` and `IllegalCharsetNameException.java`. These can be found under a jdk build in `support/gensrc` and the headers update after running `make jdk-image`. Perhaps the build is out of date? These changes are harder to track as the source is not checked in. > > That's right. Sorry for the false alarm. No worries, better safe than sorry! ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From ccleary at openjdk.java.net Wed Apr 7 18:57:28 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Wed, 7 Apr 2021 18:57:28 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio [v3] In-Reply-To: References: Message-ID: > This fix addresses the following warnings which were generated by building JDK API documentation with the `-Xdoclint:all` option enabled: > > ./build/linux-x64/support/gensrc/java.base/java/nio/charset/IllegalCharsetNameException.java:47: warning: no comment > private String charsetName; > ^ > ./open/src/java.base/share/classes/java/nio/charset/MalformedInputException.java:44: warning: no comment > private int inputLength; > ^ > ./open/src/java.base/share/classes/java/nio/charset/UnmappableCharacterException.java:44: warning: no comment > private int inputLength; > ^ > ./build/linux-x64/support/gensrc/java.base/java/nio/charset/UnsupportedCharsetException.java:47: warning: no comment > private String charsetName; > ^ > ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @param for s > private void readObject(ObjectInputStream s) > ^ > ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @throws for java.lang.ClassNotFoundException > private void readObject(ObjectInputStream s) > ^ > ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:43: warning: no comment > private final String file; > ^ > ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:44: warning: no comment > private final String other; > ^ > ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:43: warning: no comment > private int index; > ^ > ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:42: warning: no comment > private String input; > ^ > ./open/src/java.base/share/classes/java/nio/file/attribute/UserPrincipalNotFoundException.java:43: warning: no comment > private final String name; > ^ > Changes to [`genExceptions.sh`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) and the two [`exceptions`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) templates are to address the warnings concerned with `UnsupportedCharsetException.java` and `IllegalCharsetNameException.java` which are generated when `make jdk-image` is run. A CSR will be filed in due course with respect to these changes. Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: 8264779: Shortened parameter comment ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3376/files - new: https://git.openjdk.java.net/jdk/pull/3376/files/0b879f15..9f36eec4 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3376&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3376&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3376.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3376/head:pull/3376 PR: https://git.openjdk.java.net/jdk/pull/3376 From naoto at openjdk.java.net Wed Apr 7 18:57:29 2021 From: naoto at openjdk.java.net (Naoto Sato) Date: Wed, 7 Apr 2021 18:57:29 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio [v3] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 18:54:26 GMT, Conor Cleary wrote: >> This fix addresses the following warnings which were generated by building JDK API documentation with the `-Xdoclint:all` option enabled: >> >> ./build/linux-x64/support/gensrc/java.base/java/nio/charset/IllegalCharsetNameException.java:47: warning: no comment >> private String charsetName; >> ^ >> ./open/src/java.base/share/classes/java/nio/charset/MalformedInputException.java:44: warning: no comment >> private int inputLength; >> ^ >> ./open/src/java.base/share/classes/java/nio/charset/UnmappableCharacterException.java:44: warning: no comment >> private int inputLength; >> ^ >> ./build/linux-x64/support/gensrc/java.base/java/nio/charset/UnsupportedCharsetException.java:47: warning: no comment >> private String charsetName; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @param for s >> private void readObject(ObjectInputStream s) >> ^ >> ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @throws for java.lang.ClassNotFoundException >> private void readObject(ObjectInputStream s) >> ^ >> ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:43: warning: no comment >> private final String file; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:44: warning: no comment >> private final String other; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:43: warning: no comment >> private int index; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:42: warning: no comment >> private String input; >> ^ >> ./open/src/java.base/share/classes/java/nio/file/attribute/UserPrincipalNotFoundException.java:43: warning: no comment >> private final String name; >> ^ >> Changes to [`genExceptions.sh`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) and the two [`exceptions`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) templates are to address the warnings concerned with `UnsupportedCharsetException.java` and `IllegalCharsetNameException.java` which are generated when `make jdk-image` is run. A CSR will be filed in due course with respect to these changes. > > Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: > > 8264779: Shortened parameter comment Marked as reviewed by naoto (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From ccleary at openjdk.java.net Wed Apr 7 18:57:29 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Wed, 7 Apr 2021 18:57:29 GMT Subject: RFR: 8264779: Fix doclint warnings in java/nio [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 18:11:43 GMT, Naoto Sato wrote: >> or just "The length of the input" so that it is consistent with the description of the getInputLength method. > > That sounds good. Changed to "The length of the input." in the [most recent commit](https://github.com/openjdk/jdk/pull/3376/commits/9f36eec44dec7cd8f75f787c7d386e26d55e6826). ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From bpb at openjdk.java.net Wed Apr 7 20:39:43 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 7 Apr 2021 20:39:43 GMT Subject: RFR: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 18:16:34 GMT, Brian Burkhalter wrote: >> I think this looks good, I'm just wondering to myself if it is possible to get ENOTSUP from a local device. > > I am going to try to test this. I was unable to get `ENOTSUP` from local FAT32 or NTFS devices. However from the review discussion [1] of [2] it looks like `ENOTSUP` has been observed and is handled in other packages [3]. Is there any harm to leaving it in? [1] https://bugs.openjdk.java.net/browse/JDK-8080589 [2] http://mail.openjdk.java.net/pipermail/nio-dev/2015-May/003160.html [3] https://community.oracle.com/tech/developers/discussion/453477/patch-when-fnctl-fd-f-fullsync-0-returns-enotsup ------------- PR: https://git.openjdk.java.net/jdk/pull/3366 From bpb at openjdk.java.net Wed Apr 7 21:54:24 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 7 Apr 2021 21:54:24 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 11:29:29 GMT, Alan Bateman wrote: >> If we want to constrain the case insensitive comparison to fixed drives, then this might work: >> @Override >> public boolean equals(Object ob) { >> if (ob == this) >> return true; >> if (!(ob instanceof WindowsFileStore)) >> return false; >> WindowsFileStore other = (WindowsFileStore)ob; >> if (root.equals(other.root)) >> return true; >> if (volType == DRIVE_FIXED && other.volumeType() == DRIVE_FIXED) >> return root.equalsIgnoreCase(other.root); >> return false; >> } >> >> @Override >> public int hashCode() { >> return volType == DRIVE_FIXED ? >> root.toLowerCase().hashCode() : root.hashCode(); >> } > >> If we want to constrain the case insensitive comparison to fixed drives, then this might work: > > Yes, although it would be good to extend this to the DRIVE_REMOTE and other cases too, but that requires being confident that it is correct (and I'm not sure we have enough information to say this yet). > > An initial change for the DRIVE_FIXED case is okay. It will mean using Locale.ROOT or Locale.US. Also the hash code will need to be cached. (Commit message for 9260dfd intended `DRIVE_FIXED`, not `DRIVE_LOCAL`: inadvertently conflated with `MNT_LOCAL` on `macOS`.) ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Wed Apr 7 21:54:24 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 7 Apr 2021 21:54:24 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v2] In-Reply-To: References: Message-ID: > Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8264400: Constrain to DRIVE_LOCAL case ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3279/files - new: https://git.openjdk.java.net/jdk/pull/3279/files/30587898..9260dfde Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3279&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3279&range=00-01 Stats: 28 lines in 1 file changed: 16 ins; 5 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/3279.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3279/head:pull/3279 PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Wed Apr 7 21:54:24 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 7 Apr 2021 21:54:24 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 21:50:30 GMT, Brian Burkhalter wrote: >>> If we want to constrain the case insensitive comparison to fixed drives, then this might work: >> >> Yes, although it would be good to extend this to the DRIVE_REMOTE and other cases too, but that requires being confident that it is correct (and I'm not sure we have enough information to say this yet). >> >> An initial change for the DRIVE_FIXED case is okay. It will mean using Locale.ROOT or Locale.US. Also the hash code will need to be cached. > > (Commit message for 9260dfd intended `DRIVE_FIXED`, not `DRIVE_LOCAL`: inadvertently conflated with `MNT_LOCAL` on `macOS`.) (Commit message for 9260dfd intended `DRIVE_FIXED`, not `DRIVE_LOCAL`: inadvertently conflated with `MNT_LOCAL` on `macOS`.) ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From alanb at openjdk.java.net Thu Apr 8 07:33:15 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 8 Apr 2021 07:33:15 GMT Subject: RFR: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 20:35:48 GMT, Brian Burkhalter wrote: >> I am going to try to test this. > > I was unable to get `ENOTSUP` from local FAT32 or NTFS devices. However from the review discussion [1] of [2] it looks like `ENOTSUP` has been observed and is handled in other packages [3]. Is there any harm to leaving it in? > > [1] https://bugs.openjdk.java.net/browse/JDK-8080589 > [2] http://mail.openjdk.java.net/pipermail/nio-dev/2015-May/003160.html > [3] https://community.oracle.com/tech/developers/discussion/453477/patch-when-fnctl-fd-f-fullsync-0-returns-enotsup The force method needs fcntl(F_FULLSYNC) when the file system is local so I think it would be safer to drop the special handle for ENOTSUP. This will mean fallback to fsync when the file is not local. ------------- PR: https://git.openjdk.java.net/jdk/pull/3366 From github.com+10835776+stsypanov at openjdk.java.net Thu Apr 8 08:38:38 2021 From: github.com+10835776+stsypanov at openjdk.java.net (=?UTF-8?B?0KHQtdGA0LPQtdC5?= =?UTF-8?B?IA==?= =?UTF-8?B?0KbRi9C/0LDQvdC+0LI=?=) Date: Thu, 8 Apr 2021 08:38:38 GMT Subject: RFR: 8264896: Remove redundant '& 0xFF' from int-to-byte cast Message-ID: When we do byte b1 = (byte) (value & 0xFF); we keep from int only 1 lower byte and exactly the same can be achieved with plain cast. See the test below: public class Main { public static void main(String[] args) throws Exception { IntStream.range(Integer.MIN_VALUE, Integer.MAX_VALUE).forEach(value -> { byte b1 = (byte) (value & 0xFF); byte b2 = (byte) value; if (b1 != b2) { throw new RuntimeException("" + value); } }); } Also tier1 and tier2 are both OK. ------------- Commit messages: - Merge branch 'master' into bitwise - Remove redundant '& 0xFF' from int-to-byte cast Changes: https://git.openjdk.java.net/jdk/pull/2826/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2826&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264896 Stats: 50 lines in 14 files changed: 0 ins; 0 del; 50 mod Patch: https://git.openjdk.java.net/jdk/pull/2826.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2826/head:pull/2826 PR: https://git.openjdk.java.net/jdk/pull/2826 From alanb at openjdk.java.net Thu Apr 8 08:46:28 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 8 Apr 2021 08:46:28 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 21:54:24 GMT, Brian Burkhalter wrote: >> Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8264400: Constrain to DRIVE_LOCAL case src/java.base/windows/classes/sun/nio/fs/WindowsFileStore.java line 55: > 53: private final String displayName; // returned by toString > 54: > 55: private boolean hasHashCode = false; // as hashCode can be any int This is buggy. It would be simpler to drop hasHashCode and just race when hash is 0. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From github.com+10835776+stsypanov at openjdk.java.net Thu Apr 8 08:50:09 2021 From: github.com+10835776+stsypanov at openjdk.java.net (=?UTF-8?B?0KHQtdGA0LPQtdC5?= =?UTF-8?B?IA==?= =?UTF-8?B?0KbRi9C/0LDQvdC+0LI=?=) Date: Thu, 8 Apr 2021 08:50:09 GMT Subject: RFR: 8263561: Re-examine uses of LinkedList In-Reply-To: References: <-lwwnneyhUB3qMqORpmnCC2vhWNTuURohJWvKb7xEtY=.a1e980bd-fc7a-49c8-89ed-4b8f294e83f6@github.com> <6ZglAu_W4knnRCE5IzZiYkFgZn-mbf72HM4aF5ehnFU=.0b120039-1967-4d4c-939e-b1d52a884a51@github.com> <7nIzoaMqTkfI8ia87qvQv_zzkcHsewGEkSZo-TYTMn4=.dad9bbf0-3513-43c7-826e-cb918beb39eb@github.com> <8nMvQUI3qPfR-VTlVG3i0Pfm7w_xZ5aDUAotA4a3Yvs=.5ac6563e-d7fb-4b8f-8b0d-c0d7701860ac@github.com> Message-ID: On Mon, 15 Mar 2021 06:56:00 GMT, Alan Bateman wrote: >> Nothing outside of the JDK should be hacking into this private field of a non-exposed class, this should not be a concern here. > >> @AlanBateman so is it ok to keep `ArrayLists`? > > One thing to look out for is usages of 2-arg add method that inserts at a specific position. This shouldn't be a concern in URLClassPath.closeLoaders (assuming this is this usage where the question arises). @AlanBateman looks like `List.add(o, i)` is not used at all in scope of these changes. ------------- PR: https://git.openjdk.java.net/jdk/pull/2744 From github.com+1204330+overheadhunter at openjdk.java.net Thu Apr 8 08:57:17 2021 From: github.com+1204330+overheadhunter at openjdk.java.net (Sebastian Stenzel) Date: Thu, 8 Apr 2021 08:57:17 GMT Subject: RFR: 8264896: Remove redundant '& 0xFF' from int-to-byte cast In-Reply-To: References: Message-ID: On Thu, 4 Mar 2021 12:16:29 GMT, ?????? ??????? wrote: > When we do > byte b1 = (byte) (value & 0xFF); > we keep from int only 1 lower byte and exactly the same can be achieved with plain cast. See the test below: > public class Main { > public static void main(String[] args) throws Exception { > IntStream.range(Integer.MIN_VALUE, Integer.MAX_VALUE).forEach(value -> { > byte b1 = (byte) (value & 0xFF); > byte b2 = (byte) value; > if (b1 != b2) { > throw new RuntimeException("" + value); > } > }); > } > > Also tier1 and tier2 are both OK. I don't think these masks have been added because they are _required_ but rather because they explicitly show the intention: You can immediately see that losing the MSBs of an int during the cast is _not_ just an error. ------------- PR: https://git.openjdk.java.net/jdk/pull/2826 From dfuchs at openjdk.java.net Thu Apr 8 09:11:18 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Thu, 8 Apr 2021 09:11:18 GMT Subject: RFR: 8264896: Remove redundant '& 0xFF' from int-to-byte cast In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 08:54:35 GMT, Sebastian Stenzel wrote: >> When we do >> byte b1 = (byte) (value & 0xFF); >> we keep from int only 1 lower byte and exactly the same can be achieved with plain cast. See the test below: >> public class Main { >> public static void main(String[] args) throws Exception { >> IntStream.range(Integer.MIN_VALUE, Integer.MAX_VALUE).forEach(value -> { >> byte b1 = (byte) (value & 0xFF); >> byte b2 = (byte) value; >> if (b1 != b2) { >> throw new RuntimeException("" + value); >> } >> }); >> } >> >> Also tier1 and tier2 are both OK. > > I don't think these masks have been added because they are _required_ but rather because they explicitly show the intention: You can immediately see that losing the MSBs of an int during the cast is _not_ just an error. I agree with Sebastian. I believe the original code was clearer. ------------- PR: https://git.openjdk.java.net/jdk/pull/2826 From adinn at redhat.com Thu Apr 8 09:15:29 2021 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 8 Apr 2021 10:15:29 +0100 Subject: RFR: 8264896: Remove redundant '& 0xFF' from int-to-byte cast In-Reply-To: References: Message-ID: On 08/04/2021 10:11, Daniel Fuchs wrote: > On Thu, 8 Apr 2021 08:54:35 GMT, Sebastian Stenzel wrote: > >>> When we do >>> byte b1 = (byte) (value & 0xFF); >>> we keep from int only 1 lower byte and exactly the same can be achieved with plain cast. See the test below: >>> public class Main { >>> public static void main(String[] args) throws Exception { >>> IntStream.range(Integer.MIN_VALUE, Integer.MAX_VALUE).forEach(value -> { >>> byte b1 = (byte) (value & 0xFF); >>> byte b2 = (byte) value; >>> if (b1 != b2) { >>> throw new RuntimeException("" + value); >>> } >>> }); >>> } >>> >>> Also tier1 and tier2 are both OK. >> >> I don't think these masks have been added because they are _required_ but rather because they explicitly show the intention: You can immediately see that losing the MSBs of an int during the cast is _not_ just an error. > > I agree with Sebastian. I believe the original code was clearer. The same applies for the (value >>> 0) expressions that this patch also removes. The reason for keeping them is to emphasize the continuity with the (value >>> 24), (value >>> 16), etc cases that precede them. Since none of this will make any difference to performance I think it is better to keep this code as is. regards, Andrew Dinn ----------- Red Hat Distinguished Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill From ccleary at openjdk.java.net Thu Apr 8 14:32:46 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Thu, 8 Apr 2021 14:32:46 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v5] In-Reply-To: References: Message-ID: > ### Description > SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. > > - Test is run in AgentVM mode. > - Receivers bind to wildcard/localhost which has been a source of instability. > - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. > - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. > > ### Fixes > To Address these issues, the following was done respective to the order of the Issues above: > - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. > - Receivers now bind to loopback address instead of localhost. > - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). > - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: 8264200: Changed port to dstPort in Writers ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3354/files - new: https://git.openjdk.java.net/jdk/pull/3354/files/a2e25979..59424455 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3354&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3354&range=03-04 Stats: 8 lines in 1 file changed: 0 ins; 0 del; 8 mod Patch: https://git.openjdk.java.net/jdk/pull/3354.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3354/head:pull/3354 PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Thu Apr 8 14:38:18 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Thu, 8 Apr 2021 14:38:18 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v4] In-Reply-To: References: <_ece4mFrjesKXz4bMfwozRp4l7jbrYfxUxboLc1WMik=.418bdb44-9c39-429d-99a0-6ca66984547e@github.com> <7LpyXZoyHLqRYOZU25O6HtwiUT4I_NJ_ZFILFvX6Ddk=.c852eb95-d08e-43e1-b86d-91b1c4ce5b57@github.com> Message-ID: On Wed, 7 Apr 2021 11:50:45 GMT, Mark Sheppard wrote: > Both ClassicWriter and NioWriter constructors take a port, which is the destination port of the recipient rather than a port to which they'll bind -- maybe a rename of the parameter to dstPort ? [Most recent commit](https://github.com/openjdk/jdk/pull/3354/commits/594244555c95f799c1bf83841c2b042749b38b67) makes this change as suggested by @msheppar ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From msheppar at openjdk.java.net Thu Apr 8 15:07:23 2021 From: msheppar at openjdk.java.net (Mark Sheppard) Date: Thu, 8 Apr 2021 15:07:23 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v5] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 14:32:46 GMT, Conor Cleary wrote: >> ### Description >> SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. >> >> - Test is run in AgentVM mode. >> - Receivers bind to wildcard/localhost which has been a source of instability. >> - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. >> - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. >> >> ### Fixes >> To Address these issues, the following was done respective to the order of the Issues above: >> - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. >> - Receivers now bind to loopback address instead of localhost. >> - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). >> - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). > > Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: > > 8264200: Changed port to dstPort in Writers ?? ------------- Marked as reviewed by msheppar (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3354 From dfuchs at openjdk.java.net Thu Apr 8 15:22:17 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Thu, 8 Apr 2021 15:22:17 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v5] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 14:32:46 GMT, Conor Cleary wrote: >> ### Description >> SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. >> >> - Test is run in AgentVM mode. >> - Receivers bind to wildcard/localhost which has been a source of instability. >> - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. >> - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. >> >> ### Fixes >> To Address these issues, the following was done respective to the order of the Issues above: >> - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. >> - Receivers now bind to loopback address instead of localhost. >> - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). >> - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). > > Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: > > 8264200: Changed port to dstPort in Writers Marked as reviewed by dfuchs (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From bpb at openjdk.java.net Thu Apr 8 16:03:32 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 8 Apr 2021 16:03:32 GMT Subject: RFR: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) [v4] In-Reply-To: References: Message-ID: > Apparently `fcntl(fd, F_FULLFSYNC)` can fail with `ENOTTY` in addition to `ENOTSUP` although it is not so documented. The `ioctl()` case is documented as > [ENOTTY] fildes is not associated with a character special > device. > [ENOTTY] The specified request does not apply to the kind of > object that the descriptor fildes references. > > This request proposes to catch `ENOTTY` as well and fall back to `fsync(fd)`. The change was verified manually using a WebDAV server and no failures in CI test tiers 1-3 were observed. An alternative would be to fall back to `fsync(fd)` for **all** errors. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8232861: Remove ENOTSUP fallback for fcntl(fd, F_FULLFSYNC) failure ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3366/files - new: https://git.openjdk.java.net/jdk/pull/3366/files/0c01b23d..70d518a3 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3366&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3366&range=02-03 Stats: 19 lines in 1 file changed: 3 ins; 10 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/3366.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3366/head:pull/3366 PR: https://git.openjdk.java.net/jdk/pull/3366 From alanb at openjdk.java.net Thu Apr 8 16:03:33 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 8 Apr 2021 16:03:33 GMT Subject: RFR: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) [v4] In-Reply-To: References: Message-ID: <6zYFc4yaD2aHwKJT468zb9BPa83j7J6dEK_pyrpYe64=.a0ac855d-c9a6-44c0-927e-04cac87ee478@github.com> On Thu, 8 Apr 2021 16:00:44 GMT, Brian Burkhalter wrote: >> Apparently `fcntl(fd, F_FULLFSYNC)` can fail with `ENOTTY` in addition to `ENOTSUP` although it is not so documented. The `ioctl()` case is documented as >> [ENOTTY] fildes is not associated with a character special >> device. >> [ENOTTY] The specified request does not apply to the kind of >> object that the descriptor fildes references. >> >> This request proposes to catch `ENOTTY` as well and fall back to `fsync(fd)`. The change was verified manually using a WebDAV server and no failures in CI test tiers 1-3 were observed. An alternative would be to fall back to `fsync(fd)` for **all** errors. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8232861: Remove ENOTSUP fallback for fcntl(fd, F_FULLFSYNC) failure Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3366 From bpb at openjdk.java.net Thu Apr 8 16:15:23 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 8 Apr 2021 16:15:23 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v3] In-Reply-To: References: Message-ID: <6_SjO4cvxyRZ-pRTuVj0pAHKpp7LhpuZYZl10177UmI=.14b9b01c-1c2f-4e91-9d15-172f4197f9fa@github.com> > Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8264400: Remove hasHashCode and race on hashCode == 0 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3279/files - new: https://git.openjdk.java.net/jdk/pull/3279/files/9260dfde..c9e34e48 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3279&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3279&range=01-02 Stats: 4 lines in 1 file changed: 0 ins; 2 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3279.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3279/head:pull/3279 PR: https://git.openjdk.java.net/jdk/pull/3279 From alanb at openjdk.java.net Thu Apr 8 16:24:15 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 8 Apr 2021 16:24:15 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v3] In-Reply-To: <6_SjO4cvxyRZ-pRTuVj0pAHKpp7LhpuZYZl10177UmI=.14b9b01c-1c2f-4e91-9d15-172f4197f9fa@github.com> References: <6_SjO4cvxyRZ-pRTuVj0pAHKpp7LhpuZYZl10177UmI=.14b9b01c-1c2f-4e91-9d15-172f4197f9fa@github.com> Message-ID: On Thu, 8 Apr 2021 16:15:23 GMT, Brian Burkhalter wrote: >> Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8264400: Remove hasHashCode and race on hashCode == 0 src/java.base/windows/classes/sun/nio/fs/WindowsFileStore.java line 234: > 232: return supportsFileAttributeView(FileOwnerAttributeView.class); > 233: if (name.equals("user")) > 234: return supportsFileAttributeView(UserDefinedFileAttributeView.class); return false; Can you revert this change, it's not part of the fix and it makes this method harder to maintain. src/java.base/windows/classes/sun/nio/fs/WindowsFileStore.java line 241: > 239: if (ob == this) > 240: return true; > 241: if (!(ob instanceof WindowsFileStore)) We can change this to if (ob instanced WindowsFileStore other) { ... } and it will avoid the cast. src/java.base/windows/classes/sun/nio/fs/WindowsFileStore.java line 254: > 252: public int hashCode() { > 253: if (hashCode == 0) { // Don't care about race > 254: hashCode = volType == DRIVE_FIXED ? minor nit but would be a bit easier to read if changed to hashCode = (volType == DRIVE_FIXED) ? ... test/jdk/java/nio/file/FileStore/Basic.java line 89: > 87: FileSystem fs = FileSystems.getDefault(); > 88: FileStore upper = Files.getFileStore(fs.getPath("C:\\")); > 89: FileStore lower = Files.getFileStore(fs.getPath("c:\\")); Path.of("c:\") can be used if you want. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Thu Apr 8 16:24:15 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 8 Apr 2021 16:24:15 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v3] In-Reply-To: References: <6_SjO4cvxyRZ-pRTuVj0pAHKpp7LhpuZYZl10177UmI=.14b9b01c-1c2f-4e91-9d15-172f4197f9fa@github.com> Message-ID: On Thu, 8 Apr 2021 16:18:19 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8264400: Remove hasHashCode and race on hashCode == 0 > > src/java.base/windows/classes/sun/nio/fs/WindowsFileStore.java line 234: > >> 232: return supportsFileAttributeView(FileOwnerAttributeView.class); >> 233: if (name.equals("user")) >> 234: return supportsFileAttributeView(UserDefinedFileAttributeView.class); return false; > > Can you revert this change, it's not part of the fix and it makes this method harder to maintain. I did revert it before but I guess it crept back in ... sigh. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Thu Apr 8 16:37:39 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 8 Apr 2021 16:37:39 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v4] In-Reply-To: References: Message-ID: > Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8264400: Clean up equals() and hashCode() ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3279/files - new: https://git.openjdk.java.net/jdk/pull/3279/files/c9e34e48..7849c7ec Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3279&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3279&range=02-03 Stats: 10 lines in 1 file changed: 1 ins; 1 del; 8 mod Patch: https://git.openjdk.java.net/jdk/pull/3279.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3279/head:pull/3279 PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Thu Apr 8 16:37:40 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 8 Apr 2021 16:37:40 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v3] In-Reply-To: References: <6_SjO4cvxyRZ-pRTuVj0pAHKpp7LhpuZYZl10177UmI=.14b9b01c-1c2f-4e91-9d15-172f4197f9fa@github.com> Message-ID: <7JWYF9BvAB3ngt_RnyMpsRPYLZE9LgLcb_fL2mT2UlU=.027518a5-1144-4190-bc9c-fe9560de705b@github.com> On Thu, 8 Apr 2021 16:20:42 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8264400: Remove hasHashCode and race on hashCode == 0 > > test/jdk/java/nio/file/FileStore/Basic.java line 89: > >> 87: FileSystem fs = FileSystems.getDefault(); >> 88: FileStore upper = Files.getFileStore(fs.getPath("C:\\")); >> 89: FileStore lower = Files.getFileStore(fs.getPath("c:\\")); > > Path.of("c:\") can be used if you want. Will simplify to that in final commit. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From alanb at openjdk.java.net Thu Apr 8 16:56:08 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 8 Apr 2021 16:56:08 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 16:37:39 GMT, Brian Burkhalter wrote: >> Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8264400: Clean up equals() and hashCode() src/java.base/windows/classes/sun/nio/fs/WindowsFileStore.java line 56: > 54: > 55: private int hashCode = 0; > 56: Good, I think this is safe and equals/hashCode are consistent. I suspect we can extend this to other (maybe all) volume type but it needs a bit more exploration. Minor nit is that you can't need to initialise hashCode to 0. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Thu Apr 8 17:01:10 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 8 Apr 2021 17:01:10 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 16:53:30 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8264400: Clean up equals() and hashCode() > > src/java.base/windows/classes/sun/nio/fs/WindowsFileStore.java line 56: > >> 54: >> 55: private int hashCode = 0; >> 56: > > Good, I think this is safe and equals/hashCode are consistent. I suspect we can extend this to other (maybe all) volume type but it needs a bit more exploration. Minor nit is that you can't need to initialise hashCode to 0. I think this issue was specific to the `DRIVE_FIXED` case so further enhancement can be done under another issue once the information becomes knowd. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From alanb at openjdk.java.net Thu Apr 8 18:17:07 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 8 Apr 2021 18:17:07 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v4] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 16:58:12 GMT, Brian Burkhalter wrote: > I think this issue was specific to the `DRIVE_FIXED` case so further enhancement can be done under another issue once the information becomes knowd. That's okay with me. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Thu Apr 8 19:56:24 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 8 Apr 2021 19:56:24 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v5] In-Reply-To: References: Message-ID: > Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8264400: Don't initialize hashCode; use Path.of() in test ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3279/files - new: https://git.openjdk.java.net/jdk/pull/3279/files/7849c7ec..e1d7b21a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3279&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3279&range=03-04 Stats: 4 lines in 2 files changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3279.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3279/head:pull/3279 PR: https://git.openjdk.java.net/jdk/pull/3279 From alanb at openjdk.java.net Fri Apr 9 06:21:27 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 9 Apr 2021 06:21:27 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v5] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 19:56:24 GMT, Brian Burkhalter wrote: >> Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8264400: Don't initialize hashCode; use Path.of() in test Marked as reviewed by alanb (Reviewer). test/jdk/java/nio/file/FileStore/Basic.java line 87: > 85: * Test: FileStore.equals() should not be case sensitive > 86: */ > 87: FileSystem fs = FileSystems.getDefault(); I assume this line is no longer needed. ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From ccleary at openjdk.java.net Fri Apr 9 08:54:18 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Fri, 9 Apr 2021 08:54:18 GMT Subject: Integrated: 8264779: Fix doclint warnings in java/nio In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 13:22:44 GMT, Conor Cleary wrote: > This fix addresses the following warnings which were generated by building JDK API documentation with the `-Xdoclint:all` option enabled: > > ./build/linux-x64/support/gensrc/java.base/java/nio/charset/IllegalCharsetNameException.java:47: warning: no comment > private String charsetName; > ^ > ./open/src/java.base/share/classes/java/nio/charset/MalformedInputException.java:44: warning: no comment > private int inputLength; > ^ > ./open/src/java.base/share/classes/java/nio/charset/UnmappableCharacterException.java:44: warning: no comment > private int inputLength; > ^ > ./build/linux-x64/support/gensrc/java.base/java/nio/charset/UnsupportedCharsetException.java:47: warning: no comment > private String charsetName; > ^ > ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @param for s > private void readObject(ObjectInputStream s) > ^ > ./open/src/java.base/share/classes/java/nio/file/DirectoryIteratorException.java:81: warning: no @throws for java.lang.ClassNotFoundException > private void readObject(ObjectInputStream s) > ^ > ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:43: warning: no comment > private final String file; > ^ > ./open/src/java.base/share/classes/java/nio/file/FileSystemException.java:44: warning: no comment > private final String other; > ^ > ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:43: warning: no comment > private int index; > ^ > ./open/src/java.base/share/classes/java/nio/file/InvalidPathException.java:42: warning: no comment > private String input; > ^ > ./open/src/java.base/share/classes/java/nio/file/attribute/UserPrincipalNotFoundException.java:43: warning: no comment > private final String name; > ^ > Changes to [`genExceptions.sh`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) and the two [`exceptions`](https://github.com/openjdk/jdk/commit/b729d8ed7970737a8a2d4e8aa788df33789faea2) templates are to address the warnings concerned with `UnsupportedCharsetException.java` and `IllegalCharsetNameException.java` which are generated when `make jdk-image` is run. > > CSR: https://bugs.openjdk.java.net/browse/JDK-8264844 This pull request has now been integrated. Changeset: a45733f8 Author: Conor Cleary Committer: Chris Hegarty URL: https://git.openjdk.java.net/jdk/commit/a45733f8 Stats: 48 lines in 9 files changed: 38 ins; 0 del; 10 mod 8264779: Fix doclint warnings in java/nio Reviewed-by: chegar, iris, alanb, naoto ------------- PR: https://git.openjdk.java.net/jdk/pull/3376 From chegar at openjdk.java.net Fri Apr 9 09:30:49 2021 From: chegar at openjdk.java.net (Chris Hegarty) Date: Fri, 9 Apr 2021 09:30:49 GMT Subject: RFR: 8264827: Large mapped buffer/segment crash the VM when calling isLoaded Message-ID: Avoid overflow when calculating the number of pages for large mapped segment sizes. ------------- Commit messages: - Test update - Initial changes Changes: https://git.openjdk.java.net/jdk/pull/3378/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3378&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264827 Stats: 32 lines in 5 files changed: 19 ins; 0 del; 13 mod Patch: https://git.openjdk.java.net/jdk/pull/3378.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3378/head:pull/3378 PR: https://git.openjdk.java.net/jdk/pull/3378 From chegar at openjdk.java.net Fri Apr 9 10:19:21 2021 From: chegar at openjdk.java.net (Chris Hegarty) Date: Fri, 9 Apr 2021 10:19:21 GMT Subject: RFR: 8264827: Large mapped buffer/segment crash the VM when calling isLoaded In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 15:45:30 GMT, Chris Hegarty wrote: > Avoid overflow when calculating the number of pages for large mapped segment sizes. Note for reviewers on the test. A 3GB file is sufficient to demonstrate the issue in the old code, as follows (with a 4K page size, since the narrowing cast of size is the significant issue): jshell> int pageSize() { return 4 * 1024; } | created method pageSize() jshell> int pageCount(long size) { return (int)(size + (long)pageSize() - 1L) / pageSize(); } | created method pageCount(long) jshell> pageCount(3L * 1024L * 1024L * 1024L) $3 ==> -262143 ------------- PR: https://git.openjdk.java.net/jdk/pull/3378 From alanb at openjdk.java.net Fri Apr 9 11:55:14 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 9 Apr 2021 11:55:14 GMT Subject: RFR: 8264827: Large mapped buffer/segment crash the VM when calling isLoaded In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 15:45:30 GMT, Chris Hegarty wrote: > Avoid overflow when calculating the number of pages for large mapped segment sizes. Looks okay to me. ------------- Marked as reviewed by alanb (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3378 From bpb at openjdk.java.net Fri Apr 9 15:34:26 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 9 Apr 2021 15:34:26 GMT Subject: Integrated: 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 02:35:18 GMT, Brian Burkhalter wrote: > Apparently `fcntl(fd, F_FULLFSYNC)` can fail with `ENOTTY` in addition to `ENOTSUP` although it is not so documented. The `ioctl()` case is documented as > [ENOTTY] fildes is not associated with a character special > device. > [ENOTTY] The specified request does not apply to the kind of > object that the descriptor fildes references. > > This request proposes to catch `ENOTTY` as well and fall back to `fsync(fd)`. The change was verified manually using a WebDAV server and no failures in CI test tiers 1-3 were observed. An alternative would be to fall back to `fsync(fd)` for **all** errors. This pull request has now been integrated. Changeset: 6de0bb20 Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/6de0bb20 Stats: 14 lines in 1 file changed: 11 ins; 0 del; 3 mod 8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) Reviewed-by: alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/3366 From bpb at openjdk.java.net Fri Apr 9 15:34:44 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 9 Apr 2021 15:34:44 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v6] In-Reply-To: References: Message-ID: > Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8264400: Remove vestigial FileSystems.getDefault() ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3279/files - new: https://git.openjdk.java.net/jdk/pull/3279/files/e1d7b21a..a53d10f2 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3279&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3279&range=04-05 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3279.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3279/head:pull/3279 PR: https://git.openjdk.java.net/jdk/pull/3279 From bpb at openjdk.java.net Fri Apr 9 15:45:18 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 9 Apr 2021 15:45:18 GMT Subject: Integrated: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 00:22:20 GMT, Brian Burkhalter wrote: > Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. This pull request has now been integrated. Changeset: cc54de76 Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/cc54de76 Stats: 39 lines in 2 files changed: 27 ins; 1 del; 11 mod 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed Reviewed-by: alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From mdoerr at openjdk.java.net Fri Apr 9 22:01:02 2021 From: mdoerr at openjdk.java.net (Martin Doerr) Date: Fri, 9 Apr 2021 22:01:02 GMT Subject: RFR: 8265018: [AIX] FileDispatcherImpl.c:31:10: fatal error: 'sys/mount.h' file not found Message-ID: #3366 introduced new includes which are not available on AIX, but they are only used on MacOS. ------------- Commit messages: - 8265018: [AIX] FileDispatcherImpl.c:31:10: fatal error: 'sys/mount.h' file not found Changes: https://git.openjdk.java.net/jdk/pull/3424/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3424&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265018 Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3424.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3424/head:pull/3424 PR: https://git.openjdk.java.net/jdk/pull/3424 From github.com+741251+turbanoff at openjdk.java.net Fri Apr 9 22:27:27 2021 From: github.com+741251+turbanoff at openjdk.java.net (Andrey Turbanov) Date: Fri, 9 Apr 2021 22:27:27 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v6] In-Reply-To: References: Message-ID: On Fri, 9 Apr 2021 15:34:44 GMT, Brian Burkhalter wrote: >> Please consider this change to make `sun.nio.fs.WindowsFileStore.equals()` return `true` if the root strings of the two objects are equal under case insensitive comparison, and the two `WindowsFileStore`s have the same volume information. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8264400: Remove vestigial FileSystems.getDefault() src/java.base/windows/classes/sun/nio/fs/WindowsFileStore.java line 257: > 255: root.toLowerCase(Locale.ROOT).hashCode() : root.hashCode(); > 256: } > 257: return hashCode; As I know, to make data race benign, field should be read only once. But there is second read which can still return `0` ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From clanger at openjdk.java.net Fri Apr 9 22:52:32 2021 From: clanger at openjdk.java.net (Christoph Langer) Date: Fri, 9 Apr 2021 22:52:32 GMT Subject: RFR: 8265018: [AIX] FileDispatcherImpl.c:31:10: fatal error: 'sys/mount.h' file not found In-Reply-To: References: Message-ID: <-gso4HrFo2t9SvCOBoTOULX4hUy3oxVhhYQShY1L-RQ=.54be0435-375d-40e2-9ecd-e5ca7655225c@github.com> On Fri, 9 Apr 2021 21:39:53 GMT, Martin Doerr wrote: > #3366 introduced new includes which are not available on AIX, but they are only used on MacOS. Marked as reviewed by clanger (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3424 From alanb at openjdk.java.net Sat Apr 10 06:37:24 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Sat, 10 Apr 2021 06:37:24 GMT Subject: RFR: 8265018: [AIX] FileDispatcherImpl.c:31:10: fatal error: 'sys/mount.h' file not found In-Reply-To: References: Message-ID: On Fri, 9 Apr 2021 21:39:53 GMT, Martin Doerr wrote: > #3366 introduced new includes which are not available on AIX, but they are only used on MacOS. src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c line 34: > 32: #include > 33: #include > 34: #endif The macOS specific includes are #ifdef _ALLBSD_SOURCE in this file so we can move them there (sorry, there is inconsistency in this source files as to whether _ALLBSD_SOURCE or MACOSX is used, we need to clean this up sometimes). ------------- PR: https://git.openjdk.java.net/jdk/pull/3424 From alanb at openjdk.java.net Sat Apr 10 06:46:24 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Sat, 10 Apr 2021 06:46:24 GMT Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v6] In-Reply-To: References: Message-ID: <1yCLzVQjMPTLhOnpsM-3zzpcQ8z362iTJ5utHziImhg=.af52968d-8c00-4374-b7dd-0d5ec0c5c2a6@github.com> On Fri, 9 Apr 2021 22:24:03 GMT, Andrey Turbanov wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8264400: Remove vestigial FileSystems.getDefault() > > src/java.base/windows/classes/sun/nio/fs/WindowsFileStore.java line 257: > >> 255: root.toLowerCase(Locale.ROOT).hashCode() : root.hashCode(); >> 256: } >> 257: return hashCode; > > As I know, to make data race benign, field should be read only once. But there is second read which can still return `0` You can are right, missed in the final review (as this one went through many iterations). ------------- PR: https://git.openjdk.java.net/jdk/pull/3279 From alanb at openjdk.java.net Sun Apr 11 13:34:57 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Sun, 11 Apr 2021 13:34:57 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v5] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 14:32:46 GMT, Conor Cleary wrote: >> ### Description >> SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. >> >> - Test is run in AgentVM mode. >> - Receivers bind to wildcard/localhost which has been a source of instability. >> - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. >> - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. >> >> ### Fixes >> To Address these issues, the following was done respective to the order of the Issues above: >> - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. >> - Receivers now bind to loopback address instead of localhost. >> - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). >> - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). > > Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: > > 8264200: Changed port to dstPort in Writers test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 26: > 24: /* @test > 25: * @summary Test DatagramChannel's send and receive methods > 26: * @author Mike McCloskey This change is essentially replacing this JDK 1.4 era test so probably should remove the @author tag. ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From shade at openjdk.java.net Mon Apr 12 06:24:30 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 12 Apr 2021 06:24:30 GMT Subject: RFR: JDK-8262957 (fs) Fail fast in UnixFileStore.isExtendedAttributesEnabled In-Reply-To: <-FT7LavNSb-NViqSfsISQvOUoic51dQsxuaQPBoirCI=.090bd820-2ef8-49d6-973e-8fba3278cab6@github.com> References: <-FT7LavNSb-NViqSfsISQvOUoic51dQsxuaQPBoirCI=.090bd820-2ef8-49d6-973e-8fba3278cab6@github.com> Message-ID: On Wed, 7 Apr 2021 16:51:23 GMT, Alan Bateman wrote: >> Added new capability flag `UnixNativeDispatcher.SUPPORTS_XATTR`, which gets set `#ifdef _SYS_XATTR_H_`. >> >> Note that `_SYS_XATTR_H_` is defined in `xattr.h` in both [macOS/Darwin](https://github.com/apple/darwin-xnu/blob/main/bsd/sys/xattr.h), [Linux and GNU-based BSD systems using glibc](https://github.com/bminor/glibc/blob/master/misc/sys/xattr.h). It might not be defined for other operating systems that still support xattr. So if OpenJDK eventually adds support for further platforms, this might need to be adjusted as well. >> >> If xattr capabilities are missing, `UnixFileStore.isExtendedAttributesEnabled` will return false immediately, avoiding any I/O. >> >> As a side effect of this change, I redefined some other (private) capabilities. Strictly speaking, this change is not required but keeps the code consistent, as [previously discussed in the mailing list](https://mail.openjdk.java.net/pipermail/nio-dev/2021-February/008133.html). > > Marked as reviewed by alanb (Reviewer). Alan, are you good with integrating this? ------------- PR: https://git.openjdk.java.net/jdk/pull/2816 From alanb at openjdk.java.net Mon Apr 12 06:47:34 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Mon, 12 Apr 2021 06:47:34 GMT Subject: RFR: JDK-8262957 (fs) Fail fast in UnixFileStore.isExtendedAttributesEnabled In-Reply-To: References: <-FT7LavNSb-NViqSfsISQvOUoic51dQsxuaQPBoirCI=.090bd820-2ef8-49d6-973e-8fba3278cab6@github.com> Message-ID: On Mon, 12 Apr 2021 06:18:57 GMT, Aleksey Shipilev wrote: > Alan, are you good with integrating this? Yes, one I get time to do more testing (the optimisation is for AIX only and I want to double check that we don't have side effects on the main stream platforms). ------------- PR: https://git.openjdk.java.net/jdk/pull/2816 From chegar at openjdk.java.net Mon Apr 12 08:31:51 2021 From: chegar at openjdk.java.net (Chris Hegarty) Date: Mon, 12 Apr 2021 08:31:51 GMT Subject: RFR: 8264827: Large mapped buffer/segment crash the VM when calling isLoaded [v2] In-Reply-To: References: Message-ID: > Avoid overflow when calculating the number of pages for large mapped segment sizes. Chris Hegarty has updated the pull request incrementally with one additional commit since the last revision: Skip testing on 32-bit systems ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3378/files - new: https://git.openjdk.java.net/jdk/pull/3378/files/1816b200..1dbe4a63 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3378&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3378&range=00-01 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3378.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3378/head:pull/3378 PR: https://git.openjdk.java.net/jdk/pull/3378 From mdoerr at openjdk.java.net Mon Apr 12 09:06:51 2021 From: mdoerr at openjdk.java.net (Martin Doerr) Date: Mon, 12 Apr 2021 09:06:51 GMT Subject: RFR: 8265018: [AIX] FileDispatcherImpl.c:31:10: fatal error: 'sys/mount.h' file not found In-Reply-To: References: Message-ID: <99bmWynE2UuzC-4nd4TiSdSoRuIlBmKhks--awKbmKE=.cbe4cc43-077a-4bb7-a1e0-eaa4b7826d3b@github.com> On Fri, 9 Apr 2021 21:39:53 GMT, Martin Doerr wrote: > #3366 introduced new includes which are not available on AIX, but they are only used on MacOS. Thanks for the review and the comment. Right, I think the related code should be BSD specific, not MacOS specific. So a larger cleanup change will be appreciated. This PR is just to fix the AIX build. ------------- PR: https://git.openjdk.java.net/jdk/pull/3424 From mdoerr at openjdk.java.net Mon Apr 12 09:06:52 2021 From: mdoerr at openjdk.java.net (Martin Doerr) Date: Mon, 12 Apr 2021 09:06:52 GMT Subject: Integrated: 8265018: [AIX] FileDispatcherImpl.c:31:10: fatal error: 'sys/mount.h' file not found In-Reply-To: References: Message-ID: On Fri, 9 Apr 2021 21:39:53 GMT, Martin Doerr wrote: > #3366 introduced new includes which are not available on AIX, but they are only used on MacOS. This pull request has now been integrated. Changeset: cb2806db Author: Martin Doerr URL: https://git.openjdk.java.net/jdk/commit/cb2806db Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod 8265018: [AIX] FileDispatcherImpl.c:31:10: fatal error: 'sys/mount.h' file not found Reviewed-by: clanger ------------- PR: https://git.openjdk.java.net/jdk/pull/3424 From alanb at openjdk.java.net Mon Apr 12 09:10:44 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Mon, 12 Apr 2021 09:10:44 GMT Subject: RFR: 8265018: [AIX] FileDispatcherImpl.c:31:10: fatal error: 'sys/mount.h' file not found In-Reply-To: <99bmWynE2UuzC-4nd4TiSdSoRuIlBmKhks--awKbmKE=.cbe4cc43-077a-4bb7-a1e0-eaa4b7826d3b@github.com> References: <99bmWynE2UuzC-4nd4TiSdSoRuIlBmKhks--awKbmKE=.cbe4cc43-077a-4bb7-a1e0-eaa4b7826d3b@github.com> Message-ID: <0ZeRA3dH75oUxXkxu7D75GkpPxUKOkxlgtprw0lhnUA=.0dc295a9-29cf-400d-b34a-552c8182ce97@github.com> On Mon, 12 Apr 2021 09:02:42 GMT, Martin Doerr wrote: > Thanks for the review and the comment. > Right, I think the related code should be BSD specific, not MacOS specific. So a larger cleanup change will be appreciated. > This PR is just to fix the AIX build. Sure but for this PR then please don't put #ifdef MACOSX in the middle of the shared includes. ------------- PR: https://git.openjdk.java.net/jdk/pull/3424 From mcimadamore at openjdk.java.net Mon Apr 12 10:07:55 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 12 Apr 2021 10:07:55 GMT Subject: RFR: 8264827: Large mapped buffer/segment crash the VM when calling isLoaded [v2] In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 08:31:51 GMT, Chris Hegarty wrote: >> Avoid overflow when calculating the number of pages for large mapped segment sizes. > > Chris Hegarty has updated the pull request incrementally with one additional commit since the last revision: > > Skip testing on 32-bit systems Looks good - thanks for fixing. ------------- Marked as reviewed by mcimadamore (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3378 From ccleary at openjdk.java.net Mon Apr 12 10:54:24 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Mon, 12 Apr 2021 10:54:24 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v6] In-Reply-To: References: Message-ID: > ### Description > SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. > > - Test is run in AgentVM mode. > - Receivers bind to wildcard/localhost which has been a source of instability. > - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. > - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. > > ### Fixes > To Address these issues, the following was done respective to the order of the Issues above: > - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. > - Receivers now bind to loopback address instead of localhost. > - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). > - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: 8264200: Removed author tag ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3354/files - new: https://git.openjdk.java.net/jdk/pull/3354/files/59424455..33f95b3c Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3354&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3354&range=04-05 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3354.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3354/head:pull/3354 PR: https://git.openjdk.java.net/jdk/pull/3354 From ccleary at openjdk.java.net Mon Apr 12 10:54:31 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Mon, 12 Apr 2021 10:54:31 GMT Subject: RFR: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently [v5] In-Reply-To: References: Message-ID: <4vmsUf3RE3Ot28XjNSpnl4gWyD8n_GpY1oxjkshBhNk=.279033cd-a301-463c-9dd9-ca580c50cd2e@github.com> On Sun, 11 Apr 2021 13:31:18 GMT, Alan Bateman wrote: >> Conor Cleary has updated the pull request incrementally with one additional commit since the last revision: >> >> 8264200: Changed port to dstPort in Writers > > test/jdk/java/nio/channels/DatagramChannel/SRTest.java line 26: > >> 24: /* @test >> 25: * @summary Test DatagramChannel's send and receive methods >> 26: * @author Mike McCloskey > > This change is essentially replacing this JDK 1.4 era test so probably should remove the @author tag. Removed author tag, marking as resolved ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From chegar at openjdk.java.net Mon Apr 12 11:25:54 2021 From: chegar at openjdk.java.net (Chris Hegarty) Date: Mon, 12 Apr 2021 11:25:54 GMT Subject: Integrated: 8264827: Large mapped buffer/segment crash the VM when calling isLoaded In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 15:45:30 GMT, Chris Hegarty wrote: > Avoid overflow when calculating the number of pages for large mapped segment sizes. This pull request has now been integrated. Changeset: 3c9858dd Author: Chris Hegarty URL: https://git.openjdk.java.net/jdk/commit/3c9858dd Stats: 36 lines in 5 files changed: 23 ins; 0 del; 13 mod 8264827: Large mapped buffer/segment crash the VM when calling isLoaded Reviewed-by: alanb, mcimadamore ------------- PR: https://git.openjdk.java.net/jdk/pull/3378 From brian.burkhalter at oracle.com Mon Apr 12 22:29:03 2021 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 12 Apr 2021 22:29:03 +0000 Subject: RFR: 8264400: (fs) WindowsFileStore equality depends on how the FileStore was constructed [v6] In-Reply-To: <1yCLzVQjMPTLhOnpsM-3zzpcQ8z362iTJ5utHziImhg=.af52968d-8c00-4374-b7dd-0d5ec0c5c2a6@github.com> References: <1yCLzVQjMPTLhOnpsM-3zzpcQ8z362iTJ5utHziImhg=.af52968d-8c00-4374-b7dd-0d5ec0c5c2a6@github.com> Message-ID: <5A655035-19EB-4C04-A6CE-ACCBC99A3C29@oracle.com> On Apr 9, 2021, at 11:46 PM, Alan Bateman > wrote: On Fri, 9 Apr 2021 22:24:03 GMT, Andrey Turbanov > wrote: Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8264400: Remove vestigial FileSystems.getDefault() src/java.base/windows/classes/sun/nio/fs/WindowsFileStore.java line 257: 255: root.toLowerCase(Locale.ROOT).hashCode() : root.hashCode(); 256: } 257: return hashCode; As I know, to make data race benign, field should be read only once. But there is second read which can still return `0` You can are right, missed in the final review (as this one went through many iter Filed JDK-8265100 to address this oversight. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bpb at openjdk.java.net Mon Apr 12 23:19:16 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 12 Apr 2021 23:19:16 GMT Subject: RFR: 8265100: (fs) WindowsFileStore.hashCode() should read cached hash code once Message-ID: Please review this small request to clean up `WindowsFileStore.hasCode()`. ------------- Commit messages: - 8265100: (fs) WindowsFileStore.hashCode() should read cached hash code once Changes: https://git.openjdk.java.net/jdk/pull/3445/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3445&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265100 Stats: 6 lines in 1 file changed: 2 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3445.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3445/head:pull/3445 PR: https://git.openjdk.java.net/jdk/pull/3445 From alanb at openjdk.java.net Tue Apr 13 05:40:58 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 13 Apr 2021 05:40:58 GMT Subject: RFR: 8265100: (fs) WindowsFileStore.hashCode() should read cached hash code once In-Reply-To: References: Message-ID: <7XzjewPw-F3AVpf9VMjX6T4-FuzpUjFKkP5EUiawnSE=.a8e916b8-a260-4a0d-940c-639a31a59532@github.com> On Mon, 12 Apr 2021 23:11:04 GMT, Brian Burkhalter wrote: > Please review this small request to clean up `WindowsFileStore.hashCode()`. src/java.base/windows/classes/sun/nio/fs/WindowsFileStore.java line 55: > 53: private final String displayName; // returned by toString > 54: > 55: private volatile int hashCode; It doesn't needed to be volatile, it's okay to race in the hashCode method. ------------- PR: https://git.openjdk.java.net/jdk/pull/3445 From ccleary at openjdk.java.net Tue Apr 13 14:09:03 2021 From: ccleary at openjdk.java.net (Conor Cleary) Date: Tue, 13 Apr 2021 14:09:03 GMT Subject: Integrated: 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 10:24:50 GMT, Conor Cleary wrote: > ### Description > SRTest.java has been seen to fail intermittently in a similar manner to other recent failures in [DatagramChannel tests](https://github.com/openjdk/jdk/pull/679). With SRTest.java, these failures are associated with a number of issues listed below. Note that XReader refers to both ClassicReader and NioReader classes. Likewise for XWriter. > > - Test is run in AgentVM mode. > - Receivers bind to wildcard/localhost which has been a source of instability. > - Sockets are not closed properly if an exception is thrown, even more of an issue due to test running in AgentVM mode. > - An XReader instance will wait forever to receive a `DatagramPacket` if an XSender has not sent one. > > ### Fixes > To Address these issues, the following was done respective to the order of the Issues above: > - Test now runs with `testng/othervm` and uses its own thread pool. Note that the change to testng was not absolutely essential but means that if one test case fails the others will still be run which could provide useful information on why a failure occured. > - Receivers now bind to loopback address instead of localhost. > - XReader and XWriter classes implement `Autocloseable` and are declared within a try-with-resources block for each test case. As well as this, the test cases are now executed with `CompleteableFuture.allOf(futures)` such that if any Reader or Writer throws an exception during execution, the threads will exit in a safe manner and the exception concerned will be thrown (wrapped in a `CompletionException`). > - Test will now timeout with respect to timeout limit of Jtreg which is set to 20 seconds for this test. Note that this is given in the `@run` as `testng/othervm/timeout=20` tag in seconds. See Action Options in Jtreg docs [here](https://openjdk.java.net/jtreg/tag-spec.html). This pull request has now been integrated. Changeset: 784f1c1f Author: Conor Cleary Committer: Aleksei Efimov URL: https://git.openjdk.java.net/jdk/commit/784f1c1f Stats: 164 lines in 1 file changed: 70 ins; 39 del; 55 mod 8264200: java/nio/channels/DatagramChannel/SRTest.java fails intermittently Reviewed-by: dfuchs, rriggs, msheppar ------------- PR: https://git.openjdk.java.net/jdk/pull/3354 From bpb at openjdk.java.net Tue Apr 13 15:57:26 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 13 Apr 2021 15:57:26 GMT Subject: RFR: 8265100: (fs) WindowsFileStore.hashCode() should read cached hash code once [v2] In-Reply-To: References: Message-ID: <2LeDcGppBuxr8_bZQVtnd8xpSsFPGC1je2d8H9xGNKY=.850a04ec-538f-483c-8be7-5f4bc9230b6e@github.com> > Please review this small request to clean up `WindowsFileStore.hashCode()`. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8265100: Make hashCode non-volatile ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3445/files - new: https://git.openjdk.java.net/jdk/pull/3445/files/e3bc63c2..faa48706 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3445&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3445&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3445.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3445/head:pull/3445 PR: https://git.openjdk.java.net/jdk/pull/3445 From alanb at openjdk.java.net Tue Apr 13 18:32:03 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 13 Apr 2021 18:32:03 GMT Subject: RFR: 8265100: (fs) WindowsFileStore.hashCode() should read cached hash code once [v2] In-Reply-To: <2LeDcGppBuxr8_bZQVtnd8xpSsFPGC1je2d8H9xGNKY=.850a04ec-538f-483c-8be7-5f4bc9230b6e@github.com> References: <2LeDcGppBuxr8_bZQVtnd8xpSsFPGC1je2d8H9xGNKY=.850a04ec-538f-483c-8be7-5f4bc9230b6e@github.com> Message-ID: On Tue, 13 Apr 2021 15:57:26 GMT, Brian Burkhalter wrote: >> Please review this small request to clean up `WindowsFileStore.hashCode()`. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8265100: Make hashCode non-volatile Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3445 From bpb at openjdk.java.net Tue Apr 13 18:48:58 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 13 Apr 2021 18:48:58 GMT Subject: Integrated: 8265100: (fs) WindowsFileStore.hashCode() should read cached hash code once In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 23:11:04 GMT, Brian Burkhalter wrote: > Please review this small request to clean up `WindowsFileStore.hashCode()`. This pull request has now been integrated. Changeset: 3b576ed9 Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/3b576ed9 Stats: 5 lines in 1 file changed: 2 ins; 0 del; 3 mod 8265100: (fs) WindowsFileStore.hashCode() should read cached hash code once Reviewed-by: alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/3445 From bpb at openjdk.java.net Tue Apr 13 21:47:22 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 13 Apr 2021 21:47:22 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use zero-copy on Linux Message-ID: Please consider this request to change the underlying native implementation of `java.nio.file.Files.copy(Path,Path,CopyOption...)` on Linux only to perform zero-copy via `sendfile(2)`. The `sendfile()` system call is already used in `java.nio.channels.FIleChannel.transferTo?(long,long,WritableByteChannel)`. It is intentionally _not_ proposed to use `sendfile()` in the native macOS implementation as on macOS the function requires that the destination file descriptor is for a socket. This change showed some performance improvement as measured by JMH. **Before: user-space buffers (read() + write())** Benchmark (size) Mode Cnt Score Error Units FilesCopy.copy 10240 thrpt 5 39167.400 ? 683.887 ops/s FilesCopy.copy 51200 thrpt 5 20782.622 ? 558.031 ops/s FilesCopy.copy 102400 thrpt 5 13260.709 ? 176.673 ops/s FilesCopy.copy 512000 thrpt 5 3171.837 ? 175.803 ops/s FilesCopy.copy 1048568 thrpt 5 1654.253 ? 39.419 ops/s FilesCopy.copy 10485760 thrpt 5 145.328 ? 7.192 ops/s FilesCopy.copy 104857600 thrpt 5 12.440 ? 2.275 ops/s FilesCopy.copy 1073741824 thrpt 5 1.073 ? 0.081 ops/s **After: zero-copy (sendfile())** Benchmark (size) Mode Cnt Score Error Units FilesCopy.copy 10240 thrpt 5 40571.516 ? 548.977 ops/s FilesCopy.copy 51200 thrpt 5 23993.334 ? 506.817 ops/s FilesCopy.copy 102400 thrpt 5 15927.485 ? 309.081 ops/s FilesCopy.copy 512000 thrpt 5 4207.129 ? 95.454 ops/s FilesCopy.copy 1048568 thrpt 5 2147.046 ? 33.446 ops/s FilesCopy.copy 10485760 thrpt 5 148.798 ? 1.329 ops/s FilesCopy.copy 104857600 thrpt 5 14.541 ? 0.675 ops/s FilesCopy.copy 1073741824 thrpt 5 1.270 ? 0.029 ops/s ------------- Commit messages: - 8265175: (fs) Files.copy(Path,Path,CopyOption...) should use zero-copy on Linux Changes: https://git.openjdk.java.net/jdk/pull/3476/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3476&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265175 Stats: 41 lines in 1 file changed: 38 ins; 1 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3476.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3476/head:pull/3476 PR: https://git.openjdk.java.net/jdk/pull/3476 From bpb at openjdk.java.net Wed Apr 14 01:42:15 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 14 Apr 2021 01:42:15 GMT Subject: RFR: 8264821: DirectIOTest fails on a system with large block size Message-ID: Please review this request to clean up a test. The main change is to ensure that the capacity passed to `ByteBuffer.allocateDirect()` is large enough that when `alignedSlice()` is called on the new buffer, it creates an aligned buffer which does not have size zero due due to position rounding resulting in out-of-bound values. Other changes in the test are cleanup. ------------- Commit messages: - 8264821: DirectIOTest fails on a system with large block size Changes: https://git.openjdk.java.net/jdk/pull/3482/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3482&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264821 Stats: 33 lines in 1 file changed: 7 ins; 8 del; 18 mod Patch: https://git.openjdk.java.net/jdk/pull/3482.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3482/head:pull/3482 PR: https://git.openjdk.java.net/jdk/pull/3482 From github.com+1204330+overheadhunter at openjdk.java.net Wed Apr 14 09:58:58 2021 From: github.com+1204330+overheadhunter at openjdk.java.net (Sebastian Stenzel) Date: Wed, 14 Apr 2021 09:58:58 GMT Subject: Integrated: JDK-8262957 (fs) Fail fast in UnixFileStore.isExtendedAttributesEnabled In-Reply-To: References: Message-ID: On Wed, 3 Mar 2021 22:15:38 GMT, Sebastian Stenzel wrote: > Added new capability flag `UnixNativeDispatcher.SUPPORTS_XATTR`, which gets set `#ifdef _SYS_XATTR_H_`. > > Note that `_SYS_XATTR_H_` is defined in `xattr.h` in both [macOS/Darwin](https://github.com/apple/darwin-xnu/blob/main/bsd/sys/xattr.h), [Linux and GNU-based BSD systems using glibc](https://github.com/bminor/glibc/blob/master/misc/sys/xattr.h). It might not be defined for other operating systems that still support xattr. So if OpenJDK eventually adds support for further platforms, this might need to be adjusted as well. > > If xattr capabilities are missing, `UnixFileStore.isExtendedAttributesEnabled` will return false immediately, avoiding any I/O. > > As a side effect of this change, I redefined some other (private) capabilities. Strictly speaking, this change is not required but keeps the code consistent, as [previously discussed in the mailing list](https://mail.openjdk.java.net/pipermail/nio-dev/2021-February/008133.html). This pull request has now been integrated. Changeset: 27dd88bf Author: Sebastian Stenzel Committer: Alan Bateman URL: https://git.openjdk.java.net/jdk/commit/27dd88bf Stats: 21 lines in 3 files changed: 19 ins; 0 del; 2 mod 8262957: (fs) Fail fast in UnixFileStore.isExtendedAttributesEnabled Reviewed-by: alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/2816 From alanb at openjdk.java.net Wed Apr 14 10:24:58 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 14 Apr 2021 10:24:58 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use zero-copy on Linux In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 21:40:26 GMT, Brian Burkhalter wrote: > Please consider this request to change the underlying native implementation of `java.nio.file.Files.copy(Path,Path,CopyOption...)` on Linux only to perform zero-copy via `sendfile(2)`. The `sendfile()` system call is already used in `java.nio.channels.FIleChannel.transferTo?(long,long,WritableByteChannel)`. It is intentionally _not_ proposed to use `sendfile()` in the native macOS implementation as on macOS the function requires that the destination file descriptor is for a socket. > > This change showed some performance improvement as measured by JMH. > > **Before: user-space buffers (read() + write())** > > Benchmark (size) Mode Cnt Score Error Units > FilesCopy.copy 10240 thrpt 5 39167.400 ? 683.887 ops/s > FilesCopy.copy 51200 thrpt 5 20782.622 ? 558.031 ops/s > FilesCopy.copy 102400 thrpt 5 13260.709 ? 176.673 ops/s > FilesCopy.copy 512000 thrpt 5 3171.837 ? 175.803 ops/s > FilesCopy.copy 1048568 thrpt 5 1654.253 ? 39.419 ops/s > FilesCopy.copy 10485760 thrpt 5 145.328 ? 7.192 ops/s > FilesCopy.copy 104857600 thrpt 5 12.440 ? 2.275 ops/s > FilesCopy.copy 1073741824 thrpt 5 1.073 ? 0.081 ops/s > > **After: zero-copy (sendfile())** > > Benchmark (size) Mode Cnt Score Error Units > FilesCopy.copy 10240 thrpt 5 40571.516 ? 548.977 ops/s > FilesCopy.copy 51200 thrpt 5 23993.334 ? 506.817 ops/s > FilesCopy.copy 102400 thrpt 5 15927.485 ? 309.081 ops/s > FilesCopy.copy 512000 thrpt 5 4207.129 ? 95.454 ops/s > FilesCopy.copy 1048568 thrpt 5 2147.046 ? 33.446 ops/s > FilesCopy.copy 10485760 thrpt 5 148.798 ? 1.329 ops/s > FilesCopy.copy 104857600 thrpt 5 14.541 ? 0.675 ops/s > FilesCopy.copy 1073741824 thrpt 5 1.270 ? 0.029 ops/s I agree it's worth exploring using sendfile64 as file-to-file was not available when this code was written. Have you measured the user vs. kernel time when testing? I'm also interested to see if you've tried using a buffer size that is a LCM of the source and destination block sizes. For large files, can you can what byte_sent returns? I'm wondering if the cancellable mechanism is effective with this implementation. The call already has the file size so the call to fstat is not needed. I think I'd prefer to continue to read to EOF as that it is a bit friendly to files that are changing when being copied. I assume the current patch is problematic on 32-bit (use sendfile64 and off64_t to be consistent with FileChannelImpl). It will also need to handle the EINTR case. ------------- Changes requested by alanb (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3476 From chegar at openjdk.java.net Wed Apr 14 14:20:40 2021 From: chegar at openjdk.java.net (Chris Hegarty) Date: Wed, 14 Apr 2021 14:20:40 GMT Subject: RFR: 8264821: DirectIOTest fails on a system with large block size In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 01:37:19 GMT, Brian Burkhalter wrote: > Please review this request to clean up a test. The main change is to ensure that the capacity passed to `ByteBuffer.allocateDirect()` is large enough that when `alignedSlice()` is called on the new buffer, it creates an aligned buffer which does not have size zero due due to position rounding resulting in out-of-bound values. Other changes in the test are cleanup. Marked as reviewed by chegar (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3482 From lancea at openjdk.java.net Wed Apr 14 14:25:45 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Wed, 14 Apr 2021 14:25:45 GMT Subject: RFR: 8262883: doccheck: Broken links in java.base In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 14:03:01 GMT, Michael McMahon wrote: > Hi, > > Could I get the following trivial doc changes reviewed please, caused by: > > - broken tags in MethodHandles referring to package.html instead of package-summary.html > > - references to a package level #unixdomain anchor that no longer exists. > > - a tag missing a "../" in SocketChannel > > Thanks, > > Michael Marked as reviewed by lancea (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3491 From michaelm at openjdk.java.net Wed Apr 14 15:03:40 2021 From: michaelm at openjdk.java.net (Michael McMahon) Date: Wed, 14 Apr 2021 15:03:40 GMT Subject: Integrated: 8262883: doccheck: Broken links in java.base In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 14:03:01 GMT, Michael McMahon wrote: > Hi, > > Could I get the following trivial doc changes reviewed please, caused by: > > - broken tags in MethodHandles referring to package.html instead of package-summary.html > > - references to a package level #unixdomain anchor that no longer exists. > > - a tag missing a "../" in SocketChannel > > Thanks, > > Michael This pull request has now been integrated. Changeset: 46616909 Author: Michael McMahon URL: https://git.openjdk.java.net/jdk/commit/46616909 Stats: 5 lines in 4 files changed: 0 ins; 0 del; 5 mod 8262883: doccheck: Broken links in java.base Reviewed-by: lancea ------------- PR: https://git.openjdk.java.net/jdk/pull/3491 From alanb at openjdk.java.net Wed Apr 14 15:30:44 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 14 Apr 2021 15:30:44 GMT Subject: RFR: 8262883: doccheck: Broken links in java.base In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 14:03:01 GMT, Michael McMahon wrote: > Hi, > > Could I get the following trivial doc changes reviewed please, caused by: > > - broken tags in MethodHandles referring to package.html instead of package-summary.html > > - references to a package level #unixdomain anchor that no longer exists. > > - a tag missing a "../" in SocketChannel > > Thanks, > > Michael src/java.base/share/classes/java/nio/channels/DatagramChannel.java line 175: > 173: * connected. > 174: * > 175: * @apiNote {@linkplain java.net.UnixDomainSocketAddress Unix domain} sockets The parameter to this method is a ProtocolFamily and maybe this note should be say that StandardProtocolFamily#UNIX is not supported. ------------- PR: https://git.openjdk.java.net/jdk/pull/3491 From bpb at openjdk.java.net Wed Apr 14 15:36:47 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 14 Apr 2021 15:36:47 GMT Subject: Integrated: 8264821: DirectIOTest fails on a system with large block size In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 01:37:19 GMT, Brian Burkhalter wrote: > Please review this request to clean up a test. The main change is to ensure that the capacity passed to `ByteBuffer.allocateDirect()` is large enough that when `alignedSlice()` is called on the new buffer, it creates an aligned buffer which does not have size zero due due to position rounding resulting in out-of-bound values. Other changes in the test are cleanup. This pull request has now been integrated. Changeset: 7e4cd480 Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/7e4cd480 Stats: 33 lines in 1 file changed: 7 ins; 8 del; 18 mod 8264821: DirectIOTest fails on a system with large block size Reviewed-by: chegar ------------- PR: https://git.openjdk.java.net/jdk/pull/3482 From bpb at openjdk.java.net Wed Apr 14 15:41:37 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 14 Apr 2021 15:41:37 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use zero-copy on Linux In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 10:22:37 GMT, Alan Bateman wrote: > Have you measured the user vs. kernel time when testing? No. > I'm also interested to see if you've tried using a buffer size that is a LCM of the source and destination block sizes. No, the buffer size was 8192 and both block sizes 4096, which seems to be the largest that my Linux version can handle. > For large files, can you can what byte_sent returns? I'm wondering if the cancellable mechanism is effective with this implementation. For all files that I observed, bytes_sent was equal to the file size. Therefore I am also skeptical of the possibility of cancellation. From the man page: Note that a successful call to sendfile() may write fewer bytes than requested; the caller should be prepared to retry the call if there were unsent bytes. > The call already has the file size so the call to fstat is not needed. OK. > I think I'd prefer to continue to read to EOF as that it is a bit friendly to files that are changing when being copied. I assume the current patch is problematic on 32-bit (use sendfile64 and off64_t to be consistent with FileChannelImpl). It will also need to handle the EINTR case. I will revise it to address these three points. ------------- PR: https://git.openjdk.java.net/jdk/pull/3476 From alanb at openjdk.java.net Wed Apr 14 15:43:38 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 14 Apr 2021 15:43:38 GMT Subject: RFR: 8264821: DirectIOTest fails on a system with large block size In-Reply-To: References: Message-ID: <8cxrDUnOp9YFMGG8SoRhVKAID8Ku4RTRVwjN5tpwhnQ=.f5d05081-63db-4d5e-bf51-2158631a0f37@github.com> On Wed, 14 Apr 2021 01:37:19 GMT, Brian Burkhalter wrote: > Please review this request to clean up a test. The main change is to ensure that the capacity passed to `ByteBuffer.allocateDirect()` is large enough that when `alignedSlice()` is called on the new buffer, it creates an aligned buffer which does not have size zero due due to position rounding resulting in out-of-bound values. Other changes in the test are cleanup. test/jdk/java/nio/channels/FileChannel/directio/DirectIOTest.java line 72: > 70: private static int testRead(Path p) throws Exception { > 71: try (FileChannel fc = FileChannel.open(p, ExtendedOpenOption.DIRECT)) { > 72: int bs = (int)blockSize; It might be a bit clearer to pass the blockSize to the testRead and testWrite methods. ------------- PR: https://git.openjdk.java.net/jdk/pull/3482 From brian.burkhalter at oracle.com Wed Apr 14 15:46:46 2021 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 14 Apr 2021 15:46:46 +0000 Subject: RFR: 8264821: DirectIOTest fails on a system with large block size In-Reply-To: <8cxrDUnOp9YFMGG8SoRhVKAID8Ku4RTRVwjN5tpwhnQ=.f5d05081-63db-4d5e-bf51-2158631a0f37@github.com> References: <8cxrDUnOp9YFMGG8SoRhVKAID8Ku4RTRVwjN5tpwhnQ=.f5d05081-63db-4d5e-bf51-2158631a0f37@github.com> Message-ID: <5AF7DE7D-BE9C-4126-B453-9058B1F1B58F@oracle.com> On Apr 14, 2021, at 8:43 AM, Alan Bateman > wrote: test/jdk/java/nio/channels/FileChannel/directio/DirectIOTest.java line 72: 70: private static int testRead(Path p) throws Exception { 71: try (FileChannel fc = FileChannel.open(p, ExtendedOpenOption.DIRECT)) { 72: int bs = (int)blockSize; It might be a bit clearer to pass the blockSize to the testRead and testWrite methods. You are probably correct but I already integrated this change. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rriggs at openjdk.java.net Wed Apr 14 16:08:41 2021 From: rriggs at openjdk.java.net (Roger Riggs) Date: Wed, 14 Apr 2021 16:08:41 GMT Subject: RFR: 8262883: doccheck: Broken links in java.base In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 14:03:01 GMT, Michael McMahon wrote: > Hi, > > Could I get the following trivial doc changes reviewed please, caused by: > > - broken tags in MethodHandles referring to package.html instead of package-summary.html > > - references to a package level #unixdomain anchor that no longer exists. > > - a tag missing a "../" in SocketChannel > > Thanks, > > Michael Integrating less than an hour after the PR is started seems to be a bit too quick to allow useful input. Perhaps even for 'trivial' changes. ------------- PR: https://git.openjdk.java.net/jdk/pull/3491 From bpb at openjdk.java.net Wed Apr 14 16:14:10 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 14 Apr 2021 16:14:10 GMT Subject: RFR: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux [v2] In-Reply-To: References: Message-ID: > Please consider this request to change the underlying native implementation of `java.nio.file.Files.copy(Path,Path,CopyOption...)` on Linux only to perform zero-copy via `sendfile(2)`. The `sendfile()` system call is already used in `java.nio.channels.FIleChannel.transferTo?(long,long,WritableByteChannel)`. It is intentionally _not_ proposed to use `sendfile()` in the native macOS implementation as on macOS the function requires that the destination file descriptor is for a socket. > > This change showed some performance improvement as measured by JMH. > > **Before: user-space buffers (read() + write())** > > Benchmark (size) Mode Cnt Score Error Units > FilesCopy.copy 10240 thrpt 5 39167.400 ? 683.887 ops/s > FilesCopy.copy 51200 thrpt 5 20782.622 ? 558.031 ops/s > FilesCopy.copy 102400 thrpt 5 13260.709 ? 176.673 ops/s > FilesCopy.copy 512000 thrpt 5 3171.837 ? 175.803 ops/s > FilesCopy.copy 1048568 thrpt 5 1654.253 ? 39.419 ops/s > FilesCopy.copy 10485760 thrpt 5 145.328 ? 7.192 ops/s > FilesCopy.copy 104857600 thrpt 5 12.440 ? 2.275 ops/s > FilesCopy.copy 1073741824 thrpt 5 1.073 ? 0.081 ops/s > > **After: zero-copy (sendfile())** > > Benchmark (size) Mode Cnt Score Error Units > FilesCopy.copy 10240 thrpt 5 40571.516 ? 548.977 ops/s > FilesCopy.copy 51200 thrpt 5 23993.334 ? 506.817 ops/s > FilesCopy.copy 102400 thrpt 5 15927.485 ? 309.081 ops/s > FilesCopy.copy 512000 thrpt 5 4207.129 ? 95.454 ops/s > FilesCopy.copy 1048568 thrpt 5 2147.046 ? 33.446 ops/s > FilesCopy.copy 10485760 thrpt 5 148.798 ? 1.329 ops/s > FilesCopy.copy 104857600 thrpt 5 14.541 ? 0.675 ops/s > FilesCopy.copy 1073741824 thrpt 5 1.270 ? 0.029 ops/s Brian Burkhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: 8265175: (fs) Files.copy(Path,Path,CopyOption...) should use sendfile on Linux ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3476/files - new: https://git.openjdk.java.net/jdk/pull/3476/files/ded30fca..b180a4c4 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3476&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3476&range=00-01 Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3476.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3476/head:pull/3476 PR: https://git.openjdk.java.net/jdk/pull/3476 From michaelm at openjdk.java.net Wed Apr 14 16:20:51 2021 From: michaelm at openjdk.java.net (Michael McMahon) Date: Wed, 14 Apr 2021 16:20:51 GMT Subject: RFR: 8262883: doccheck: Broken links in java.base In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 15:27:36 GMT, Alan Bateman wrote: >> Hi, >> >> Could I get the following trivial doc changes reviewed please, caused by: >> >> - broken tags in MethodHandles referring to package.html instead of package-summary.html >> >> - references to a package level #unixdomain anchor that no longer exists. >> >> - a tag missing a "../" in SocketChannel >> >> Thanks, >> >> Michael > > src/java.base/share/classes/java/nio/channels/DatagramChannel.java line 175: > >> 173: * connected. >> 174: * >> 175: * @apiNote {@linkplain java.net.UnixDomainSocketAddress Unix domain} sockets > > The parameter to this method is a ProtocolFamily and maybe this note should be say that StandardProtocolFamily#UNIX is not supported. Okay, reasonable suggestion. I was a bit quick integrating. I will file a new issue to update the spec as suggested. ------------- PR: https://git.openjdk.java.net/jdk/pull/3491 From bpb at openjdk.java.net Wed Apr 14 18:33:45 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 14 Apr 2021 18:33:45 GMT Subject: RFR: 8265231: (fc) ReadDirect and WriteDirect tests fail after fix for JDK-8264821 Message-ID: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> Please review this request to fix a test build failure caused by removing the `isDirectIOSupportedByFS()` method from the test `DirectIOTest` but not removing its use in other tests. This also addresses the comment [1] in PR [2] of [3]. [1] https://github.com/openjdk/jdk/pull/3482#discussion_r613362581 [2] https://github.com/openjdk/jdk/pull/3482 [3] https://bugs.openjdk.java.net/browse/JDK-8264821 ------------- Commit messages: - 8265231: (fc) ReadDirect and WriteDirect tests fail after fix for JDK-8264821 Changes: https://git.openjdk.java.net/jdk/pull/3498/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3498&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265231 Stats: 26 lines in 5 files changed: 0 ins; 17 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/3498.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3498/head:pull/3498 PR: https://git.openjdk.java.net/jdk/pull/3498 From michaelm at openjdk.java.net Wed Apr 14 18:53:49 2021 From: michaelm at openjdk.java.net (Michael McMahon) Date: Wed, 14 Apr 2021 18:53:49 GMT Subject: RFR: 8265226: Follow on comments from 8262883: doccheck: Broken links in java.base Message-ID: Hi, Could I get the following change reviewed please (which may need a CSR)? It should have been considered as part of (8262883) which was integrated earlier today. Any other comments on that change can be considered here too. This is the change: diff a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java --- a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java +++ b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java @@ -170,12 +170,11 @@ * java.nio.channels.spi.SelectorProvider#openDatagramChannel(ProtocolFamily) * openDatagramChannel} method of the system-wide default {@link * java.nio.channels.spi.SelectorProvider} object. The channel will not be * connected. * - * @apiNote {@linkplain java.net.UnixDomainSocketAddress Unix domain} sockets - * are not supported by DatagramChannel. + * @apiNote {@link java.net.StandardProtocolFamily#UNIX} is not supported. * * @param family * The protocol family * * @return A new datagram channel Thanks, Michael. ------------- Commit messages: - updated DatagramChannel Changes: https://git.openjdk.java.net/jdk/pull/3494/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3494&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265226 Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3494.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3494/head:pull/3494 PR: https://git.openjdk.java.net/jdk/pull/3494 From alanb at openjdk.java.net Wed Apr 14 19:43:40 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 14 Apr 2021 19:43:40 GMT Subject: RFR: 8265231: (fc) ReadDirect and WriteDirect tests fail after fix for JDK-8264821 In-Reply-To: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> References: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> Message-ID: On Wed, 14 Apr 2021 18:28:06 GMT, Brian Burkhalter wrote: > Please review this request to fix a test build failure caused by removing the `isDirectIOSupportedByFS()` method from the test `DirectIOTest` but not removing its use in other tests. > > This also addresses the comment [1] in PR [2] of [3]. > > [1] https://github.com/openjdk/jdk/pull/3482#discussion_r613362581 > [2] https://github.com/openjdk/jdk/pull/3482 > [3] https://bugs.openjdk.java.net/browse/JDK-8264821 Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3498 From darcy at openjdk.java.net Wed Apr 14 19:43:40 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Wed, 14 Apr 2021 19:43:40 GMT Subject: RFR: 8265231: (fc) ReadDirect and WriteDirect tests fail after fix for JDK-8264821 In-Reply-To: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> References: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> Message-ID: On Wed, 14 Apr 2021 18:28:06 GMT, Brian Burkhalter wrote: > Please review this request to fix a test build failure caused by removing the `isDirectIOSupportedByFS()` method from the test `DirectIOTest` but not removing its use in other tests. > > This also addresses the comment [1] in PR [2] of [3]. > > [1] https://github.com/openjdk/jdk/pull/3482#discussion_r613362581 > [2] https://github.com/openjdk/jdk/pull/3482 > [3] https://bugs.openjdk.java.net/browse/JDK-8264821 Marked as reviewed by darcy (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3498 From dcubed at openjdk.java.net Wed Apr 14 19:43:40 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 14 Apr 2021 19:43:40 GMT Subject: RFR: 8265231: (fc) ReadDirect and WriteDirect tests fail after fix for JDK-8264821 In-Reply-To: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> References: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> Message-ID: On Wed, 14 Apr 2021 18:28:06 GMT, Brian Burkhalter wrote: > Please review this request to fix a test build failure caused by removing the `isDirectIOSupportedByFS()` method from the test `DirectIOTest` but not removing its use in other tests. > > This also addresses the comment [1] in PR [2] of [3]. > > [1] https://github.com/openjdk/jdk/pull/3482#discussion_r613362581 > [2] https://github.com/openjdk/jdk/pull/3482 > [3] https://bugs.openjdk.java.net/browse/JDK-8264821 Thumbs up! This is a trivial fix and does not need to wait 24 hours. I sanity checked by searching the repo for `isDirectIOSupportedByFS` and you've gotten the only remaining references. ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3498 From chegar at openjdk.java.net Wed Apr 14 19:43:41 2021 From: chegar at openjdk.java.net (Chris Hegarty) Date: Wed, 14 Apr 2021 19:43:41 GMT Subject: RFR: 8265231: (fc) ReadDirect and WriteDirect tests fail after fix for JDK-8264821 In-Reply-To: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> References: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> Message-ID: On Wed, 14 Apr 2021 18:28:06 GMT, Brian Burkhalter wrote: > Please review this request to fix a test build failure caused by removing the `isDirectIOSupportedByFS()` method from the test `DirectIOTest` but not removing its use in other tests. > > This also addresses the comment [1] in PR [2] of [3]. > > [1] https://github.com/openjdk/jdk/pull/3482#discussion_r613362581 > [2] https://github.com/openjdk/jdk/pull/3482 > [3] https://bugs.openjdk.java.net/browse/JDK-8264821 Marked as reviewed by chegar (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3498 From dcubed at openjdk.java.net Wed Apr 14 19:50:38 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Wed, 14 Apr 2021 19:50:38 GMT Subject: RFR: 8265231: (fc) ReadDirect and WriteDirect tests fail after fix for JDK-8264821 In-Reply-To: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> References: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> Message-ID: On Wed, 14 Apr 2021 18:28:06 GMT, Brian Burkhalter wrote: > Please review this request to fix a test build failure caused by removing the `isDirectIOSupportedByFS()` method from the test `DirectIOTest` but not removing its use in other tests. > > This also addresses the comment [1] in PR [2] of [3]. > > [1] https://github.com/openjdk/jdk/pull/3482#discussion_r613362581 > [2] https://github.com/openjdk/jdk/pull/3482 > [3] https://bugs.openjdk.java.net/browse/JDK-8264821 I don't think the bots will let me do this but: ------------- PR: https://git.openjdk.java.net/jdk/pull/3498 From bpb at openjdk.java.net Wed Apr 14 19:56:41 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 14 Apr 2021 19:56:41 GMT Subject: Integrated: 8265231: (fc) ReadDirect and WriteDirect tests fail after fix for JDK-8264821 In-Reply-To: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> References: <8AqgNFl-4pN5w4ifpqdyYqZOEfWdGHnAqNO35BT2Z7w=.edd0867c-dacf-412a-9c74-4faee3c75b63@github.com> Message-ID: On Wed, 14 Apr 2021 18:28:06 GMT, Brian Burkhalter wrote: > Please review this request to fix a test build failure caused by removing the `isDirectIOSupportedByFS()` method from the test `DirectIOTest` but not removing its use in other tests. > > This also addresses the comment [1] in PR [2] of [3]. > > [1] https://github.com/openjdk/jdk/pull/3482#discussion_r613362581 > [2] https://github.com/openjdk/jdk/pull/3482 > [3] https://bugs.openjdk.java.net/browse/JDK-8264821 This pull request has now been integrated. Changeset: d1b28e7a Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/d1b28e7a Stats: 26 lines in 5 files changed: 0 ins; 17 del; 9 mod 8265231: (fc) ReadDirect and WriteDirect tests fail after fix for JDK-8264821 Reviewed-by: alanb, dcubed, darcy, chegar ------------- PR: https://git.openjdk.java.net/jdk/pull/3498 From bpb at openjdk.java.net Wed Apr 14 23:28:38 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 14 Apr 2021 23:28:38 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux [v2] In-Reply-To: References: Message-ID: <_j7tIiNwocs6lWfgXHqgVGebnak8glLbvSfeMgAqIp4=.425b26cc-292d-45a8-b940-a224ea9d8820@github.com> On Wed, 14 Apr 2021 16:14:10 GMT, Brian Burkhalter wrote: >> Please consider this request to change the underlying native implementation of `java.nio.file.Files.copy(Path,Path,CopyOption...)` on Linux only to perform zero-copy via `sendfile(2)`. The `sendfile()` system call is already used in `java.nio.channels.FIleChannel.transferTo?(long,long,WritableByteChannel)`. It is intentionally _not_ proposed to use `sendfile()` in the native macOS implementation as on macOS the function requires that the destination file descriptor is for a socket. >> >> This change showed some performance improvement as measured by JMH. >> >> **Before: user-space buffers (read() + write())** >> >> Benchmark (size) Mode Cnt Score Error Units >> FilesCopy.copy 10240 thrpt 5 39167.400 ? 683.887 ops/s >> FilesCopy.copy 51200 thrpt 5 20782.622 ? 558.031 ops/s >> FilesCopy.copy 102400 thrpt 5 13260.709 ? 176.673 ops/s >> FilesCopy.copy 512000 thrpt 5 3171.837 ? 175.803 ops/s >> FilesCopy.copy 1048568 thrpt 5 1654.253 ? 39.419 ops/s >> FilesCopy.copy 10485760 thrpt 5 145.328 ? 7.192 ops/s >> FilesCopy.copy 104857600 thrpt 5 12.440 ? 2.275 ops/s >> FilesCopy.copy 1073741824 thrpt 5 1.073 ? 0.081 ops/s >> >> **After: zero-copy (sendfile())** >> >> Benchmark (size) Mode Cnt Score Error Units >> FilesCopy.copy 10240 thrpt 5 40571.516 ? 548.977 ops/s >> FilesCopy.copy 51200 thrpt 5 23993.334 ? 506.817 ops/s >> FilesCopy.copy 102400 thrpt 5 15927.485 ? 309.081 ops/s >> FilesCopy.copy 512000 thrpt 5 4207.129 ? 95.454 ops/s >> FilesCopy.copy 1048568 thrpt 5 2147.046 ? 33.446 ops/s >> FilesCopy.copy 10485760 thrpt 5 148.798 ? 1.329 ops/s >> FilesCopy.copy 104857600 thrpt 5 14.541 ? 0.675 ops/s >> FilesCopy.copy 1073741824 thrpt 5 1.270 ? 0.029 ops/s > > Brian Burkhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: > > 8265175: (fs) Files.copy(Path,Path,CopyOption...) should use sendfile on Linux >From the notes on the man page for `sendfile(2)`: sendfile() will transfer at most 0x7ffff000 (2,147,479,552) bytes, returning the number of bytes actually transferred. (This is true on both 32-bit and 64-bit systems.) I verified this behavior on Ubuntu. For files larger than 2147479552, it transfers chunks of size 2147479552 except possibly for the last chunk in which case the number of remaining bytes is returned. At EOF, zero is returned. Thus for such large files the cancel mechanism should work. ------------- PR: https://git.openjdk.java.net/jdk/pull/3476 From bpb at openjdk.java.net Wed Apr 14 23:47:57 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 14 Apr 2021 23:47:57 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux [v3] In-Reply-To: References: Message-ID: > Please consider this request to change the underlying native implementation of `java.nio.file.Files.copy(Path,Path,CopyOption...)` on Linux only to perform zero-copy via `sendfile(2)`. The `sendfile()` system call is already used in `java.nio.channels.FIleChannel.transferTo?(long,long,WritableByteChannel)`. It is intentionally _not_ proposed to use `sendfile()` in the native macOS implementation as on macOS the function requires that the destination file descriptor is for a socket. > > This change showed some performance improvement as measured by JMH. > > **Before: user-space buffers (read() + write())** > > Benchmark (size) Mode Cnt Score Error Units > FilesCopy.copy 10240 thrpt 5 39167.400 ? 683.887 ops/s > FilesCopy.copy 51200 thrpt 5 20782.622 ? 558.031 ops/s > FilesCopy.copy 102400 thrpt 5 13260.709 ? 176.673 ops/s > FilesCopy.copy 512000 thrpt 5 3171.837 ? 175.803 ops/s > FilesCopy.copy 1048568 thrpt 5 1654.253 ? 39.419 ops/s > FilesCopy.copy 10485760 thrpt 5 145.328 ? 7.192 ops/s > FilesCopy.copy 104857600 thrpt 5 12.440 ? 2.275 ops/s > FilesCopy.copy 1073741824 thrpt 5 1.073 ? 0.081 ops/s > > **After: zero-copy (sendfile())** > > Benchmark (size) Mode Cnt Score Error Units > FilesCopy.copy 10240 thrpt 5 40571.516 ? 548.977 ops/s > FilesCopy.copy 51200 thrpt 5 23993.334 ? 506.817 ops/s > FilesCopy.copy 102400 thrpt 5 15927.485 ? 309.081 ops/s > FilesCopy.copy 512000 thrpt 5 4207.129 ? 95.454 ops/s > FilesCopy.copy 1048568 thrpt 5 2147.046 ? 33.446 ops/s > FilesCopy.copy 10485760 thrpt 5 148.798 ? 1.329 ops/s > FilesCopy.copy 104857600 thrpt 5 14.541 ? 0.675 ops/s > FilesCopy.copy 1073741824 thrpt 5 1.270 ? 0.029 ops/s Brian Burkhalter has updated the pull request incrementally with three additional commits since the last revision: - Merge - 8265175: Delete fstat() calls, read to EOF, change sendfile() to sendfile64(), handle EINTR case - 8265175: (fs) Files.copy(Path,Path,CopyOption...) should use sendfile on Linux ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3476/files - new: https://git.openjdk.java.net/jdk/pull/3476/files/b180a4c4..ccdbe5e7 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3476&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3476&range=01-02 Stats: 27 lines in 1 file changed: 4 ins; 16 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/3476.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3476/head:pull/3476 PR: https://git.openjdk.java.net/jdk/pull/3476 From bpb at openjdk.java.net Wed Apr 14 23:47:57 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 14 Apr 2021 23:47:57 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux [v2] In-Reply-To: References: Message-ID: <54VZjQGlh_Uf0bK2S5NpVM3kR_-lZ6OvgH_ApiCMRk4=.ae2bc17b-f7cf-4036-87c3-b2cb1915ccb9@github.com> On Wed, 14 Apr 2021 16:14:10 GMT, Brian Burkhalter wrote: >> Please consider this request to change the underlying native implementation of `java.nio.file.Files.copy(Path,Path,CopyOption...)` on Linux only to perform zero-copy via `sendfile(2)`. The `sendfile()` system call is already used in `java.nio.channels.FIleChannel.transferTo?(long,long,WritableByteChannel)`. It is intentionally _not_ proposed to use `sendfile()` in the native macOS implementation as on macOS the function requires that the destination file descriptor is for a socket. >> >> This change showed some performance improvement as measured by JMH. >> >> **Before: user-space buffers (read() + write())** >> >> Benchmark (size) Mode Cnt Score Error Units >> FilesCopy.copy 10240 thrpt 5 39167.400 ? 683.887 ops/s >> FilesCopy.copy 51200 thrpt 5 20782.622 ? 558.031 ops/s >> FilesCopy.copy 102400 thrpt 5 13260.709 ? 176.673 ops/s >> FilesCopy.copy 512000 thrpt 5 3171.837 ? 175.803 ops/s >> FilesCopy.copy 1048568 thrpt 5 1654.253 ? 39.419 ops/s >> FilesCopy.copy 10485760 thrpt 5 145.328 ? 7.192 ops/s >> FilesCopy.copy 104857600 thrpt 5 12.440 ? 2.275 ops/s >> FilesCopy.copy 1073741824 thrpt 5 1.073 ? 0.081 ops/s >> >> **After: zero-copy (sendfile())** >> >> Benchmark (size) Mode Cnt Score Error Units >> FilesCopy.copy 10240 thrpt 5 40571.516 ? 548.977 ops/s >> FilesCopy.copy 51200 thrpt 5 23993.334 ? 506.817 ops/s >> FilesCopy.copy 102400 thrpt 5 15927.485 ? 309.081 ops/s >> FilesCopy.copy 512000 thrpt 5 4207.129 ? 95.454 ops/s >> FilesCopy.copy 1048568 thrpt 5 2147.046 ? 33.446 ops/s >> FilesCopy.copy 10485760 thrpt 5 148.798 ? 1.329 ops/s >> FilesCopy.copy 104857600 thrpt 5 14.541 ? 0.675 ops/s >> FilesCopy.copy 1073741824 thrpt 5 1.270 ? 0.029 ops/s > > Brian Burkhalter has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: > > 8265175: (fs) Files.copy(Path,Path,CopyOption...) should use sendfile on Linux The commit eddfa30 was verified on 32- and 64-bit JDKs in Ubuntu 20.04. ------------- PR: https://git.openjdk.java.net/jdk/pull/3476 From alanb at openjdk.java.net Thu Apr 15 08:25:34 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 15 Apr 2021 08:25:34 GMT Subject: RFR: 8265226: Follow on comments from 8262883: doccheck: Broken links in java.base In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 16:49:56 GMT, Michael McMahon wrote: > Hi, > > Could I get the following change reviewed please (which may need a CSR)? > > It should have been considered as part of (8262883) which was integrated earlier today. > Any other comments on that change can be considered here too. > > This is the change: > > > diff a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java > --- a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java > +++ b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java > @@ -170,12 +170,11 @@ > * java.nio.channels.spi.SelectorProvider#openDatagramChannel(ProtocolFamily) > * openDatagramChannel} method of the system-wide default {@link > * java.nio.channels.spi.SelectorProvider} object. The channel will not be > * connected. > * > - * @apiNote {@linkplain java.net.UnixDomainSocketAddress Unix domain} sockets > - * are not supported by DatagramChannel. > + * @apiNote {@link java.net.StandardProtocolFamily#UNIX} is not supported. > * > * @param family > * The protocol family > * > * @return A new datagram channel > > > Thanks, > > Michael. src/java.base/share/classes/java/nio/channels/DatagramChannel.java line 175: > 173: * connected. > 174: * > 175: * @apiNote {@link java.net.StandardProtocolFamily#UNIX} is not supported. It might be clearer to restore the original text ("Unix domain sockets are not support by DatagramChannel") and add something like "; invoking this method with the UNIX protocol family throws UnsupportedOperationException". (btw: I had changed the description of the JBS issue to reflect what this issue about, hopefully the commit can use the same description). ------------- PR: https://git.openjdk.java.net/jdk/pull/3494 From michaelm at openjdk.java.net Thu Apr 15 08:54:36 2021 From: michaelm at openjdk.java.net (Michael McMahon) Date: Thu, 15 Apr 2021 08:54:36 GMT Subject: RFR: 8265226: (dc) API note in DatagramChannel.open should link to StandardProtocolFamily.UNIX In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 08:22:43 GMT, Alan Bateman wrote: >> Hi, >> >> Could I get the following change reviewed please (which may need a CSR)? >> >> It should have been considered as part of (8262883) which was integrated earlier today. >> Any other comments on that change can be considered here too. >> >> This is the change: >> >> >> diff a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java >> --- a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java >> +++ b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java >> @@ -170,12 +170,11 @@ >> * java.nio.channels.spi.SelectorProvider#openDatagramChannel(ProtocolFamily) >> * openDatagramChannel} method of the system-wide default {@link >> * java.nio.channels.spi.SelectorProvider} object. The channel will not be >> * connected. >> * >> - * @apiNote {@linkplain java.net.UnixDomainSocketAddress Unix domain} sockets >> - * are not supported by DatagramChannel. >> + * @apiNote {@link java.net.StandardProtocolFamily#UNIX} is not supported. >> * >> * @param family >> * The protocol family >> * >> * @return A new datagram channel >> >> >> Thanks, >> >> Michael. > > src/java.base/share/classes/java/nio/channels/DatagramChannel.java line 175: > >> 173: * connected. >> 174: * >> 175: * @apiNote {@link java.net.StandardProtocolFamily#UNIX} is not supported. > > It might be clearer to restore the original text ("Unix domain sockets are not support by DatagramChannel") and add something like "; invoking this method with the UNIX protocol family throws UnsupportedOperationException". > > (btw: I had changed the description of the JBS issue to reflect what this issue about, hopefully the commit can use the same description). I've updated the PR to be the same as the bug report. But, is it necessary to mention UOE, given that the @throws already says: * @throws UnsupportedOperationException * If the specified protocol family is not supported. For example, * suppose the parameter is specified as {@link * java.net.StandardProtocolFamily#INET6 StandardProtocolFamily.INET6} * but IPv6 is not enabled on the platform. ------------- PR: https://git.openjdk.java.net/jdk/pull/3494 From michaelm at openjdk.java.net Thu Apr 15 09:25:35 2021 From: michaelm at openjdk.java.net (Michael McMahon) Date: Thu, 15 Apr 2021 09:25:35 GMT Subject: RFR: 8265226: (dc) API note in DatagramChannel.open should link to StandardProtocolFamily.UNIX In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 08:51:57 GMT, Michael McMahon wrote: >> src/java.base/share/classes/java/nio/channels/DatagramChannel.java line 175: >> >>> 173: * connected. >>> 174: * >>> 175: * @apiNote {@link java.net.StandardProtocolFamily#UNIX} is not supported. >> >> It might be clearer to restore the original text ("Unix domain sockets are not support by DatagramChannel") and add something like "; invoking this method with the UNIX protocol family throws UnsupportedOperationException". >> >> (btw: I had changed the description of the JBS issue to reflect what this issue about, hopefully the commit can use the same description). > > I've updated the PR to be the same as the bug report. But, is it necessary to mention UOE, given that the @throws already says: > > * @throws UnsupportedOperationException > * If the specified protocol family is not supported. For example, > * suppose the parameter is specified as {@link > * java.net.StandardProtocolFamily#INET6 StandardProtocolFamily.INET6} > * but IPv6 is not enabled on the platform. > > So, maybe just "Unix domain sockets are not supported by DatagramChannel" with the plain link to UDSA? How about the following? * @apiNote {@linkplain java.net.StandardProtocolFamily#UNIX Unix domain} * sockets are not supported. ------------- PR: https://git.openjdk.java.net/jdk/pull/3494 From alanb at openjdk.java.net Thu Apr 15 10:58:38 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 15 Apr 2021 10:58:38 GMT Subject: RFR: 8265226: (dc) API note in DatagramChannel.open should link to StandardProtocolFamily.UNIX In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 09:22:37 GMT, Michael McMahon wrote: >> I've updated the PR to be the same as the bug report. But, is it necessary to mention UOE, given that the @throws already says: >> >> * @throws UnsupportedOperationException >> * If the specified protocol family is not supported. For example, >> * suppose the parameter is specified as {@link >> * java.net.StandardProtocolFamily#INET6 StandardProtocolFamily.INET6} >> * but IPv6 is not enabled on the platform. >> >> So, maybe just "Unix domain sockets are not supported by DatagramChannel" with the plain link to UDSA? > > How about the following? > > * @apiNote {@linkplain java.net.StandardProtocolFamily#UNIX Unix domain} > * sockets are not supported. My concern with linking to UDSA is that it's a SocketAddress and the parameter is a ProtocolFamily. So if there is a link then I think it should be to StandardProtocolFamily#UNIX. The text above 'Unix domain sockets are not supported by DatagramSocket" without a link is okay too. ------------- PR: https://git.openjdk.java.net/jdk/pull/3494 From alanb at openjdk.java.net Thu Apr 15 11:12:35 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 15 Apr 2021 11:12:35 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux [v3] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 23:47:57 GMT, Brian Burkhalter wrote: >> Please consider this request to change the underlying native implementation of `java.nio.file.Files.copy(Path,Path,CopyOption...)` on Linux only to perform zero-copy via `sendfile(2)`. The `sendfile()` system call is already used in `java.nio.channels.FIleChannel.transferTo?(long,long,WritableByteChannel)`. It is intentionally _not_ proposed to use `sendfile()` in the native macOS implementation as on macOS the function requires that the destination file descriptor is for a socket. >> >> This change showed some performance improvement as measured by JMH. >> >> **Before: user-space buffers (read() + write())** >> >> Benchmark (size) Mode Cnt Score Error Units >> FilesCopy.copy 10240 thrpt 5 39167.400 ? 683.887 ops/s >> FilesCopy.copy 51200 thrpt 5 20782.622 ? 558.031 ops/s >> FilesCopy.copy 102400 thrpt 5 13260.709 ? 176.673 ops/s >> FilesCopy.copy 512000 thrpt 5 3171.837 ? 175.803 ops/s >> FilesCopy.copy 1048568 thrpt 5 1654.253 ? 39.419 ops/s >> FilesCopy.copy 10485760 thrpt 5 145.328 ? 7.192 ops/s >> FilesCopy.copy 104857600 thrpt 5 12.440 ? 2.275 ops/s >> FilesCopy.copy 1073741824 thrpt 5 1.073 ? 0.081 ops/s >> >> **After: zero-copy (sendfile())** >> >> Benchmark (size) Mode Cnt Score Error Units >> FilesCopy.copy 10240 thrpt 5 40571.516 ? 548.977 ops/s >> FilesCopy.copy 51200 thrpt 5 23993.334 ? 506.817 ops/s >> FilesCopy.copy 102400 thrpt 5 15927.485 ? 309.081 ops/s >> FilesCopy.copy 512000 thrpt 5 4207.129 ? 95.454 ops/s >> FilesCopy.copy 1048568 thrpt 5 2147.046 ? 33.446 ops/s >> FilesCopy.copy 10485760 thrpt 5 148.798 ? 1.329 ops/s >> FilesCopy.copy 104857600 thrpt 5 14.541 ? 0.675 ops/s >> FilesCopy.copy 1073741824 thrpt 5 1.270 ? 0.029 ops/s > > Brian Burkhalter has updated the pull request incrementally with three additional commits since the last revision: > > - Merge > - 8265175: Delete fstat() calls, read to EOF, change sendfile() to sendfile64(), handle EINTR case > - 8265175: (fs) Files.copy(Path,Path,CopyOption...) should use sendfile on Linux src/java.base/unix/native/libnio/fs/UnixCopyFile.c line 79: > 77: return; > 78: } > 79: #endif This version looks much better but it missing the error checking. If bytes_sent is -1 then we should throw UnixException with the errno like the existing code. What would you think about setting count to 1MB or 4MB so that there is at least some chance for cancellation when the file is huge. Given that the new implementation will call sendfile64 until EOF then there is no reason to fall through to the old implementation, the old implementation goes into the #else ... ------------- PR: https://git.openjdk.java.net/jdk/pull/3476 From michaelm at openjdk.java.net Thu Apr 15 13:01:04 2021 From: michaelm at openjdk.java.net (Michael McMahon) Date: Thu, 15 Apr 2021 13:01:04 GMT Subject: RFR: 8265226: (dc) API note in DatagramChannel.open should link to StandardProtocolFamily.UNIX [v2] In-Reply-To: References: Message-ID: > Hi, > > Could I get the following change reviewed please (which may need a CSR)? > > It should have been considered as part of (8262883) which was integrated earlier today. > Any other comments on that change can be considered here too. > > This is the change: > > > diff a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java > --- a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java > +++ b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java > @@ -170,12 +170,11 @@ > * java.nio.channels.spi.SelectorProvider#openDatagramChannel(ProtocolFamily) > * openDatagramChannel} method of the system-wide default {@link > * java.nio.channels.spi.SelectorProvider} object. The channel will not be > * connected. > * > - * @apiNote {@linkplain java.net.UnixDomainSocketAddress Unix domain} sockets > - * are not supported by DatagramChannel. > + * @apiNote {@link java.net.StandardProtocolFamily#UNIX} is not supported. > * > * @param family > * The protocol family > * > * @return A new datagram channel > > > Thanks, > > Michael. Michael McMahon has updated the pull request incrementally with one additional commit since the last revision: changed link from USDA to StandardProtocolFamily#UNIX ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3494/files - new: https://git.openjdk.java.net/jdk/pull/3494/files/4542900b..6f32b9a5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3494&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3494&range=00-01 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3494.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3494/head:pull/3494 PR: https://git.openjdk.java.net/jdk/pull/3494 From iris at openjdk.java.net Thu Apr 15 17:10:36 2021 From: iris at openjdk.java.net (Iris Clark) Date: Thu, 15 Apr 2021 17:10:36 GMT Subject: RFR: 8265226: (dc) API note in DatagramChannel.open should link to StandardProtocolFamily.UNIX [v2] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 13:01:04 GMT, Michael McMahon wrote: >> Hi, >> >> Could I get the following change reviewed please (which may need a CSR)? >> >> It should have been considered as part of (8262883) which was integrated earlier today. >> Any other comments on that change can be considered here too. >> >> This is the change: >> >> >> diff a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java >> --- a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java >> +++ b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java >> @@ -170,12 +170,11 @@ >> * java.nio.channels.spi.SelectorProvider#openDatagramChannel(ProtocolFamily) >> * openDatagramChannel} method of the system-wide default {@link >> * java.nio.channels.spi.SelectorProvider} object. The channel will not be >> * connected. >> * >> - * @apiNote {@linkplain java.net.UnixDomainSocketAddress Unix domain} sockets >> - * are not supported by DatagramChannel. >> + * @apiNote {@link java.net.StandardProtocolFamily#UNIX} is not supported. >> * >> * @param family >> * The protocol family >> * >> * @return A new datagram channel >> >> >> Thanks, >> >> Michael. > > Michael McMahon has updated the pull request incrementally with one additional commit since the last revision: > > changed link from USDA to StandardProtocolFamily#UNIX Marked as reviewed by iris (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3494 From bpb at openjdk.java.net Thu Apr 15 18:39:02 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 15 Apr 2021 18:39:02 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux [v4] In-Reply-To: References: Message-ID: > Please consider this request to change the underlying native implementation of `java.nio.file.Files.copy(Path,Path,CopyOption...)` on Linux only to perform zero-copy via `sendfile(2)`. The `sendfile()` system call is already used in `java.nio.channels.FIleChannel.transferTo?(long,long,WritableByteChannel)`. It is intentionally _not_ proposed to use `sendfile()` in the native macOS implementation as on macOS the function requires that the destination file descriptor is for a socket. > > This change showed some performance improvement as measured by JMH. > > **Before: user-space buffers (read() + write())** > > Benchmark (size) Mode Cnt Score Error Units > FilesCopy.copy 10240 thrpt 5 39167.400 ? 683.887 ops/s > FilesCopy.copy 51200 thrpt 5 20782.622 ? 558.031 ops/s > FilesCopy.copy 102400 thrpt 5 13260.709 ? 176.673 ops/s > FilesCopy.copy 512000 thrpt 5 3171.837 ? 175.803 ops/s > FilesCopy.copy 1048568 thrpt 5 1654.253 ? 39.419 ops/s > FilesCopy.copy 10485760 thrpt 5 145.328 ? 7.192 ops/s > FilesCopy.copy 104857600 thrpt 5 12.440 ? 2.275 ops/s > FilesCopy.copy 1073741824 thrpt 5 1.073 ? 0.081 ops/s > > **After: zero-copy (sendfile())** > > Benchmark (size) Mode Cnt Score Error Units > FilesCopy.copy 10240 thrpt 5 40571.516 ? 548.977 ops/s > FilesCopy.copy 51200 thrpt 5 23993.334 ? 506.817 ops/s > FilesCopy.copy 102400 thrpt 5 15927.485 ? 309.081 ops/s > FilesCopy.copy 512000 thrpt 5 4207.129 ? 95.454 ops/s > FilesCopy.copy 1048568 thrpt 5 2147.046 ? 33.446 ops/s > FilesCopy.copy 10485760 thrpt 5 148.798 ? 1.329 ops/s > FilesCopy.copy 104857600 thrpt 5 14.541 ? 0.675 ops/s > FilesCopy.copy 1073741824 thrpt 5 1.270 ? 0.029 ops/s Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8265175: Transfer in 1MB chunks; fail with errno if sendfile() returns -1; no fallback to read-write transfer ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3476/files - new: https://git.openjdk.java.net/jdk/pull/3476/files/ccdbe5e7..96ecba18 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3476&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3476&range=02-03 Stats: 16 lines in 1 file changed: 6 ins; 6 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3476.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3476/head:pull/3476 PR: https://git.openjdk.java.net/jdk/pull/3476 From bpb at openjdk.java.net Thu Apr 15 18:39:04 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 15 Apr 2021 18:39:04 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux [v3] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 11:10:00 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with three additional commits since the last revision: >> >> - Merge >> - 8265175: Delete fstat() calls, read to EOF, change sendfile() to sendfile64(), handle EINTR case >> - 8265175: (fs) Files.copy(Path,Path,CopyOption...) should use sendfile on Linux > > src/java.base/unix/native/libnio/fs/UnixCopyFile.c line 79: > >> 77: return; >> 78: } >> 79: #endif > > This version looks much better but it missing the error checking. If bytes_sent is -1 then we should throw UnixException with the errno like the existing code. > > What would you think about setting count to 1MB or 4MB so that there is at least some chance for cancellation when the file is huge. > > Given that the new implementation will call sendfile64 until EOF then there is no reason to fall through to the old implementation, the old implementation goes into the #else ... Changing `count` to 1MB and 4MB instead of 2147479552 bytes somewhat surprisingly did not have a huge effect on the throughput for the two largest transfer sizes tested. ------------- PR: https://git.openjdk.java.net/jdk/pull/3476 From alanb at openjdk.java.net Thu Apr 15 18:44:43 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 15 Apr 2021 18:44:43 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux [v4] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 18:39:02 GMT, Brian Burkhalter wrote: >> Please consider this request to change the underlying native implementation of `java.nio.file.Files.copy(Path,Path,CopyOption...)` on Linux only to perform zero-copy via `sendfile(2)`. The `sendfile()` system call is already used in `java.nio.channels.FIleChannel.transferTo?(long,long,WritableByteChannel)`. It is intentionally _not_ proposed to use `sendfile()` in the native macOS implementation as on macOS the function requires that the destination file descriptor is for a socket. >> >> This change showed some performance improvement as measured by JMH. >> >> **Before: user-space buffers (read() + write())** >> >> Benchmark (size) Mode Cnt Score Error Units >> FilesCopy.copy 10240 thrpt 5 39167.400 ? 683.887 ops/s >> FilesCopy.copy 51200 thrpt 5 20782.622 ? 558.031 ops/s >> FilesCopy.copy 102400 thrpt 5 13260.709 ? 176.673 ops/s >> FilesCopy.copy 512000 thrpt 5 3171.837 ? 175.803 ops/s >> FilesCopy.copy 1048568 thrpt 5 1654.253 ? 39.419 ops/s >> FilesCopy.copy 10485760 thrpt 5 145.328 ? 7.192 ops/s >> FilesCopy.copy 104857600 thrpt 5 12.440 ? 2.275 ops/s >> FilesCopy.copy 1073741824 thrpt 5 1.073 ? 0.081 ops/s >> >> **After: zero-copy (sendfile())** >> >> Benchmark (size) Mode Cnt Score Error Units >> FilesCopy.copy 10240 thrpt 5 40571.516 ? 548.977 ops/s >> FilesCopy.copy 51200 thrpt 5 23993.334 ? 506.817 ops/s >> FilesCopy.copy 102400 thrpt 5 15927.485 ? 309.081 ops/s >> FilesCopy.copy 512000 thrpt 5 4207.129 ? 95.454 ops/s >> FilesCopy.copy 1048568 thrpt 5 2147.046 ? 33.446 ops/s >> FilesCopy.copy 10485760 thrpt 5 148.798 ? 1.329 ops/s >> FilesCopy.copy 104857600 thrpt 5 14.541 ? 0.675 ops/s >> FilesCopy.copy 1073741824 thrpt 5 1.270 ? 0.029 ops/s > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8265175: Transfer in 1MB chunks; fail with errno if sendfile() returns -1; no fallback to read-write transfer src/java.base/unix/native/libnio/fs/UnixCopyFile.c line 79: > 77: } > 78: } while (bytes_sent > 0); > 79: #else Thanks, I think this right. I'll double it on Friday. ------------- PR: https://git.openjdk.java.net/jdk/pull/3476 From bpb at openjdk.java.net Thu Apr 15 18:51:36 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 15 Apr 2021 18:51:36 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux [v4] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 18:42:20 GMT, Alan Bateman wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8265175: Transfer in 1MB chunks; fail with errno if sendfile() returns -1; no fallback to read-write transfer > > src/java.base/unix/native/libnio/fs/UnixCopyFile.c line 79: > >> 77: } >> 78: } while (bytes_sent > 0); >> 79: #else > > Thanks, I think this right. I'll double it on Friday. All right, thanks. ------------- PR: https://git.openjdk.java.net/jdk/pull/3476 From alanb at openjdk.java.net Fri Apr 16 11:28:36 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 16 Apr 2021 11:28:36 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux [v4] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 18:39:02 GMT, Brian Burkhalter wrote: >> Please consider this request to change the underlying native implementation of `java.nio.file.Files.copy(Path,Path,CopyOption...)` on Linux only to perform zero-copy via `sendfile(2)`. The `sendfile()` system call is already used in `java.nio.channels.FIleChannel.transferTo?(long,long,WritableByteChannel)`. It is intentionally _not_ proposed to use `sendfile()` in the native macOS implementation as on macOS the function requires that the destination file descriptor is for a socket. >> >> This change showed some performance improvement as measured by JMH. >> >> **Before: user-space buffers (read() + write())** >> >> Benchmark (size) Mode Cnt Score Error Units >> FilesCopy.copy 10240 thrpt 5 39167.400 ? 683.887 ops/s >> FilesCopy.copy 51200 thrpt 5 20782.622 ? 558.031 ops/s >> FilesCopy.copy 102400 thrpt 5 13260.709 ? 176.673 ops/s >> FilesCopy.copy 512000 thrpt 5 3171.837 ? 175.803 ops/s >> FilesCopy.copy 1048568 thrpt 5 1654.253 ? 39.419 ops/s >> FilesCopy.copy 10485760 thrpt 5 145.328 ? 7.192 ops/s >> FilesCopy.copy 104857600 thrpt 5 12.440 ? 2.275 ops/s >> FilesCopy.copy 1073741824 thrpt 5 1.073 ? 0.081 ops/s >> >> **After: zero-copy (sendfile())** >> >> Benchmark (size) Mode Cnt Score Error Units >> FilesCopy.copy 10240 thrpt 5 40571.516 ? 548.977 ops/s >> FilesCopy.copy 51200 thrpt 5 23993.334 ? 506.817 ops/s >> FilesCopy.copy 102400 thrpt 5 15927.485 ? 309.081 ops/s >> FilesCopy.copy 512000 thrpt 5 4207.129 ? 95.454 ops/s >> FilesCopy.copy 1048568 thrpt 5 2147.046 ? 33.446 ops/s >> FilesCopy.copy 10485760 thrpt 5 148.798 ? 1.329 ops/s >> FilesCopy.copy 104857600 thrpt 5 14.541 ? 0.675 ops/s >> FilesCopy.copy 1073741824 thrpt 5 1.270 ? 0.029 ops/s > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8265175: Transfer in 1MB chunks; fail with errno if sendfile() returns -1; no fallback to read-write transfer Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3476 From alanb at openjdk.java.net Fri Apr 16 11:35:34 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 16 Apr 2021 11:35:34 GMT Subject: RFR: 8265226: (dc) API note in DatagramChannel.open should link to StandardProtocolFamily.UNIX [v2] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 13:01:04 GMT, Michael McMahon wrote: >> Hi, >> >> Could I get the following change reviewed please (which may need a CSR)? >> >> It should have been considered as part of (8262883) which was integrated earlier today. >> Any other comments on that change can be considered here too. >> >> This is the change: >> >> >> diff a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java >> --- a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java >> +++ b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java >> @@ -170,12 +170,11 @@ >> * java.nio.channels.spi.SelectorProvider#openDatagramChannel(ProtocolFamily) >> * openDatagramChannel} method of the system-wide default {@link >> * java.nio.channels.spi.SelectorProvider} object. The channel will not be >> * connected. >> * >> - * @apiNote {@linkplain java.net.UnixDomainSocketAddress Unix domain} sockets >> - * are not supported by DatagramChannel. >> + * @apiNote {@link java.net.StandardProtocolFamily#UNIX} is not supported. >> * >> * @param family >> * The protocol family >> * >> * @return A new datagram channel >> >> >> Thanks, >> >> Michael. > > Michael McMahon has updated the pull request incrementally with one additional commit since the last revision: > > changed link from USDA to StandardProtocolFamily#UNIX Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3494 From github.com+124581+dmlloyd at openjdk.java.net Fri Apr 16 12:16:38 2021 From: github.com+124581+dmlloyd at openjdk.java.net (David M.Lloyd) Date: Fri, 16 Apr 2021 12:16:38 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux [v4] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 18:39:02 GMT, Brian Burkhalter wrote: >> Please consider this request to change the underlying native implementation of `java.nio.file.Files.copy(Path,Path,CopyOption...)` on Linux only to perform zero-copy via `sendfile(2)`. The `sendfile()` system call is already used in `java.nio.channels.FIleChannel.transferTo?(long,long,WritableByteChannel)`. It is intentionally _not_ proposed to use `sendfile()` in the native macOS implementation as on macOS the function requires that the destination file descriptor is for a socket. >> >> This change showed some performance improvement as measured by JMH. >> >> **Before: user-space buffers (read() + write())** >> >> Benchmark (size) Mode Cnt Score Error Units >> FilesCopy.copy 10240 thrpt 5 39167.400 ? 683.887 ops/s >> FilesCopy.copy 51200 thrpt 5 20782.622 ? 558.031 ops/s >> FilesCopy.copy 102400 thrpt 5 13260.709 ? 176.673 ops/s >> FilesCopy.copy 512000 thrpt 5 3171.837 ? 175.803 ops/s >> FilesCopy.copy 1048568 thrpt 5 1654.253 ? 39.419 ops/s >> FilesCopy.copy 10485760 thrpt 5 145.328 ? 7.192 ops/s >> FilesCopy.copy 104857600 thrpt 5 12.440 ? 2.275 ops/s >> FilesCopy.copy 1073741824 thrpt 5 1.073 ? 0.081 ops/s >> >> **After: zero-copy (sendfile())** >> >> Benchmark (size) Mode Cnt Score Error Units >> FilesCopy.copy 10240 thrpt 5 40571.516 ? 548.977 ops/s >> FilesCopy.copy 51200 thrpt 5 23993.334 ? 506.817 ops/s >> FilesCopy.copy 102400 thrpt 5 15927.485 ? 309.081 ops/s >> FilesCopy.copy 512000 thrpt 5 4207.129 ? 95.454 ops/s >> FilesCopy.copy 1048568 thrpt 5 2147.046 ? 33.446 ops/s >> FilesCopy.copy 10485760 thrpt 5 148.798 ? 1.329 ops/s >> FilesCopy.copy 104857600 thrpt 5 14.541 ? 0.675 ops/s >> FilesCopy.copy 1073741824 thrpt 5 1.270 ? 0.029 ops/s > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8265175: Transfer in 1MB chunks; fail with errno if sendfile() returns -1; no fallback to read-write transfer It might also be worth having a look at `copy_file_range`: https://man7.org/linux/man-pages/man2/copy_file_range.2.html ------------- PR: https://git.openjdk.java.net/jdk/pull/3476 From bpb at openjdk.java.net Fri Apr 16 15:49:35 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 16 Apr 2021 15:49:35 GMT Subject: RFR: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux [v4] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 12:13:02 GMT, David M. Lloyd wrote: > It might also be worth having a look at `copy_file_range`: https://man7.org/linux/man-pages/man2/copy_file_range.2.html Thanks for the suggestion, but that requires a version of Linux too new for our production builds. ------------- PR: https://git.openjdk.java.net/jdk/pull/3476 From bpb at openjdk.java.net Fri Apr 16 15:49:36 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 16 Apr 2021 15:49:36 GMT Subject: Integrated: 8265175: (fs) Files.copy(Path, Path, CopyOption...) should use sendfile on Linux In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 21:40:26 GMT, Brian Burkhalter wrote: > Please consider this request to change the underlying native implementation of `java.nio.file.Files.copy(Path,Path,CopyOption...)` on Linux only to perform zero-copy via `sendfile(2)`. The `sendfile()` system call is already used in `java.nio.channels.FIleChannel.transferTo?(long,long,WritableByteChannel)`. It is intentionally _not_ proposed to use `sendfile()` in the native macOS implementation as on macOS the function requires that the destination file descriptor is for a socket. > > This change showed some performance improvement as measured by JMH. > > **Before: user-space buffers (read() + write())** > > Benchmark (size) Mode Cnt Score Error Units > FilesCopy.copy 10240 thrpt 5 39167.400 ? 683.887 ops/s > FilesCopy.copy 51200 thrpt 5 20782.622 ? 558.031 ops/s > FilesCopy.copy 102400 thrpt 5 13260.709 ? 176.673 ops/s > FilesCopy.copy 512000 thrpt 5 3171.837 ? 175.803 ops/s > FilesCopy.copy 1048568 thrpt 5 1654.253 ? 39.419 ops/s > FilesCopy.copy 10485760 thrpt 5 145.328 ? 7.192 ops/s > FilesCopy.copy 104857600 thrpt 5 12.440 ? 2.275 ops/s > FilesCopy.copy 1073741824 thrpt 5 1.073 ? 0.081 ops/s > > **After: zero-copy (sendfile())** > > Benchmark (size) Mode Cnt Score Error Units > FilesCopy.copy 10240 thrpt 5 40571.516 ? 548.977 ops/s > FilesCopy.copy 51200 thrpt 5 23993.334 ? 506.817 ops/s > FilesCopy.copy 102400 thrpt 5 15927.485 ? 309.081 ops/s > FilesCopy.copy 512000 thrpt 5 4207.129 ? 95.454 ops/s > FilesCopy.copy 1048568 thrpt 5 2147.046 ? 33.446 ops/s > FilesCopy.copy 10485760 thrpt 5 148.798 ? 1.329 ops/s > FilesCopy.copy 104857600 thrpt 5 14.541 ? 0.675 ops/s > FilesCopy.copy 1073741824 thrpt 5 1.270 ? 0.029 ops/s This pull request has now been integrated. Changeset: 1c3fd46d Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/1c3fd46d Stats: 29 lines in 1 file changed: 26 ins; 1 del; 2 mod 8265175: (fs) Files.copy(Path,Path,CopyOption...) should use sendfile on Linux Reviewed-by: alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/3476 From michaelm at openjdk.java.net Mon Apr 19 09:35:09 2021 From: michaelm at openjdk.java.net (Michael McMahon) Date: Mon, 19 Apr 2021 09:35:09 GMT Subject: RFR: 8265226: (dc) API note in DatagramChannel.open should link to StandardProtocolFamily.UNIX [v3] In-Reply-To: References: Message-ID: > Hi, > > Could I get the following change reviewed please (which may need a CSR)? > > It should have been considered as part of (8262883) which was integrated earlier today. > Any other comments on that change can be considered here too. > > This is the change: > > > diff a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java > --- a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java > +++ b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java > @@ -170,12 +170,11 @@ > * java.nio.channels.spi.SelectorProvider#openDatagramChannel(ProtocolFamily) > * openDatagramChannel} method of the system-wide default {@link > * java.nio.channels.spi.SelectorProvider} object. The channel will not be > * connected. > * > - * @apiNote {@linkplain java.net.UnixDomainSocketAddress Unix domain} sockets > - * are not supported by DatagramChannel. > + * @apiNote {@link java.net.StandardProtocolFamily#UNIX} is not supported. > * > * @param family > * The protocol family > * > * @return A new datagram channel > > > Thanks, > > Michael. Michael McMahon has updated the pull request incrementally with one additional commit since the last revision: fixed whitespace error ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3494/files - new: https://git.openjdk.java.net/jdk/pull/3494/files/6f32b9a5..d6595144 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3494&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3494&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3494.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3494/head:pull/3494 PR: https://git.openjdk.java.net/jdk/pull/3494 From michaelm at openjdk.java.net Mon Apr 19 09:35:10 2021 From: michaelm at openjdk.java.net (Michael McMahon) Date: Mon, 19 Apr 2021 09:35:10 GMT Subject: Integrated: 8265226: (dc) API note in DatagramChannel.open should link to StandardProtocolFamily.UNIX In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 16:49:56 GMT, Michael McMahon wrote: > Hi, > > Could I get the following change reviewed please (which may need a CSR)? > > It should have been considered as part of (8262883) which was integrated earlier today. > Any other comments on that change can be considered here too. > > This is the change: > > > diff a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java > --- a/src/java.base/share/classes/java/nio/channels/DatagramChannel.java > +++ b/src/java.base/share/classes/java/nio/channels/DatagramChannel.java > @@ -170,12 +170,11 @@ > * java.nio.channels.spi.SelectorProvider#openDatagramChannel(ProtocolFamily) > * openDatagramChannel} method of the system-wide default {@link > * java.nio.channels.spi.SelectorProvider} object. The channel will not be > * connected. > * > - * @apiNote {@linkplain java.net.UnixDomainSocketAddress Unix domain} sockets > - * are not supported by DatagramChannel. > + * @apiNote {@link java.net.StandardProtocolFamily#UNIX} is not supported. > * > * @param family > * The protocol family > * > * @return A new datagram channel > > > Thanks, > > Michael. This pull request has now been integrated. Changeset: d9e19f16 Author: Michael McMahon URL: https://git.openjdk.java.net/jdk/commit/d9e19f16 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod 8265226: (dc) API note in DatagramChannel.open should link to StandardProtocolFamily.UNIX Reviewed-by: iris, alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/3494 From bpb at openjdk.java.net Wed Apr 21 16:57:38 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 21 Apr 2021 16:57:38 GMT Subject: RFR: 8057113: (fs) Path should have a method to obtain the filename extension [v8] In-Reply-To: References: Message-ID: <8mZiewa8FCzM4ZPk8KndBnuROnSXk_VVAMbwfX6O6XM=.786b996d-dae6-4d1d-b173-410e9238e7a8@github.com> On Fri, 12 Mar 2021 21:33:18 GMT, Brian Burkhalter wrote: >> Please review this proposed change to add a method `java.nio.file.Path.getExtension()`. This was initially discussed in the thread http://mail.openjdk.java.net/pipermail/nio-dev/2018-February/004716.html. This method would return the filename extension of the file name of the `Path`. The extension is defined to be the portion of the file name after the last dot `(?.?)`. >> >> The definitions of file extension for about fifteen platforms and languages were surveyed to try to find a reasonable compromise for the definition of extension. The most common definition was the last segment of the name including and after the last dot. The second definition omitted the last dot from the extension. Java-related platforms all exclude the last dot. (One divergent definition in the internal Java NIO method `AbstractFileTypeDetector.getExtension(String)` defines the extension as the part after the *first* dot.) >> >> All examined cases define the extension to be an empty string if it cannot be determined. None of these cases used `null` to represent an indeterminate extension. >> >> Little in the way of specifying behavior for special cases (consisting mainly of file names with one or more leading dots) was found. Most definitions concern themselves only with the last dot and what comes after it and ignore leading dots altogether. A few definitions ignore a leading dot at the zeroth character. The current proposal ignores a dot at character zero. >> >> The behavior of the proposed method for some example cases is as: >> >> >> . -> >> .. -> >> .a.b -> b >> ...... -> >> .....a -> a >> ....a.b -> b >> ..foo -> foo >> test.rb -> rb >> a/b/d/test.rb -> rb >> .a/b/d/test.rb -> rb >> foo. -> >> test -> >> .profile -> >> .profile.sh -> sh >> ..foo -> foo >> .....foo -> foo >> .vimrc -> >> test. -> >> test.. -> >> test... -> >> foo.tar.gz -> gz >> foo.bar. -> >> image.jpg -> jpg >> music.mp3 -> mp3 >> video.mp4 -> mp4 >> document.txt -> txt >> foo.tar.gz -> gz >> foo.bar. -> > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8057113: Change first sentence; change param name PR extension requested. ------------- PR: https://git.openjdk.java.net/jdk/pull/2319 From bpb at openjdk.java.net Wed Apr 21 19:43:53 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 21 Apr 2021 19:43:53 GMT Subject: RFR: 8265699: (bf) Scopes passed to ScopedMemoryAccess.copy[Swap]Memory in incorrect order Message-ID: Please consider this request to correct what appear to be `jdk.internal.misc.ScopedMemoryAccess$Scope`s passed in incorrect order to the `copyMemory()` and `copySwapMemory()` methods of `jdk.internal.misc.ScopedMemoryAccess` in some of the `java.nio.Buffer` bulk transfer methods. ------------- Commit messages: - 8265699: (bf) Scopes passed to ScopedMemoryAccess.copy[Swap]Memory in incorrect order Changes: https://git.openjdk.java.net/jdk/pull/3607/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3607&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265699 Stats: 7 lines in 2 files changed: 0 ins; 0 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/3607.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3607/head:pull/3607 PR: https://git.openjdk.java.net/jdk/pull/3607 From mcimadamore at openjdk.java.net Wed Apr 21 20:13:33 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 21 Apr 2021 20:13:33 GMT Subject: RFR: 8265699: (bf) Scopes passed to ScopedMemoryAccess.copy[Swap]Memory in incorrect order In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 19:36:57 GMT, Brian Burkhalter wrote: > Please consider this request to correct what appear to be `jdk.internal.misc.ScopedMemoryAccess$Scope`s passed in incorrect order to the `copyMemory()` and `copySwapMemory()` methods of `jdk.internal.misc.ScopedMemoryAccess` in some of the `java.nio.Buffer` bulk transfer methods. Looks good except for one edit src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template line 551: > 549: int rem = (pos <= lim ? lim - pos : 0); > 550: try { > 551: SCOPED_MEMORY_ACCESS.copyMemory(scope(), scope(), null, Is this deliberate? same strange to use same scope for both address and target? ------------- PR: https://git.openjdk.java.net/jdk/pull/3607 From bpb at openjdk.java.net Wed Apr 21 20:18:35 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 21 Apr 2021 20:18:35 GMT Subject: RFR: 8265699: (bf) Scopes passed to ScopedMemoryAccess.copy[Swap]Memory in incorrect order In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 20:10:17 GMT, Maurizio Cimadamore wrote: >> Please consider this request to correct what appear to be `jdk.internal.misc.ScopedMemoryAccess$Scope`s passed in incorrect order to the `copyMemory()` and `copySwapMemory()` methods of `jdk.internal.misc.ScopedMemoryAccess` in some of the `java.nio.Buffer` bulk transfer methods. > > src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template line 551: > >> 549: int rem = (pos <= lim ? lim - pos : 0); >> 550: try { >> 551: SCOPED_MEMORY_ACCESS.copyMemory(scope(), scope(), null, > > Is this deliberate? same strange to use same scope for both address and target? Yes, it is. In this case the buffer is copying to itself, moving the data down to the beginning of the buffer, position zero. ------------- PR: https://git.openjdk.java.net/jdk/pull/3607 From bpb at openjdk.java.net Wed Apr 21 20:37:11 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Wed, 21 Apr 2021 20:37:11 GMT Subject: RFR: 8265699: (bf) Scopes passed to ScopedMemoryAccess.copy[Swap]Memory in incorrect order [v2] In-Reply-To: References: Message-ID: > Please consider this request to correct what appear to be `jdk.internal.misc.ScopedMemoryAccess$Scope`s passed in incorrect order to the `copyMemory()` and `copySwapMemory()` methods of `jdk.internal.misc.ScopedMemoryAccess` in some of the `java.nio.Buffer` bulk transfer methods. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8265699: Revert change to compact() and add a comment ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3607/files - new: https://git.openjdk.java.net/jdk/pull/3607/files/4d703bce..6c99b85d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3607&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3607&range=00-01 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3607.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3607/head:pull/3607 PR: https://git.openjdk.java.net/jdk/pull/3607 From mcimadamore at openjdk.java.net Thu Apr 22 13:26:24 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Thu, 22 Apr 2021 13:26:24 GMT Subject: RFR: 8265699: (bf) Scopes passed to ScopedMemoryAccess.copy[Swap]Memory in incorrect order [v2] In-Reply-To: References: Message-ID: <4Y7_hN4fdsofYnjU0KyBDXPXbA0v1fVRtfOWKw21iKA=.f35d7446-c541-457f-ac07-2387755f8dfe@github.com> On Wed, 21 Apr 2021 20:37:11 GMT, Brian Burkhalter wrote: >> Please consider this request to correct what appear to be `jdk.internal.misc.ScopedMemoryAccess$Scope`s passed in incorrect order to the `copyMemory()` and `copySwapMemory()` methods of `jdk.internal.misc.ScopedMemoryAccess` in some of the `java.nio.Buffer` bulk transfer methods. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8265699: Revert change to compact() and add a comment Looks good! Marked as reviewed by mcimadamore (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3607 From bpb at openjdk.java.net Thu Apr 22 19:47:25 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 22 Apr 2021 19:47:25 GMT Subject: Integrated: 8265699: (bf) Scopes passed to ScopedMemoryAccess.copy[Swap]Memory in incorrect order In-Reply-To: References: Message-ID: <6FcZm-ic9-XFy-iUTy-bpzxJBememnxC2OED9quM5N0=.c41457a0-e1dc-4e83-abd2-9e4efc149d2c@github.com> On Wed, 21 Apr 2021 19:36:57 GMT, Brian Burkhalter wrote: > Please consider this request to correct what appear to be `jdk.internal.misc.ScopedMemoryAccess$Scope`s passed in incorrect order to the `copyMemory()` and `copySwapMemory()` methods of `jdk.internal.misc.ScopedMemoryAccess` in some of the `java.nio.Buffer` bulk transfer methods. This pull request has now been integrated. Changeset: ca0de266 Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/ca0de266 Stats: 7 lines in 2 files changed: 1 ins; 0 del; 6 mod 8265699: (bf) Scopes passed to ScopedMemoryAccess.copy[Swap]Memory in incorrect order Reviewed-by: mcimadamore ------------- PR: https://git.openjdk.java.net/jdk/pull/3607 From yyang at openjdk.java.net Fri Apr 23 03:50:54 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Fri, 23 Apr 2021 03:50:54 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v2] In-Reply-To: References: Message-ID: > The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. > > In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. > > But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: > > 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. > 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag > > Testing: compiler and jdk Yi Yang has updated the pull request incrementally with one additional commit since the last revision: remove java_nio_Buffer in javaClasses.hpp ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3615/files - new: https://git.openjdk.java.net/jdk/pull/3615/files/a6affc54..7f30dc48 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3615&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3615&range=00-01 Stats: 27 lines in 2 files changed: 0 ins; 27 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3615.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3615/head:pull/3615 PR: https://git.openjdk.java.net/jdk/pull/3615 From bpb at openjdk.java.net Fri Apr 23 19:32:36 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 23 Apr 2021 19:32:36 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow Message-ID: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Please consider this request to accelerate absolute and relative bulk array transfer on views of heap byte buffers where the element size is greater than one. What currently happens is that the transfer devolves to a ?loopy? element-by-element copy such as int end = offset + length; for (int i = offset, j = index; i < end; i++, j++) dst[i] = get(j); for `get()`, and int end = offset + length; for (int i = offset, j = index; i < end; i++, j++) this.put(j, src[i]); for `put()`. This is of course relatively slow. The change proposed hoists the accelerated versions of these methods using the `ScopedMemoryAccess` methods `copyMemory()` and `copySwapMemory()` from `Direct-X-Buffer` to `X-Buffer`. The array bulk transfer methods are removed from `Direct-X-Buffer` itself. The number of lines of code in the templates decreases by 87. With this change the throughput of array bulk `put()` and `get()` for heap view buffers is increased by a factor of 6 to 11 compared with the current code. The performance of direct view buffers does not appear to be affected. No tests are modified or added as existing tests already cover these methods. All tests in CI tiers 1-3 passed on all platforms. ------------- Commit messages: - 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow Changes: https://git.openjdk.java.net/jdk/pull/3660/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3660&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264762 Stats: 298 lines in 2 files changed: 96 ins; 183 del; 19 mod Patch: https://git.openjdk.java.net/jdk/pull/3660.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3660/head:pull/3660 PR: https://git.openjdk.java.net/jdk/pull/3660 From yyang at openjdk.java.net Mon Apr 26 06:33:31 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Mon, 26 Apr 2021 06:33:31 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v2] In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 03:50:54 GMT, Yi Yang wrote: >> The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. >> >> In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. >> >> But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: >> >> 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. >> 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag >> >> Testing: cds, compiler and jdk > > Yi Yang has updated the pull request incrementally with one additional commit since the last revision: > > remove java_nio_Buffer in javaClasses.hpp Can I have a review of this PR? Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/3615 From github.com+782446+simon04 at openjdk.java.net Mon Apr 26 07:27:37 2021 From: github.com+782446+simon04 at openjdk.java.net (Simon Legner) Date: Mon, 26 Apr 2021 07:27:37 GMT Subject: Integrated: 8264031: (zipfs) Typo in ZipFileSystem.deleteFile ZipException In-Reply-To: References: Message-ID: <_a9xdZNG8ymj_gTaKocQDPfxfMb20cpbXJnkyOPVpqA=.0563fba7-c7fe-48f9-895b-e7f699c29f59@github.com> On Tue, 16 Feb 2021 13:48:20 GMT, Simon Legner wrote: > 8264031: (zipfs) Typo in ZipFileSystem.deleteFile ZipException This pull request has now been integrated. Changeset: 7b609a20 Author: Simon Legner Committer: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/7b609a20 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8264031: (zipfs) Typo in ZipFileSystem.deleteFile ZipException Reviewed-by: shade, lancea ------------- PR: https://git.openjdk.java.net/jdk/pull/2588 From alanb at openjdk.java.net Mon Apr 26 13:23:27 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Mon, 26 Apr 2021 13:23:27 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v2] In-Reply-To: References: Message-ID: <9-2H-KUkJbuLQ4wO0bSrKG4OgihZzAU30TryjlFi3jc=.bdd81481-05e6-473b-b837-2ae17fd19067@github.com> On Fri, 23 Apr 2021 03:50:54 GMT, Yi Yang wrote: >> The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. >> >> In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. >> >> But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: >> >> 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. >> 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag >> >> Testing: cds, compiler and jdk > > Yi Yang has updated the pull request incrementally with one additional commit since the last revision: > > remove java_nio_Buffer in javaClasses.hpp src/java.base/share/classes/java/nio/Buffer.java line 740: > 738: */ > 739: final int checkIndex(int i) { // package-private > 740: return Objects.checkIndex(i, limit); Changing Buffer.checkIndex to use Objects.checkIndex is okay. ------------- PR: https://git.openjdk.java.net/jdk/pull/3615 From alanb at openjdk.java.net Mon Apr 26 13:46:28 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Mon, 26 Apr 2021 13:46:28 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow In-Reply-To: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Message-ID: <7tADOsLtee0HtMXFcAG75PDPpvSx8GXLoL0JoJmypcY=.586eb233-30de-4184-af0a-96c5648bb133@github.com> On Fri, 23 Apr 2021 19:23:13 GMT, Brian Burkhalter wrote: > Please consider this request to accelerate absolute and relative bulk array transfer on views of heap byte buffers where the element size is greater than one. What currently happens is that the transfer devolves to a ?loopy? element-by-element copy such as > > int end = offset + length; > for (int i = offset, j = index; i < end; i++, j++) > dst[i] = get(j); > > for `get()`, and > > int end = offset + length; > for (int i = offset, j = index; i < end; i++, j++) > this.put(j, src[i]); > > for `put()`. This is of course relatively slow. > > The change proposed hoists the accelerated versions of these methods using the `ScopedMemoryAccess` methods `copyMemory()` and `copySwapMemory()` from `Direct-X-Buffer` to `X-Buffer`. The array bulk transfer methods are removed from `Direct-X-Buffer` itself. The number of lines of code in the templates decreases by 87. > > With this change the throughput of array bulk `put()` and `get()` for heap view buffers is increased by a factor of 6 to 11 compared with the current code. The performance of direct view buffers does not appear to be affected. > > No tests are modified or added as existing tests already cover these methods. All tests in CI tiers 1-3 passed on all platforms. The hoisting looks right but I think I need to checkout the branch and look at the generated code for the direct buffer and the one or two views to make sure. In passing, I see we have an inconsistency with the ordering of the index and whether is read-only buffer, we might want to adjust that at some point so that have consistency behavior when both conditions arise. ------------- PR: https://git.openjdk.java.net/jdk/pull/3660 From bpb at openjdk.java.net Mon Apr 26 17:24:27 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 26 Apr 2021 17:24:27 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow In-Reply-To: <7tADOsLtee0HtMXFcAG75PDPpvSx8GXLoL0JoJmypcY=.586eb233-30de-4184-af0a-96c5648bb133@github.com> References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> <7tADOsLtee0HtMXFcAG75PDPpvSx8GXLoL0JoJmypcY=.586eb233-30de-4184-af0a-96c5648bb133@github.com> Message-ID: On Mon, 26 Apr 2021 13:43:43 GMT, Alan Bateman wrote: > In passing, I see we have an inconsistency with the ordering of the index and whether is read-only buffer, we might want to adjust that at some point so that have consistency behavior when both conditions arise. I think you mean the ordering of the check on the indices and whether the buffer is read-only, correct? ------------- PR: https://git.openjdk.java.net/jdk/pull/3660 From psandoz at openjdk.java.net Mon Apr 26 17:25:06 2021 From: psandoz at openjdk.java.net (Paul Sandoz) Date: Mon, 26 Apr 2021 17:25:06 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow In-Reply-To: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Message-ID: On Fri, 23 Apr 2021 19:23:13 GMT, Brian Burkhalter wrote: > Please consider this request to accelerate absolute and relative bulk array transfer on views of heap byte buffers where the element size is greater than one. What currently happens is that the transfer devolves to a ?loopy? element-by-element copy such as > > int end = offset + length; > for (int i = offset, j = index; i < end; i++, j++) > dst[i] = get(j); > > for `get()`, and > > int end = offset + length; > for (int i = offset, j = index; i < end; i++, j++) > this.put(j, src[i]); > > for `put()`. This is of course relatively slow. > > The change proposed hoists the accelerated versions of these methods using the `ScopedMemoryAccess` methods `copyMemory()` and `copySwapMemory()` from `Direct-X-Buffer` to `X-Buffer`. The array bulk transfer methods are removed from `Direct-X-Buffer` itself. The number of lines of code in the templates decreases by 87. > > With this change the throughput of array bulk `put()` and `get()` for heap view buffers is increased by a factor of 6 to 11 compared with the current code. The performance of direct view buffers does not appear to be affected. > > No tests are modified or added as existing tests already cover these methods. All tests in CI tiers 1-3 passed on all platforms. src/java.base/share/classes/java/nio/X-Buffer.java.template line 275: > 273: > 274: // Number of bytes per $type$ > 275: private static final long ELEMENT_SIZE = 1L << $LG_BYTES_PER_VALUE$; IIRC you could use the constant `$Type$.BYTES` rather than declare a new static field. ------------- PR: https://git.openjdk.java.net/jdk/pull/3660 From bpb at openjdk.java.net Mon Apr 26 17:25:18 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 26 Apr 2021 17:25:18 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow In-Reply-To: References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Message-ID: On Mon, 26 Apr 2021 16:46:42 GMT, Paul Sandoz wrote: >> Please consider this request to accelerate absolute and relative bulk array transfer on views of heap byte buffers where the element size is greater than one. What currently happens is that the transfer devolves to a ?loopy? element-by-element copy such as >> >> int end = offset + length; >> for (int i = offset, j = index; i < end; i++, j++) >> dst[i] = get(j); >> >> for `get()`, and >> >> int end = offset + length; >> for (int i = offset, j = index; i < end; i++, j++) >> this.put(j, src[i]); >> >> for `put()`. This is of course relatively slow. >> >> The change proposed hoists the accelerated versions of these methods using the `ScopedMemoryAccess` methods `copyMemory()` and `copySwapMemory()` from `Direct-X-Buffer` to `X-Buffer`. The array bulk transfer methods are removed from `Direct-X-Buffer` itself. The number of lines of code in the templates decreases by 87. >> >> With this change the throughput of array bulk `put()` and `get()` for heap view buffers is increased by a factor of 6 to 11 compared with the current code. The performance of direct view buffers does not appear to be affected. >> >> No tests are modified or added as existing tests already cover these methods. All tests in CI tiers 1-3 passed on all platforms. > > src/java.base/share/classes/java/nio/X-Buffer.java.template line 275: > >> 273: >> 274: // Number of bytes per $type$ >> 275: private static final long ELEMENT_SIZE = 1L << $LG_BYTES_PER_VALUE$; > > IIRC you could use the constant `$Type$.BYTES` rather than declare a new static field. Yes, you are correct; will update. ------------- PR: https://git.openjdk.java.net/jdk/pull/3660 From kvn at openjdk.java.net Mon Apr 26 18:57:43 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Mon, 26 Apr 2021 18:57:43 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v2] In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 03:50:54 GMT, Yi Yang wrote: >> The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. >> >> In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. >> >> But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: >> >> 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. >> 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag >> >> Testing: cds, compiler and jdk > > Yi Yang has updated the pull request incrementally with one additional commit since the last revision: > > remove java_nio_Buffer in javaClasses.hpp @veresov please look on C1 changes. ------------- PR: https://git.openjdk.java.net/jdk/pull/3615 From bpb at openjdk.java.net Mon Apr 26 21:35:50 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 26 Apr 2021 21:35:50 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow [v2] In-Reply-To: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Message-ID: > Please consider this request to accelerate absolute and relative bulk array transfer on views of heap byte buffers where the element size is greater than one. What currently happens is that the transfer devolves to a ?loopy? element-by-element copy such as > > int end = offset + length; > for (int i = offset, j = index; i < end; i++, j++) > dst[i] = get(j); > > for `get()`, and > > int end = offset + length; > for (int i = offset, j = index; i < end; i++, j++) > this.put(j, src[i]); > > for `put()`. This is of course relatively slow. > > The change proposed hoists the accelerated versions of these methods using the `ScopedMemoryAccess` methods `copyMemory()` and `copySwapMemory()` from `Direct-X-Buffer` to `X-Buffer`. The array bulk transfer methods are removed from `Direct-X-Buffer` itself. The number of lines of code in the templates decreases by 87. > > With this change the throughput of array bulk `put()` and `get()` for heap view buffers is increased by a factor of 6 to 11 compared with the current code. The performance of direct view buffers does not appear to be affected. > > No tests are modified or added as existing tests already cover these methods. All tests in CI tiers 1-3 passed on all platforms. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8264762: In X-Buffer use $.BYTES instead of ELEMENT_SIZE; remove useless ReadOnlyBufferException from getArray(). ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3660/files - new: https://git.openjdk.java.net/jdk/pull/3660/files/de6e3447..719b8d84 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3660&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3660&range=00-01 Stats: 10 lines in 1 file changed: 0 ins; 7 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3660.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3660/head:pull/3660 PR: https://git.openjdk.java.net/jdk/pull/3660 From bpb at openjdk.java.net Mon Apr 26 21:41:36 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Mon, 26 Apr 2021 21:41:36 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow [v2] In-Reply-To: References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Message-ID: On Mon, 26 Apr 2021 21:35:50 GMT, Brian Burkhalter wrote: >> Please consider this request to accelerate absolute and relative bulk array transfer on views of heap byte buffers where the element size is greater than one. What currently happens is that the transfer devolves to a ?loopy? element-by-element copy such as >> >> int end = offset + length; >> for (int i = offset, j = index; i < end; i++, j++) >> dst[i] = get(j); >> >> for `get()`, and >> >> int end = offset + length; >> for (int i = offset, j = index; i < end; i++, j++) >> this.put(j, src[i]); >> >> for `put()`. This is of course relatively slow. >> >> The change proposed hoists the accelerated versions of these methods using the `ScopedMemoryAccess` methods `copyMemory()` and `copySwapMemory()` from `Direct-X-Buffer` to `X-Buffer`. The array bulk transfer methods are removed from `Direct-X-Buffer` itself. The number of lines of code in the templates decreases by 87. >> >> With this change the throughput of array bulk `put()` and `get()` for heap view buffers is increased by a factor of 6 to 11 compared with the current code. The performance of direct view buffers does not appear to be affected. >> >> No tests are modified or added as existing tests already cover these methods. All tests in CI tiers 1-3 passed on all platforms. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8264762: In X-Buffer use $.BYTES instead of ELEMENT_SIZE; remove useless ReadOnlyBufferException from getArray(). I verified that there is indeed inconsistency in the order of checks done in the absolute and relative bulk operations. The checking order is however the same before and after the commits in this request are applied. It might be better to defer any change in the order of checks to a separate issue. ------------- PR: https://git.openjdk.java.net/jdk/pull/3660 From alanb at openjdk.java.net Tue Apr 27 14:50:39 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 27 Apr 2021 14:50:39 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow [v2] In-Reply-To: References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Message-ID: On Mon, 26 Apr 2021 21:35:50 GMT, Brian Burkhalter wrote: >> Please consider this request to accelerate absolute and relative bulk array transfer on views of heap byte buffers where the element size is greater than one. What currently happens is that the transfer devolves to a ?loopy? element-by-element copy such as >> >> int end = offset + length; >> for (int i = offset, j = index; i < end; i++, j++) >> dst[i] = get(j); >> >> for `get()`, and >> >> int end = offset + length; >> for (int i = offset, j = index; i < end; i++, j++) >> this.put(j, src[i]); >> >> for `put()`. This is of course relatively slow. >> >> The change proposed hoists the accelerated versions of these methods using the `ScopedMemoryAccess` methods `copyMemory()` and `copySwapMemory()` from `Direct-X-Buffer` to `X-Buffer`. The array bulk transfer methods are removed from `Direct-X-Buffer` itself. The number of lines of code in the templates decreases by 87. >> >> With this change the throughput of array bulk `put()` and `get()` for heap view buffers is increased by a factor of 6 to 11 compared with the current code. The performance of direct view buffers does not appear to be affected. >> >> No tests are modified or added as existing tests already cover these methods. All tests in CI tiers 1-3 passed on all platforms. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8264762: In X-Buffer use $.BYTES instead of ELEMENT_SIZE; remove useless ReadOnlyBufferException from getArray(). Marked as reviewed by alanb (Reviewer). Okay, I can believe that inconsistencies have crept in. I think the changes look okay, I just wish the if-expression in the generated getArray and putArray were easier to read. This is really an issue with the preprocessor script of course. ------------- PR: https://git.openjdk.java.net/jdk/pull/3660 From psandoz at openjdk.java.net Tue Apr 27 15:22:38 2021 From: psandoz at openjdk.java.net (Paul Sandoz) Date: Tue, 27 Apr 2021 15:22:38 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow [v2] In-Reply-To: References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Message-ID: On Mon, 26 Apr 2021 21:35:50 GMT, Brian Burkhalter wrote: >> Please consider this request to accelerate absolute and relative bulk array transfer on views of heap byte buffers where the element size is greater than one. What currently happens is that the transfer devolves to a ?loopy? element-by-element copy such as >> >> int end = offset + length; >> for (int i = offset, j = index; i < end; i++, j++) >> dst[i] = get(j); >> >> for `get()`, and >> >> int end = offset + length; >> for (int i = offset, j = index; i < end; i++, j++) >> this.put(j, src[i]); >> >> for `put()`. This is of course relatively slow. >> >> The change proposed hoists the accelerated versions of these methods using the `ScopedMemoryAccess` methods `copyMemory()` and `copySwapMemory()` from `Direct-X-Buffer` to `X-Buffer`. The array bulk transfer methods are removed from `Direct-X-Buffer` itself. The number of lines of code in the templates decreases by 87. >> >> With this change the throughput of array bulk `put()` and `get()` for heap view buffers is increased by a factor of 6 to 11 compared with the current code. The performance of direct view buffers does not appear to be affected. >> >> No tests are modified or added as existing tests already cover these methods. All tests in CI tiers 1-3 passed on all platforms. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8264762: In X-Buffer use $.BYTES instead of ELEMENT_SIZE; remove useless ReadOnlyBufferException from getArray(). Marked as reviewed by psandoz (Reviewer). src/java.base/share/classes/java/nio/X-Buffer.java.template line 799: > 797: if (length > limit() - pos) > 798: throw new BufferUnderflowException(); > 799: int end = offset + length; Dead code? ------------- PR: https://git.openjdk.java.net/jdk/pull/3660 From bpb at openjdk.java.net Tue Apr 27 16:22:35 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 27 Apr 2021 16:22:35 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow [v2] In-Reply-To: References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Message-ID: On Tue, 27 Apr 2021 14:58:15 GMT, Paul Sandoz wrote: >> Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: >> >> 8264762: In X-Buffer use $.BYTES instead of ELEMENT_SIZE; remove useless ReadOnlyBufferException from getArray(). > > src/java.base/share/classes/java/nio/X-Buffer.java.template line 799: > >> 797: if (length > limit() - pos) >> 798: throw new BufferUnderflowException(); >> 799: int end = offset + length; > > Dead code? Yep. Cremated. ------------- PR: https://git.openjdk.java.net/jdk/pull/3660 From itakiguchi at openjdk.java.net Tue Apr 27 16:56:58 2021 From: itakiguchi at openjdk.java.net (Ichiroh Takiguchi) Date: Tue, 27 Apr 2021 16:56:58 GMT Subject: RFR: 8266013: Unexpected replacement character handling on stateful CharsetEncoder Message-ID: When an invalid character is converted by getBytes() method, the character is converted to replacement byte data. Shift code (SO/SI) may not be added into right place by EBCDIC Mix charset. EBCDIC Mix charset encoder is stateful encoder. Shift code should be added by switching character set. On x-IBM1364, "\u3000\uD800" should be converted to "\x0E\x40\x40\x0F\x6F", but "\x0E\x40\x40\x6F\x0F" SI is not in right place. Also ISO2022 related charsets use escape sequence to switch character set. But same kind of issue is there. ------------- Commit messages: - 8266013: Unexpected replacement character handling on stateful CharsetEncoder Changes: https://git.openjdk.java.net/jdk/pull/3719/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3719&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266013 Stats: 248 lines in 3 files changed: 228 ins; 12 del; 8 mod Patch: https://git.openjdk.java.net/jdk/pull/3719.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3719/head:pull/3719 PR: https://git.openjdk.java.net/jdk/pull/3719 From mcimadamore at openjdk.java.net Tue Apr 27 18:01:41 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Tue, 27 Apr 2021 18:01:41 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) Message-ID: This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. [1] - https://openjdk.java.net/jeps/412 ------------- Commit messages: - Add linker test excludes for x86 and other platforms which do not support `CLinker` - Integrate latest API tweaks - Fix x86 build - Fix `TestLinkToNativeRBP` - Initial push Changes: https://git.openjdk.java.net/jdk/pull/3699/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264774 Stats: 14083 lines in 213 files changed: 8502 ins; 3605 del; 1976 mod Patch: https://git.openjdk.java.net/jdk/pull/3699.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3699/head:pull/3699 PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Tue Apr 27 18:01:41 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Tue, 27 Apr 2021 18:01:41 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 17:10:13 GMT, Maurizio Cimadamore wrote: > This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/412 Here we list the main changes introduced in this PR. As usual, a big thank to all who helped along the way: @ChrisHegarty, @iwanowww, @JornVernee, @PaulSandoz and @sundararajana. ### Managing memory segments: `ResourceScope` This PR introduces a new abstraction (first discussed [here](https://inside.java/2021/01/25/memory-access-pulling-all-the-threads/)), namely `ResourceScope` which is used to manage the lifecycle of resources associated with off-heap memory (such as `MemorySegment`, `VaList`, etc). This is probably the biggest change in the API, as `MemorySegment` is no longer `AutoCloseable`: it instead features a *scope accessor* which can be used to access the memory segment's `ResourceScope`; the `ResourceScope` is the new `AutoCloseable`. In other words, code like this: try (MemorySegment segment = MemorySegment.allocateNative(100)) { ... } Now becomes like this: try (ResourceScope scope = ResourceScope.ofConfinedScope()) { MemorySegment segment = MemorySegment.allocateNative(100, scope); ... } While simple cases where only one segment is allocated become a little more verbose, this new API idiom obviously scales much better when multiple segments are created with the same lifecycle. Another important fact, which is captured by the name of the `ResourceScope` factory in the above snippet, is that segments no longer feature dynamic ownership changes. These were cool, but ultimately too expensive to support in the shared case. Instead, the API now requires clients to make a choice upfront (confined, shared or *implicit* - where the latter means GC-managed, like direct buffers). Implementation-wise, `ResourceScope` is implemented by a bunch of internal classes: `ResourceScopeImpl`, `ConfinedScope` and `SharedScope`. A resource scope impl has a so called *resource list* which can be also shared or confined. This is where cleanup actions are added; the resource list can be attached to a `Cleaner` to get implicit deallocation. There is a new test `TestResourceScope` to stress test the behavior of resource scopes, as well as a couple of microbenchmarks to assess the cost of creating/closing scopes (`ResourceScopeClose`) and acquiring/releasing them (`BulkMismatchAcquire`). ### IO operation on shared buffer views In the previous iteration of the Memory Access API we have introduced the concept of *shared* segments. Shared segments are as easy to use as confined ones, and they are as fast. One problem with shared segments was that it wasn't clear how to support IO operations on byte buffers derived from such segments: since the segment memory could be released at any time, there was simply no way to guarantee that a shared segment could not be closed in the middle of a (possibly async) IO operation. In this iteration, shared segments are just segments backed by a *shared resource scope*. The new API introduces way to manage the new complexity, in the form of two methods `ResourceScope::acquire` and `ResourceScope::release`, respectively, which can be used to *acquire* a resource scope. When a resource scope is in the acquired state, it cannot be closed (you can think of it as some slightly better and asymmetric form of an atomic reference counter). This means we are now finally in a position to add support for IO operations on all byte buffers, including those derived from shared segments. A big thank to @ChrisHegarty who lead the effort here. More info on this work are included in his [writeup](https://inside.java/2021/04/21/fma-and-nio-channels/). Most of the implementation for this feature occurs in the internal NIO packages; a new method on `Buffer` has been added to facilitate acquiring from NIO code - most of the logic associated with acquiring is in the `IOUtil` class. @ChrisHegarty has added many good tests for scoped IO operations under the `foreign/channels` folder, to check for all possible buffer/scope flavors. ### Allocating at speed: `SegmentAllocator` Another abstraction introduced in this JEP is that of `SegmentAllocator`. A segment allocator is a functional interface which can be used to tell other APIs (and, crucially, the `CLinker` API) how segments should be allocated, if the need arise. For instance, think about some code which turns a Java string into a C string. Such code will invariably: 1. allocate a memory segment off heap 2. bulk copy (where possible) the content of the Java string into the off-heap segment 3. add a NULL terminator So, in (1) such string conversion routine need to allocate a new off-heap segment; how is that done? Is that a call to malloc? Or something else? In the previous iteration of the Foreign Linker API, the method `CLinker::toCString` had two overloads: a simple version, only taking a Java string parameter; and a more advanced version taking a `NativeScope`. A `NativeScope` was, at its core, a custom segment allocator - but the allocation scheme was fixed in `NativeScope` as that class always acted as an arena-style allocator. `SegmentAllocator` is like `NativeScope` in spirit, in that it helps programs allocating segments - but it does so in a more general way than `NativeScope`, since a `SegmentAllocator` is not tied to any specific allocation strategy: in fact the strategy is left there to be defined by the user. As before, `SegmentAllocator` does provide some common factories, e.g. to create an arena allocator similar to `NativeScope` - but the `CLinker` API is now free to work with _any_ implementations of the `SegmentAllocator` interface. This generalization is crucial, given that, when operating with off-heap memory, allocation performance is often the bottleneck. Not only is `SegmentAllocator` accepted by all methods in the `CLinker` API which need to allocate memory: even the behavior of downcall method handle can now be affected by segment allocators: when linking a native function which returns a struct by value, the `CLinker` API will in fact need to dynamically allocate a segment to hold the result. In such cases, the method handle generated by `CLinker` will now accept an additional *prefix* parameter of type `SegmentAllocator` which tells `CLinker` *how* should memory be allocated for the result value. For instance, now clients can tell `CLinker` to return structs by value in *heap* segments, by using a `SegmentAllocator` which allocates memory on the heap; this might be useful if the segment is quickly discarded after use. There's not much implementation for `SegmentAllocator` as most of it is defined in terms of `default` methods in the interface itself. However we do have implementation classes for the arena allocation scheme (`ArenaAllocator.java`). We support confined allocation and shared allocation. The shared allocation achieves lock-free by using a `ThreadLocal` of confined arena allocators. `TestSegmentAllocators` is the test which checks most of the arena allocation flavors. ### `MemoryAddress` as scoped entities A natural consequence of introducing the `ResourceScope` abstraction is that now not only `MemorySegment` are associated with a scope, but even instances of `MemoryAddress` can be. This means extra safety, because passing addresses which are associated with a closed scope to a native function will issue an exception. As before, it is possible to have memory addresses which the runtime knows nothing about (those returned by native calls, or those created via `MemoryAddress::ofLong`); these addresses are simply associated with the so called *global scope* - meaning that they are not actively managed by the user and are considered to be "always alive" by the runtime (as before). Implementation-wise, you will now see that `MemoryAddressImpl` is no longer a pair of `Object`/`long`. It is now a pair of `MemorySegment`/`long`. The `MemorySegment`, if present, tells us which segment this address has been obtained from (and hence which scope is associated with the address). If null, if means that the address has no segment, and therefore is associated with the global scope. The `long` part acts as an offset into the segment (if segment is non-null), or as an absolute address. A new test `SafeFunctionAccessTest` attempts to call native functions with (closed) scoped addresses to see if exceptions are thrown. ### *Virtual* downcall method handles There are cases where the address of a downcall handle cannot be specified when a downcall method handle is linked, but can only be known subsequently, by doing more native calls. To better support these use cases, `CLinker` now provides a factory for downcall method handles which does *not* require any function entry point. Instead, such entry point will be provided *dynamically*, via an additional prefix parameter (of type `MemoryAddress`). Many thanks to @JornVernee who implemented this improvement. The implementation changes for this range from tweaking the Java ABI support (to make sure that the prefix argument is handled as expected), to low-level hotspot changes to parameterize the generated compiled stub to use the address (dynamic) parameter. Note that regular downcall method handles (the ones that are bound to an address) are now simply obtained by getting a "virtual" method handle, and inserting a `MemoryAddress` coordinate in the first argument position. `TestVirtualCalls` has been written explicitly to test dynamic passing of address parameters but, in reality, all existing downcall tests are stressing the new implementation logic (since, as said before, the old logic is expressed as an adaptation of the new virtual method handles). The benchmark we used to test downcall performances `CallOverhead` has now been split into two: `CallOverheadConstant` vs. `CallOverheadVirtual`. ### Optimized upcall support The previous iteration of the Foreign Linker API supported intrinsification of downcall handles, which allows calls to downcall method handles to perform as fast as a regular JNI call. The dual case, calling Java code from native code (upcalls) was left unoptimized. In this iteration, @JornVernee has added intrinsics support for upcalls as well as downcalls, based on some prior work from @iwanowww. As for downcalls, a lot of the adaptation now happens in Java code, before we jump into the target method handle. As for the code which calls such target handle, changes have been made so that the native code can jump to the optimized entry point (if one exists) for such method handle more directly. The performance improvements with this new approach are rather nice, with `CLinker` upcalls being 3x-4x faster compared with regular upcalls via JNI. Again, here we have changes in the guts of the Java ABI support, as we needed to adjust the method handle specialization logic to be able to work in two directions (both from Java to native and from native to Java). On the Hotspot front, the optimization changes are in `universalUpcallHandler_x86_64.cpp`. ### Accessing restricted methods It is still the case that some of the methods in the API are "restricted" and access to these methods is disabled by default. In previous iterations, access to such methods was granted by setting a JDK read-only runtime property: `-Dforeign.restricted=permit`. In this iteration we have refined the story for accessing restricted methods (thanks @sundararajana ), by introducing a new experimental command line option in the Java launcher, namely `--enable-native-access=`. This options accepts a list of modules (separated by commas), where a module name can also be `ALL-UNNAMED` (for the unnamed module). Adding this command line flag to the launcher has the effect of allowing access to restricted methods to a given set of modules (the list of modules specified in the command line option). Access to restricted methods from any other module not in the list is disallowed and will result in an `IllegalAccessException`. When implementing this flag we considered two options: adding some resolution-time checks in the JVM (e.g. in `linkResolver`); or use `@CallerSensitive` methods. In the end we opted for the latter given that `@CallerSensitive` are generally well understood and optimized, and the general feeling was that inventing another form of callsite-dependent check might have been unnecessarily risky, given that the same checks can be implemented in Java using `@CallerSensitive`. We plan (not in 17) to add `javadoc` support by means of an annotation (like we do for preview API methods) so that the text that is currently copied and pasted in all restricted methods can be inferred automagically by javadoc. ### GitHub testing status Most platforms build and tests pass. There are a bunch of *additional* Linux platforms which do not yet work correctly: * Zero * arm * ppc * s390 The first two can be addresses easily by stubbing out few functions (I'll do that shortly). The last two are harder, as this patch moves some static functions (e.g. `long_move`, `float_move`) up to `SharedRuntime`; unfortunately, while most platforms use the same signatures for these function, on ppc and s390 that's not the case and function with same name, but incompatible signatures are defined there, leading to build issues. We will try to tweak the code around this, to make sure that these platforms remain buildable. Javadoc: http://cr.openjdk.java.net/~mcimadamore/JEP-412/v1/javadoc/jdk/incubator/foreign/package-summary.html Specdiff: http://cr.openjdk.java.net/~mcimadamore/JEP-412/v1/specdiff/overview-summary.html ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From alanb at openjdk.java.net Tue Apr 27 18:51:38 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 27 Apr 2021 18:51:38 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) In-Reply-To: References: Message-ID: <6SevcbQtVmLuygupdqgGyTXI943L_DitH0AK5r-Ou40=.d2ea73a1-0f30-46f2-b2d6-8f59db9b8a83@github.com> On Mon, 26 Apr 2021 17:10:13 GMT, Maurizio Cimadamore wrote: > This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/412 src/java.base/share/classes/java/lang/Module.java line 253: > 251: } > 252: > 253: boolean isEnableNativeAccess() { Would it be possible change isEnableNativeAccess and addNEnableNativeAccess to keep them consistent with the existing methods in this file. src/java.base/share/classes/jdk/internal/access/JavaLangAccess.java line 384: > 382: Module addEnableNativeAccess(Module m); > 383: > 384: boolean isEnableNativeAccess(Module m); Probably better to co-locate these with the other module methods, and add a comment so that it's consistent with the existing methods. src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java line 272: > 270: BootLoader.loadModule(base); > 271: SharedSecrets.getJavaLangAccess() > 272: .addEnableNativeAccess(Modules.defineModule(null, base.descriptor(), baseUri)); This would be cleaner if you replace it with: Module baseModule = Modules.defineModule(null, base.descriptor(), baseUri); JLA.addEnableNativeAccess(baseModule); You can use JLA in addEnableNativeAccess too, no need for "jla". src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java line 876: > 874: } > 875: > 876: private static void addEnableNativeAccess(ModuleLayer layer) { It would be useful to add a method description in the same style as the existing methods, just to keep these methods consistent. src/java.base/share/classes/sun/nio/ch/IOUtil.java line 57: > 55: > 56: static int write(FileDescriptor fd, ByteBuffer src, long position, > 57: boolean async, NativeDispatcher nd) UnixAsynchronousSocketChannel is an outlier user of IOUtil and I think we need to see if we can avoid extending this class to async if possible. src/java.base/share/classes/sun/nio/ch/IOUtil.java line 466: > 464: } > 465: > 466: private static final JavaNioAccess NIO_ACCESS = SharedSecrets.getJavaNioAccess(); It might be cleaner to move to acquire/release methods to their own supporting class as it's not really IOUtil. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From bpb at openjdk.java.net Tue Apr 27 19:00:58 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Tue, 27 Apr 2021 19:00:58 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow [v3] In-Reply-To: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Message-ID: > Please consider this request to accelerate absolute and relative bulk array transfer on views of heap byte buffers where the element size is greater than one. What currently happens is that the transfer devolves to a ?loopy? element-by-element copy such as > > int end = offset + length; > for (int i = offset, j = index; i < end; i++, j++) > dst[i] = get(j); > > for `get()`, and > > int end = offset + length; > for (int i = offset, j = index; i < end; i++, j++) > this.put(j, src[i]); > > for `put()`. This is of course relatively slow. > > The change proposed hoists the accelerated versions of these methods using the `ScopedMemoryAccess` methods `copyMemory()` and `copySwapMemory()` from `Direct-X-Buffer` to `X-Buffer`. The array bulk transfer methods are removed from `Direct-X-Buffer` itself. The number of lines of code in the templates decreases by 87. > > With this change the throughput of array bulk `put()` and `get()` for heap view buffers is increased by a factor of 6 to 11 compared with the current code. The performance of direct view buffers does not appear to be affected. > > No tests are modified or added as existing tests already cover these methods. All tests in CI tiers 1-3 passed on all platforms. Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: 8264762: Remove dead code at X-Buffer:799 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3660/files - new: https://git.openjdk.java.net/jdk/pull/3660/files/719b8d84..bfd374b7 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3660&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3660&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3660.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3660/head:pull/3660 PR: https://git.openjdk.java.net/jdk/pull/3660 From mcimadamore at openjdk.java.net Tue Apr 27 20:23:10 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Tue, 27 Apr 2021 20:23:10 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) In-Reply-To: <6SevcbQtVmLuygupdqgGyTXI943L_DitH0AK5r-Ou40=.d2ea73a1-0f30-46f2-b2d6-8f59db9b8a83@github.com> References: <6SevcbQtVmLuygupdqgGyTXI943L_DitH0AK5r-Ou40=.d2ea73a1-0f30-46f2-b2d6-8f59db9b8a83@github.com> Message-ID: On Tue, 27 Apr 2021 18:46:02 GMT, Alan Bateman wrote: >> This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/412 > > src/java.base/share/classes/java/lang/Module.java line 253: > >> 251: } >> 252: >> 253: boolean isEnableNativeAccess() { > > Would it be possible change isEnableNativeAccess and addNEnableNativeAccess to keep them consistent with the existing methods in this file. Change as in "adding javadoc" ? Or is the naming of the method something that's bothering? > src/java.base/share/classes/jdk/internal/access/JavaLangAccess.java line 384: > >> 382: Module addEnableNativeAccess(Module m); >> 383: >> 384: boolean isEnableNativeAccess(Module m); > > Probably better to co-locate these with the other module methods, and add a comment so that it's consistent with the existing methods. sure I'll fix this > src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java line 272: > >> 270: BootLoader.loadModule(base); >> 271: SharedSecrets.getJavaLangAccess() >> 272: .addEnableNativeAccess(Modules.defineModule(null, base.descriptor(), baseUri)); > > This would be cleaner if you replace it with: > > Module baseModule = Modules.defineModule(null, base.descriptor(), baseUri); > JLA.addEnableNativeAccess(baseModule); > > You can use JLA in addEnableNativeAccess too, no need for "jla". ok > src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java line 876: > >> 874: } >> 875: >> 876: private static void addEnableNativeAccess(ModuleLayer layer) { > > It would be useful to add a method description in the same style as the existing methods, just to keep these methods consistent. I'll fix ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From psandoz at openjdk.java.net Wed Apr 28 00:17:54 2021 From: psandoz at openjdk.java.net (Paul Sandoz) Date: Wed, 28 Apr 2021 00:17:54 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 17:10:13 GMT, Maurizio Cimadamore wrote: > This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/412 The Java changes are mostly familiar to me, having reviewed many PRs in the Panama repository, hence the lack of specific comments here. I did a pass through the changes and nothing major stood out to me. The new API changes make it much easy to build APIs around memory segments, exposing and reusing resource scope, acquiring/releasing on resource scope, where necessary. ------------- Marked as reviewed by psandoz (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3699 From chegar at openjdk.java.net Wed Apr 28 08:22:52 2021 From: chegar at openjdk.java.net (Chris Hegarty) Date: Wed, 28 Apr 2021 08:22:52 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) In-Reply-To: <6SevcbQtVmLuygupdqgGyTXI943L_DitH0AK5r-Ou40=.d2ea73a1-0f30-46f2-b2d6-8f59db9b8a83@github.com> References: <6SevcbQtVmLuygupdqgGyTXI943L_DitH0AK5r-Ou40=.d2ea73a1-0f30-46f2-b2d6-8f59db9b8a83@github.com> Message-ID: On Tue, 27 Apr 2021 18:40:24 GMT, Alan Bateman wrote: >> This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/412 > > src/java.base/share/classes/sun/nio/ch/IOUtil.java line 466: > >> 464: } >> 465: >> 466: private static final JavaNioAccess NIO_ACCESS = SharedSecrets.getJavaNioAccess(); > > It might be cleaner to move to acquire/release methods to their own supporting class as it's not really IOUtil. I went back and forth on this a number of times already. I think where we landed is a reasonable place, given the current shape of the code. Scope is a private property of Buffer, and as such should be consulted anywhere where a buffer's address is being accessed. In fact, a prior prototype would not allow access to the underlying address value without the caller passing a valid handle for the buffer view's scope. It's hard to find the sweet-spot here between code reuse and safety, but the high-order bit is that the code accessing the address is auditable and testable to avoid accessing memory unsafely. Maybe there is a better alternative implementation code structure (at the cost of some duplication), but it is not obvious to me what that is (and I have given it some thought). Suggestions welcome. Note, there is a little more follow-on work to be done in this area, if we are to expand support to other non-TCP channel implementations. Maybe investigation into possible code refactorings could be done as part of that? ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From chegar at openjdk.java.net Wed Apr 28 09:14:53 2021 From: chegar at openjdk.java.net (Chris Hegarty) Date: Wed, 28 Apr 2021 09:14:53 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 17:10:13 GMT, Maurizio Cimadamore wrote: > This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/412 src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ResourceScope.java line 135: > 133: } finally { > 134: segment.scope().release(segmentHandle); > 135: } I do like the symmetry in this example, but just to add an alternative idiom: `segmentHandle.scope().release(segmentHandle)` , which guarantees to avoid release throwing and IAE src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ResourceScope.java line 186: > 184: *
  • this resource scope is confined, and this method is called from a thread other than the thread owning this resource scope
  • > 185: *
  • this resource scope is shared and a resource associated with this scope is accessed while this method is called
  • > 186: *
  • one or more handles (see {@link #acquire()}) associated with this resource scope have not been closed
  • Minor spec suggestion: "... associated with this resource scope have not been *released*" src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ResourceScope.java line 205: > 203: * until all the resource scope handles acquired from it have been {@link #release(Handle)} released}. > 204: * @return a resource scope handle. > 205: */ Given recent changes, it might be good to generalise the opening sentence here. Maybe : " Acquires a resource scope handle associated with this resource scope. If the resource scope is explicit ... " Or some such. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 10:20:53 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 10:20:53 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 09:06:29 GMT, Chris Hegarty wrote: >> This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/412 > > src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ResourceScope.java line 205: > >> 203: * until all the resource scope handles acquired from it have been {@link #release(Handle)} released}. >> 204: * @return a resource scope handle. >> 205: */ > > Given recent changes, it might be good to generalise the opening sentence here. Maybe : > " Acquires a resource scope handle associated with this resource scope. If the resource scope is explicit ... " Or some such. I'm afraid I don't get this comment ;-) ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 10:42:55 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 10:42:55 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 09:10:37 GMT, Chris Hegarty wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> Address first batch of review comments > > src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ResourceScope.java line 135: > >> 133: } finally { >> 134: segment.scope().release(segmentHandle); >> 135: } > > I do like the symmetry in this example, but just to add an alternative idiom: > `segmentHandle.scope().release(segmentHandle)` > , which guarantees to avoid release throwing and IAE I see what you mean - I don't think I want to encourage that style too much by giving it prominence in the javadoc. It's true you can go back to the scope from the handle, so that you are guaranteed to release the right thing, but I think that should be unnecessary in most idiomatic uses. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 10:42:54 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 10:42:54 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/412 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Address first batch of review comments ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3699/files - new: https://git.openjdk.java.net/jdk/pull/3699/files/7545f71f..a80d8180 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=00-01 Stats: 42 lines in 4 files changed: 25 ins; 11 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/3699.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3699/head:pull/3699 PR: https://git.openjdk.java.net/jdk/pull/3699 From chegar at openjdk.java.net Wed Apr 28 10:42:56 2021 From: chegar at openjdk.java.net (Chris Hegarty) Date: Wed, 28 Apr 2021 10:42:56 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 09:06:29 GMT, Chris Hegarty wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> Address first batch of review comments > > src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ResourceScope.java line 205: > >> 203: * until all the resource scope handles acquired from it have been {@link #release(Handle)} released}. >> 204: * @return a resource scope handle. >> 205: */ > > Given recent changes, it might be good to generalise the opening sentence here. Maybe : > " Acquires a resource scope handle associated with this resource scope. If the resource scope is explicit ... " Or some such. The spec for the acquire method talks only of explicit resource scopes. The comment is suggesting that the spec could be generalised a little, since it is possible to acquire a resource scope handle associated with implicit scopes. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From chegar at openjdk.java.net Wed Apr 28 13:12:55 2021 From: chegar at openjdk.java.net (Chris Hegarty) Date: Wed, 28 Apr 2021 13:12:55 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 10:42:54 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/412 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Address first batch of review comments Like Paul, I tracked the changes to this API in the Panama repo. Most of my remaining comments are small and come from re-reading the API docs. src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/CLinker.java line 270: > 268: > 269: /** > 270: * Converts a Java string into a null-terminated C string, using the platform's default charset, Sorry if this has come up before, but, is the platform's default charset the right choice here? For other areas, we choose UTF-8 as the default. In fact, there is a draft JEP to move the default charset to UTF-8. So if there is an implicit need to match the underlying platform's charset then this may need to be revisited. For now, I just want to check that this is not an accidental reliance on the platform's default charset, but a deliberate one. src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 101: > 99: *

    Lifecycle and confinement

    > 100: * > 101: * Memory segments are associated to a resource scope (see {@link ResourceScope}), which can be accessed using Typo?? "Memory segments are associated *with* a resource scope" src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 112: > 110: MemoryAccess.getLong(segment); // already closed! > 111: * } > 112: * Additionally, access to a memory segment in subject to the thread-confinement checks enforced by the owning scope; that is, Typo?? "Additionally, access to a memory segment *is* subject to ..." src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 114: > 112: * Additionally, access to a memory segment in subject to the thread-confinement checks enforced by the owning scope; that is, > 113: * if the segment is associated with a shared scope, it can be accessed by multiple threads; if it is associated with a confined > 114: * scope, it can only be accessed by the thread which own the scope. Typo?? "it can only be accessed by the thread which ownS the scope." src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 693: > 691: */ > 692: static MemorySegment allocateNative(MemoryLayout layout, ResourceScope scope) { > 693: Objects.requireNonNull(scope); Should the allocateNative methods declare that they throw ISE, if the given ResourceScope is not alive? ( I found myself asking this q, then considering the behaviour of a SegmentAllocator that is asked to allocate after a RS has been closed ) ------------- Marked as reviewed by chegar (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 13:45:53 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 13:45:53 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 13:08:26 GMT, Chris Hegarty wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> Address first batch of review comments > > src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 693: > >> 691: */ >> 692: static MemorySegment allocateNative(MemoryLayout layout, ResourceScope scope) { >> 693: Objects.requireNonNull(scope); > > Should the allocateNative methods declare that they throw ISE, if the given ResourceScope is not alive? ( I found myself asking this q, then considering the behaviour of a SegmentAllocator that is asked to allocate after a RS has been closed ) Good point, yes it should ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 13:50:55 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 13:50:55 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 12:47:36 GMT, Chris Hegarty wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> Address first batch of review comments > > src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/CLinker.java line 270: > >> 268: >> 269: /** >> 270: * Converts a Java string into a null-terminated C string, using the platform's default charset, > > Sorry if this has come up before, but, is the platform's default charset the right choice here? For other areas, we choose UTF-8 as the default. In fact, there is a draft JEP to move the default charset to UTF-8. So if there is an implicit need to match the underlying platform's charset then this may need to be revisited. For now, I just want to check that this is not an accidental reliance on the platform's default charset, but a deliberate one. I believe here the goal is to be consistent with `String::getBytes`: /** * Encodes this {@code String} into a sequence of bytes using the * platform's default charset, storing the result into a new byte array. * *

    The behavior of this method when this string cannot be encoded in * the default charset is unspecified. The {@link * java.nio.charset.CharsetEncoder} class should be used when more control * over the encoding process is required. * * @return The resultant byte array * * @since 1.1 */ public byte[] getBytes() { return encode(Charset.defaultCharset(), coder(), value); } So, you are right in that there's an element of platform-dependency here - but I think it's a dependency that users learned to "ignore" mostly. If developers want to be precise, and platform independent, they can always use the Charset-accepting method. Of course this could be revisited, but I think there is some consistency in what the API is trying to do. If, in the future, defaultCharset will just be Utf8 - well, that's ok too - as long as the method is specified to be "defaultCharset" dependent, what's behind "defaultCharset" is an implementation detail that the user will have to be aware of one way or another. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From alanb at openjdk.java.net Wed Apr 28 14:44:54 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 28 Apr 2021 14:44:54 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow [v3] In-Reply-To: References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Message-ID: On Tue, 27 Apr 2021 19:00:58 GMT, Brian Burkhalter wrote: >> Please consider this request to accelerate absolute and relative bulk array transfer on views of heap byte buffers where the element size is greater than one. What currently happens is that the transfer devolves to a ?loopy? element-by-element copy such as >> >> int end = offset + length; >> for (int i = offset, j = index; i < end; i++, j++) >> dst[i] = get(j); >> >> for `get()`, and >> >> int end = offset + length; >> for (int i = offset, j = index; i < end; i++, j++) >> this.put(j, src[i]); >> >> for `put()`. This is of course relatively slow. >> >> The change proposed hoists the accelerated versions of these methods using the `ScopedMemoryAccess` methods `copyMemory()` and `copySwapMemory()` from `Direct-X-Buffer` to `X-Buffer`. The array bulk transfer methods are removed from `Direct-X-Buffer` itself. The number of lines of code in the templates decreases by 87. >> >> With this change the throughput of array bulk `put()` and `get()` for heap view buffers is increased by a factor of 6 to 11 compared with the current code. The performance of direct view buffers does not appear to be affected. >> >> No tests are modified or added as existing tests already cover these methods. All tests in CI tiers 1-3 passed on all platforms. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8264762: Remove dead code at X-Buffer:799 Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3660 From chegar at openjdk.java.net Wed Apr 28 15:15:51 2021 From: chegar at openjdk.java.net (Chris Hegarty) Date: Wed, 28 Apr 2021 15:15:51 GMT Subject: RFR: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow [v3] In-Reply-To: References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Message-ID: On Tue, 27 Apr 2021 19:00:58 GMT, Brian Burkhalter wrote: >> Please consider this request to accelerate absolute and relative bulk array transfer on views of heap byte buffers where the element size is greater than one. What currently happens is that the transfer devolves to a ?loopy? element-by-element copy such as >> >> int end = offset + length; >> for (int i = offset, j = index; i < end; i++, j++) >> dst[i] = get(j); >> >> for `get()`, and >> >> int end = offset + length; >> for (int i = offset, j = index; i < end; i++, j++) >> this.put(j, src[i]); >> >> for `put()`. This is of course relatively slow. >> >> The change proposed hoists the accelerated versions of these methods using the `ScopedMemoryAccess` methods `copyMemory()` and `copySwapMemory()` from `Direct-X-Buffer` to `X-Buffer`. The array bulk transfer methods are removed from `Direct-X-Buffer` itself. The number of lines of code in the templates decreases by 87. >> >> With this change the throughput of array bulk `put()` and `get()` for heap view buffers is increased by a factor of 6 to 11 compared with the current code. The performance of direct view buffers does not appear to be affected. >> >> No tests are modified or added as existing tests already cover these methods. All tests in CI tiers 1-3 passed on all platforms. > > Brian Burkhalter has updated the pull request incrementally with one additional commit since the last revision: > > 8264762: Remove dead code at X-Buffer:799 Marked as reviewed by chegar (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3660 From alanb at openjdk.java.net Wed Apr 28 16:17:53 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 28 Apr 2021 16:17:53 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: <0P9z_Fj6-ogctJUqlsZRC3JfOXGCbx0YLNh8me9grio=.0d53fd2d-d6b0-4c69-9da1-4781a1194405@github.com> On Wed, 28 Apr 2021 13:47:43 GMT, Maurizio Cimadamore wrote: >> src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/CLinker.java line 270: >> >>> 268: >>> 269: /** >>> 270: * Converts a Java string into a null-terminated C string, using the platform's default charset, >> >> Sorry if this has come up before, but, is the platform's default charset the right choice here? For other areas, we choose UTF-8 as the default. In fact, there is a draft JEP to move the default charset to UTF-8. So if there is an implicit need to match the underlying platform's charset then this may need to be revisited. For now, I just want to check that this is not an accidental reliance on the platform's default charset, but a deliberate one. > > I believe here the goal is to be consistent with `String::getBytes`: > > > /** > * Encodes this {@code String} into a sequence of bytes using the > * platform's default charset, storing the result into a new byte array. > * > *

    The behavior of this method when this string cannot be encoded in > * the default charset is unspecified. The {@link > * java.nio.charset.CharsetEncoder} class should be used when more control > * over the encoding process is required. > * > * @return The resultant byte array > * > * @since 1.1 > */ > public byte[] getBytes() { > return encode(Charset.defaultCharset(), coder(), value); > } > > > So, you are right in that there's an element of platform-dependency here - but I think it's a dependency that users learned to "ignore" mostly. If developers want to be precise, and platform independent, they can always use the Charset-accepting method. Of course this could be revisited, but I think there is some consistency in what the API is trying to do. If, in the future, defaultCharset will just be Utf8 - well, that's ok too - as long as the method is specified to be "defaultCharset" dependent, what's behind "defaultCharset" is an implementation detail that the user will have to be aware of one way or another. Naoto is working on a couple of changes in advance of JEP 400. One of these is to expose a system property with the host charset and I suspect that the CLinker method will want to use that instead of Charset.defaultCharset. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From dfuchs at openjdk.java.net Wed Apr 28 17:28:53 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Wed, 28 Apr 2021 17:28:53 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: <689o3jJsku_BVrUSQATPla7V0BfqOYZuzlJXtmy347E=.6871722d-66a7-4364-8e19-3eecb6df2e3a@github.com> On Wed, 28 Apr 2021 10:42:54 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/412 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Address first batch of review comments src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/package-info.java line 224: > 222: * Some methods in this package are considered restricted. Restricted methods are typically used to bind native > 223: * foreign data and/or functions to first-class Java API elements which can then be used directly by client. For instance > 224: * the restricted method {@link jdk.incubator.foreign.MemoryAddress#asSegment(long, ResourceScope)} can be used to create typo: `used directly by client.` => `used directly by clients.` ? ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From iveresov at openjdk.java.net Wed Apr 28 17:35:55 2021 From: iveresov at openjdk.java.net (Igor Veresov) Date: Wed, 28 Apr 2021 17:35:55 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v2] In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 03:50:54 GMT, Yi Yang wrote: >> The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. >> >> In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. >> >> But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: >> >> 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. >> 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag >> >> Testing: cds, compiler and jdk > > Yi Yang has updated the pull request incrementally with one additional commit since the last revision: > > remove java_nio_Buffer in javaClasses.hpp src/hotspot/share/c1/c1_GraphBuilder.cpp line 2052: > 2050: return; > 2051: } > 2052: Do we need to keep this flag? ------------- PR: https://git.openjdk.java.net/jdk/pull/3615 From mchung at openjdk.java.net Wed Apr 28 18:04:53 2021 From: mchung at openjdk.java.net (Mandy Chung) Date: Wed, 28 Apr 2021 18:04:53 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 10:42:54 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/412 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Address first batch of review comments I reviewed the `--enable-native-access` related change that looks fine. > Access to restricted methods from any other module not in the list is disallowed and will result in an IllegalAccessException. I think you meant to say `IllegalCallerException` instead of `IllegalAccessException`. Also do you intend to have javadoc to generate `@throw IllegalCallerException` for the restricted methods automatically besides the javadoc description? Making the restricted methods as `@CallerSensitive` in order to get the caller class for native access check is the proper approach. However, some interface methods are restricted methods such as `CLinker::downcallHandle` whose the implementation method is `@CallerSensitive`. I concern with the security issue with method handle and type aliasing. On the other hand, `CLinker` is a sealed interface and only implemented by the platform and so it's less of a concern. I think the interface method should also be `@CallerSensitive` so that for example a method handle for `CLinker::downcallHandle` will be produced with the proper caller-sensitive context. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From dfuchs at openjdk.java.net Wed Apr 28 18:10:54 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Wed, 28 Apr 2021 18:10:54 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: <9aIvV_N89PFpzwfhedEwNju-eKqhpZoFHL59F5pUv6g=.9c515555-c446-4280-8f24-88ffd753712f@github.com> On Wed, 28 Apr 2021 10:42:54 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/412 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Address first batch of review comments src/java.base/share/classes/jdk/internal/module/IllegalNativeAccessChecker.java line 40: > 38: > 39: private IllegalNativeAccessChecker(Set allowedModuleNames, boolean allowAllUnnamedModules) { > 40: this.allowedModuleNames = Collections.unmodifiableSet(allowedModuleNames); Should that be Set.copyOf() to take advantage of the immutability of `SetN` (but at the expense of additional copying)... src/java.base/share/classes/jdk/internal/module/IllegalNativeAccessChecker.java line 78: > 76: int index = 0; > 77: // the system property is removed after decoding > 78: String value = getAndRemoveProperty(prefix + index); I am not sure what is going on with the removal of the properties, but if I'm not mistaken this is racy: from the implementation of the checker() method above, it looks as if two different threads could trigger a call to the decode() function concurrently, which can result in a random partitioning of the properties against the two checkers being instantiated, with one of them being eventually set as the system-wide checker. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 18:17:59 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 18:17:59 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 18:02:03 GMT, Mandy Chung wrote: > I reviewed the `--enable-native-access` related change that looks fine. > > > Access to restricted methods from any other module not in the list is disallowed and will result in an IllegalAccessException. > > I think you meant to say `IllegalCallerException` instead of `IllegalAccessException`. Also do you intend to have javadoc to generate `@throw IllegalCallerException` for the restricted methods automatically besides the javadoc description? > IllegalCalller is probably better yes - we started off with an access-like check, so things have evolved a bit. I'll also add the @throws. > Making the restricted methods as `@CallerSensitive` in order to get the caller class for native access check is the proper approach. However, some interface methods are restricted methods such as `CLinker::downcallHandle` whose the implementation method is `@CallerSensitive`. I concern with the security issue with method handle and type aliasing. On the other hand, `CLinker` is a sealed interface and only implemented by the platform and so it's less of a concern. I think the interface method should also be `@CallerSensitive` so that for example a method handle for `CLinker::downcallHandle` will be produced with the proper caller-sensitive context. I believe that we had to move @CallerSensitive out of interfaces because there was a test that was checking that @CS was not put on "virtual" methods. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 18:21:54 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 18:21:54 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: <9aIvV_N89PFpzwfhedEwNju-eKqhpZoFHL59F5pUv6g=.9c515555-c446-4280-8f24-88ffd753712f@github.com> References: <9aIvV_N89PFpzwfhedEwNju-eKqhpZoFHL59F5pUv6g=.9c515555-c446-4280-8f24-88ffd753712f@github.com> Message-ID: On Wed, 28 Apr 2021 18:07:32 GMT, Daniel Fuchs wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> Address first batch of review comments > > src/java.base/share/classes/jdk/internal/module/IllegalNativeAccessChecker.java line 78: > >> 76: int index = 0; >> 77: // the system property is removed after decoding >> 78: String value = getAndRemoveProperty(prefix + index); > > I am not sure what is going on with the removal of the properties, but if I'm not mistaken this is racy: from the implementation of the checker() method above, it looks as if two different threads could trigger a call to the decode() function concurrently, which can result in a random partitioning of the properties against the two checkers being instantiated, with one of them being eventually set as the system-wide checker. I think the method is called during module bootstrap - I don't think there is a race in practice. This method is also called in other parts of ModuleBootstrap. The code you allude to is called during initialization of the IllegalNativeAccessChecker singleton, which should happen only once, and only from one thread. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mchung at openjdk.java.net Wed Apr 28 18:34:23 2021 From: mchung at openjdk.java.net (Mandy Chung) Date: Wed, 28 Apr 2021 18:34:23 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 10:42:54 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/412 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Address first batch of review comments > I believe that we had to move @CallerSensitive out of interfaces because there was a test that was checking that @cs was not put on "virtual" methods. Good if we do have such test. We need to re-examine this with the security team. I suggest to create a JBS issue and follow up separately. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 18:33:36 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 18:33:36 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v3] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/412 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Address review comments: * fix typos in javadoc * document ISE being thrown in all methods accepting a scope; add more tests for that ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3699/files - new: https://git.openjdk.java.net/jdk/pull/3699/files/a80d8180..24e554c5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=01-02 Stats: 112 lines in 6 files changed: 50 ins; 0 del; 62 mod Patch: https://git.openjdk.java.net/jdk/pull/3699.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3699/head:pull/3699 PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 18:35:54 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 18:35:54 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: <9aIvV_N89PFpzwfhedEwNju-eKqhpZoFHL59F5pUv6g=.9c515555-c446-4280-8f24-88ffd753712f@github.com> References: <9aIvV_N89PFpzwfhedEwNju-eKqhpZoFHL59F5pUv6g=.9c515555-c446-4280-8f24-88ffd753712f@github.com> Message-ID: <9u5yskXth55699_itCsdggIyKauvUauftz9zy0svpLI=.744431e1-4a6d-4fe2-a664-09b35e53eaf7@github.com> On Wed, 28 Apr 2021 17:53:44 GMT, Daniel Fuchs wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> Address first batch of review comments > > src/java.base/share/classes/jdk/internal/module/IllegalNativeAccessChecker.java line 40: > >> 38: >> 39: private IllegalNativeAccessChecker(Set allowedModuleNames, boolean allowAllUnnamedModules) { >> 40: this.allowedModuleNames = Collections.unmodifiableSet(allowedModuleNames); > > Should that be Set.copyOf() to take advantage of the immutability of `SetN` (but at the expense of additional copying)... Honestly, I'm not even sure why we should be concerned about mutation of the set here - since we create this with a bunch of values read from a system property... e.g. should we just store `allowedModuleNames` period? ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 18:34:41 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 18:34:41 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 18:23:41 GMT, Mandy Chung wrote: > > I believe that we had to move @CallerSensitive out of interfaces because there was a test that was checking that @cs was not put on "virtual" methods. > > Good if we do have such test. We need to re-examine this with the security team. I suggest to create a JBS issue and follow up separately. The test in question is: test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mchung at openjdk.java.net Wed Apr 28 18:38:28 2021 From: mchung at openjdk.java.net (Mandy Chung) Date: Wed, 28 Apr 2021 18:38:28 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 18:26:30 GMT, Maurizio Cimadamore wrote: > test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java Yes that's the test. That test misses to catch your case where aliasing may be possible. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From dfuchs at openjdk.java.net Wed Apr 28 18:47:08 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Wed, 28 Apr 2021 18:47:08 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: <9aIvV_N89PFpzwfhedEwNju-eKqhpZoFHL59F5pUv6g=.9c515555-c446-4280-8f24-88ffd753712f@github.com> Message-ID: On Wed, 28 Apr 2021 18:19:14 GMT, Maurizio Cimadamore wrote: >> src/java.base/share/classes/jdk/internal/module/IllegalNativeAccessChecker.java line 78: >> >>> 76: int index = 0; >>> 77: // the system property is removed after decoding >>> 78: String value = getAndRemoveProperty(prefix + index); >> >> I am not sure what is going on with the removal of the properties, but if I'm not mistaken this is racy: from the implementation of the checker() method above, it looks as if two different threads could trigger a call to the decode() function concurrently, which can result in a random partitioning of the properties against the two checkers being instantiated, with one of them being eventually set as the system-wide checker. > > I think the method is called during module bootstrap - I don't think there is a race in practice. This method is also called in other parts of ModuleBootstrap. The code you allude to is called during initialization of the IllegalNativeAccessChecker singleton, which should happen only once, and only from one thread. I'll take your word for it - the use of a volatile variable to store the singleton instance made this suspicious. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From dfuchs at openjdk.java.net Wed Apr 28 19:09:54 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Wed, 28 Apr 2021 19:09:54 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 10:42:54 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/412 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Address first batch of review comments src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/LibraryLookup.java line 52: > 50: *

    > 51: * For {@link #lookup(String) memory addresses} obtained from a library lookup object, > 52: * since {@link CLinker#downcallHandle(Addressable, MethodType, FunctionDescriptor) native method handles} These should be `{@linkplain }` since the text of the link is plain text (and not code) src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/LibraryLookup.java line 88: > 86: * @return the memory segment associated with the library symbol (if any). > 87: * @throws IllegalArgumentException if the address associated with the lookup symbol do not match the > 88: * {@link MemoryLayout#byteAlignment() alignment constraints} in {@code layout}. Same remark here (`{@linkplain }`) src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryAddress.java line 43: > 41: * when performing memory dereference operations using a memory access var handle (see {@link MemoryHandles}). > 42: *

    > 43: * A memory address is associated with a {@link ResourceScope resource scope}; the resource scope determines the `{@linkplain }` src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryAddress.java line 46: > 44: * lifecycle of the memory address, and whether the address can be used from multiple threads. Memory addresses > 45: * obtained from {@link #ofLong(long) numeric values}, or from native code, are associated with the > 46: * {@link ResourceScope#globalScope() global resource scope}. Memory addresses obtained from segments ... and here to (`{@linkplain }`) src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryAddress.java line 102: > 100: * @param segment the segment relative to which this address offset should be computed > 101: * @throws IllegalArgumentException if {@code segment} is not compatible with this address; this can happen, for instance, > 102: * when {@code segment} models an heap memory region, while this address is a {@link #isNative() native} address. `{@linkplain }` src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryAddress.java line 209: > 207: /** > 208: * The native memory address instance modelling the {@code NULL} address, associated > 209: * with the {@link ResourceScope#globalScope() global} resource scope. `{@linkplain }` ? ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 20:38:55 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 20:38:55 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 19:02:57 GMT, Daniel Fuchs wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> Address first batch of review comments > > src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryAddress.java line 209: > >> 207: /** >> 208: * The native memory address instance modelling the {@code NULL} address, associated >> 209: * with the {@link ResourceScope#globalScope() global} resource scope. > > `{@linkplain }` ? thanks - I'll do a pass - I think I probably got all of them wrong ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 20:43:54 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 20:43:54 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v3] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 18:33:36 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/412 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Address review comments: > * fix typos in javadoc > * document ISE being thrown in all methods accepting a scope; add more tests for that > > test/jdk/jdk/internal/reflect/CallerSensitive/CheckCSMs.java > > Yes that's the test. That test misses to catch your case where aliasing may be possible. > Yes that's the test. That test misses to catch your case where aliasing may be possible. To be clear - by aliasing you mean when the @CallerSensitive implementation is called with invokeinterface - so, e.g. doing `MethodHandles.lookup().findVirtual(CLinker.class, ...)` right? ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mchung at openjdk.java.net Wed Apr 28 20:43:54 2021 From: mchung at openjdk.java.net (Mandy Chung) Date: Wed, 28 Apr 2021 20:43:54 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v3] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 20:38:47 GMT, Maurizio Cimadamore wrote: > To be clear - by aliasing you mean when the @CallerSensitive implementation is called with invokeinterface - so, e.g. doing `MethodHandles.lookup().findVirtual(CLinker.class, ...)` right? Yes. The caller would be java.base if it's invoked via method handle. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 21:13:53 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 21:13:53 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v3] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 20:40:49 GMT, Mandy Chung wrote: > > To be clear - by aliasing you mean when the @CallerSensitive implementation is called with invokeinterface - so, e.g. doing `MethodHandles.lookup().findVirtual(CLinker.class, ...)` right? > > Yes. The caller would be java.base if it's invoked via method handle. I just did a test: public class TestLookup { public static void main(String[] args) throws Throwable { MethodHandle handle = MethodHandles.lookup().findVirtual(CLinker.class, "downcallHandle", MethodType.methodType(MethodHandle.class, Addressable.class, MethodType.class, FunctionDescriptor.class)); CLinker linker = CLinker.getInstance(); handle.invoke(linker, MemoryAddress.NULL, MethodType.methodType(void.class), FunctionDescriptor.ofVoid()); } } this fails as expected when the handle is invoked. To test I had to disable the check on CLinker.getInstance - otherwise that would have always throw anyway. Also, on IllegalCaller vs. IllegalAccess - looking more, I think our impl throws IllegalCaller - now that was done because IllegalAccess is a checked exception and we don't want a checked exception here - but the option is called "enableNativeAccess" - is that still ok? ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Wed Apr 28 21:49:50 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 28 Apr 2021 21:49:50 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v2] In-Reply-To: References: <9aIvV_N89PFpzwfhedEwNju-eKqhpZoFHL59F5pUv6g=.9c515555-c446-4280-8f24-88ffd753712f@github.com> Message-ID: On Wed, 28 Apr 2021 18:44:18 GMT, Daniel Fuchs wrote: >> I think the method is called during module bootstrap - I don't think there is a race in practice. This method is also called in other parts of ModuleBootstrap. The code you allude to is called during initialization of the IllegalNativeAccessChecker singleton, which should happen only once, and only from one thread. > > I'll take your word for it - the use of a volatile variable to store the singleton instance made this suspicious. good point - I think that came in when I adapted the code from IllegalAccessLogger - which can indeed be accessed from multiple threads. In this case the initialization is effectively part of bootstrap, and volatile is not required. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mchung at openjdk.java.net Wed Apr 28 23:26:01 2021 From: mchung at openjdk.java.net (Mandy Chung) Date: Wed, 28 Apr 2021 23:26:01 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v3] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 21:10:33 GMT, Maurizio Cimadamore wrote: > I just did a test: > > ``` > public class TestLookup { > public static void main(String[] args) throws Throwable { > MethodHandle handle = MethodHandles.lookup().findVirtual(CLinker.class, "downcallHandle", MethodType.methodType(MethodHandle.class, Addressable.class, MethodType.class, FunctionDescriptor.class)); > CLinker linker = CLinker.getInstance(); > handle.invoke(linker, MemoryAddress.NULL, MethodType.methodType(void.class), FunctionDescriptor.ofVoid()); > } > } > ``` > > this fails as expected when the handle is invoked. To test I had to disable the check on CLinker.getInstance - otherwise that would have always throw anyway. My statement was overly simplified. If `handle` is invoked in another module B and invoked by a class in module B, which module (the `lookup`'s module or ) do you expect be the caller to check against for native access check? `CLinker::downcallHandle` is not caller-sensitive but its implementation is. The method handle of a caller-sensitive method behaves as if it were called from an instruction contained in the lookup class [1]. [1] https://download.java.net/java/early_access/jdk17/docs/api/java.base/java/lang/invoke/MethodHandles.Lookup.html#callsens > Also, on IllegalCaller vs. IllegalAccess - looking more, I think our impl throws IllegalCaller - now that was done because IllegalAccess is a checked exception and we don't want a checked exception here - but the option is called "enableNativeAccess" - is that still ok? Yes the implementation throws `IllegalCallerException` which is why I point out this. Hmm... this seems more of `IllegalAccess` as the caller does not have access to this restricted method. OTOH, `Module::addOpens` grants deep reflection access to the named module if the caller has access. Otherwise, `IllegalCallerException` is thrown. So I think it's okay to throw ICE. Others may have a different opinion. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mik3hall at gmail.com Thu Apr 29 00:13:35 2021 From: mik3hall at gmail.com (Michael Hall) Date: Wed, 28 Apr 2021 19:13:35 -0500 Subject: FileSystemProvider Message-ID: <526680A4-7424-49B9-AA08-D690D0244126@gmail.com> Possibly related to https://bugs.openjdk.java.net/browse/JDK-8263940 I have recreated this with the JDK-16. I think it begins after JDK-12. I don?t have.a 13 it is present in 14. javac -d . ~/Documents/GitHub/jdk16-master/test/jdk/java/nio/file/spi/TestProvider.java Your test case. Not properly set up on git for me, but there. The error is if -Djava.security.manager and -Djava.security.policy are used in invocation?. /usr/libexec/java_home -v 14 --exec java -cp . -Djava.security.manager -Djava.security.policy=Classes/all.policy -Djava.nio.file.spi.DefaultFileSystemProvider=TestProvider Test Exception in thread "main? java.lang.StackOverflowError at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:228) at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:593) at java.base/java.lang.StringBuilder.append(StringBuilder.java:173) at java.base/java.security.Security.getProperty(Security.java:765) at java.base/sun.security.provider.PolicyFile$1.run(PolicyFile.java:310) at java.base/sun.security.provider.PolicyFile$1.run(PolicyFile.java:306) at java.base/java.security.AccessController.doPrivileged(AccessController.java:312) at java.base/sun.security.provider.PolicyFile.init(PolicyFile.java:306) at java.base/sun.security.provider.PolicyFile.(PolicyFile.java:283) at java.base/java.security.Policy.loadPolicyProvider(Policy.java:207) at java.base/java.security.Policy.getPolicyNoCheck(Policy.java:178) at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321) at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353) at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450) at java.base/java.security.AccessController.checkPermission(AccessController.java:1036) at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:408) at java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:747) at java.base/sun.nio.fs.UnixChannelFactory.open(UnixChannelFactory.java:255) at java.base/sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:143) at java.base/sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:156) at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:217) at TestProvider.newByteChannel(TestProvider.java:173) at java.base/java.nio.file.Files.newByteChannel(Files.java:375) at java.base/java.nio.file.Files.newByteChannel(Files.java:426) at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420) at java.base/java.nio.file.Files.newInputStream(Files.java:160) at java.base/java.nio.file.Files.newBufferedReader(Files.java:2916) at java.base/java.nio.file.Files.newBufferedReader(Files.java:2948) at java.base/sun.security.provider.PolicyFile.initDefaultPolicy(PolicyFile.java:485) at java.base/sun.security.provider.PolicyFile$2.run(PolicyFile.java:340) at java.base/sun.security.provider.PolicyFile$2.run(PolicyFile.java:337) at java.base/java.security.AccessController.doPrivileged(AccessController.java:312) at java.base/sun.security.provider.PolicyFile.initPolicyFile(PolicyFile.java:337) at java.base/sun.security.provider.PolicyFile.init(PolicyFile.java:330) at java.base/sun.security.provider.PolicyFile.(PolicyFile.java:283) at java.base/java.security.Policy.loadPolicyProvider(Policy.java:207) at java.base/java.security.Policy.getPolicyNoCheck(Policy.java:178) at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321) at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353) at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450) at java.base/java.security.AccessController.checkPermission(AccessController.java:1036) at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:408) at java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:747) at java.base/sun.nio.fs.UnixChannelFactory.open(UnixChannelFactory.java:255) at java.base/sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:143) at java.base/sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:156) at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:217) at TestProvider.newByteChannel(TestProvider.java:173) It loops until the StackOverflowError. If not related should I do a bug report? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mik3hall at gmail.com Thu Apr 29 00:18:40 2021 From: mik3hall at gmail.com (Michael Hall) Date: Wed, 28 Apr 2021 19:18:40 -0500 Subject: FileSystemProvider In-Reply-To: <526680A4-7424-49B9-AA08-D690D0244126@gmail.com> References: <526680A4-7424-49B9-AA08-D690D0244126@gmail.com> Message-ID: <3D35610B-E2A7-4075-BBED-7C289766C171@gmail.com> > On Apr 28, 2021, at 7:13 PM, Michael Hall wrote: > > Possibly related to https://bugs.openjdk.java.net/browse/JDK-8263940 > > I have recreated this with the JDK-16. I think it begins after JDK-12. I don?t have.a 13 it is present in 14. > > javac -d . ~/Documents/GitHub/jdk16-master/test/jdk/java/nio/file/spi/TestProvider.java > > Your test case. Not properly set up on git for me, but there. > > The error is if -Djava.security.manager and -Djava.security.policy are used in invocation?. > > /usr/libexec/java_home -v 14 --exec java -cp . -Djava.security.manager -Djava.security.policy=Classes/all.policy -Djava.nio.file.spi.DefaultFileSystemProvider=TestProvider Test > Exception in thread "main? java.lang.StackOverflowError > at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:228) > at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:593) I can provide the start of the stack trace if needed? MFSP newByteChannel proxy /Library/Java/JavaVirtualMachines/jdk-16.jdk/Contents/Home/lib/security/default.policy From mine with debugging messages. It appears to try and get this file. In the process it does another security check before initialized that triggers the loop. My guess as to what it appears to do. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yyang at openjdk.java.net Thu Apr 29 02:33:52 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Thu, 29 Apr 2021 02:33:52 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 17:32:18 GMT, Igor Veresov wrote: > Do we need to keep this flag? Exactly, the flag should be removed. Thanks, Yang ------------- PR: https://git.openjdk.java.net/jdk/pull/3615 From yyang at openjdk.java.net Thu Apr 29 02:47:19 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Thu, 29 Apr 2021 02:47:19 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v3] In-Reply-To: References: Message-ID: > The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. > > In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. > > But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: > > 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. > 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag > > Testing: cds, compiler and jdk Yi Yang has updated the pull request incrementally with one additional commit since the last revision: remove InlineNIOCheckIndex flag ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3615/files - new: https://git.openjdk.java.net/jdk/pull/3615/files/7f30dc48..db8b6ef4 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3615&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3615&range=01-02 Stats: 23 lines in 3 files changed: 11 ins; 11 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3615.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3615/head:pull/3615 PR: https://git.openjdk.java.net/jdk/pull/3615 From yyang at openjdk.java.net Thu Apr 29 02:53:18 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Thu, 29 Apr 2021 02:53:18 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v4] In-Reply-To: References: Message-ID: > The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. > > In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. > > But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: > > 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. > 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag > > Testing: cds, compiler and jdk Yi Yang has updated the pull request incrementally with one additional commit since the last revision: remove extra newline ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3615/files - new: https://git.openjdk.java.net/jdk/pull/3615/files/db8b6ef4..bbdf4b9e Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3615&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3615&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3615.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3615/head:pull/3615 PR: https://git.openjdk.java.net/jdk/pull/3615 From dfuchs at openjdk.java.net Thu Apr 29 09:33:58 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Thu, 29 Apr 2021 09:33:58 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v4] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 02:53:18 GMT, Yi Yang wrote: >> The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. >> >> In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. >> >> But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: >> >> 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. >> 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag >> >> Testing: cds, compiler and jdk > > Yi Yang has updated the pull request incrementally with one additional commit since the last revision: > > remove extra newline test/hotspot/jtreg/compiler/c1/TestCheckIndexC1Intrinsic.java line 60: > 58: } catch (IndexOutOfBoundsException e) { > 59: // got it! > 60: } In all places where `Precondition.checkIndex` is expected to throw, an AssertionError should be generated if it doesn't throw: try { Preconditions.checkIndex(1, 1, null); throw new AssertionError("Expected IndexOutOfBoundsException not thrown"); } catch (IndexOutOfBoundsException e) { // got it! } ------------- PR: https://git.openjdk.java.net/jdk/pull/3615 From yyang at openjdk.java.net Thu Apr 29 10:01:52 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Thu, 29 Apr 2021 10:01:52 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v4] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 09:30:50 GMT, Daniel Fuchs wrote: >> Yi Yang has updated the pull request incrementally with one additional commit since the last revision: >> >> remove extra newline > > test/hotspot/jtreg/compiler/c1/TestCheckIndexC1Intrinsic.java line 60: > >> 58: } catch (IndexOutOfBoundsException e) { >> 59: // got it! >> 60: } > > In all places where `Precondition.checkIndex` is expected to throw, an AssertionError should be generated if it doesn't throw: > > > try { > Preconditions.checkIndex(1, 1, null); > throw new AssertionError("Expected IndexOutOfBoundsException not thrown"); > } catch (IndexOutOfBoundsException e) { > // got it! > } Yes, it does make sense ------------- PR: https://git.openjdk.java.net/jdk/pull/3615 From dfuchs at openjdk.java.net Thu Apr 29 10:16:20 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Thu, 29 Apr 2021 10:16:20 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v5] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 10:13:16 GMT, Yi Yang wrote: >> The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. >> >> In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. >> >> But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: >> >> 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. >> 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag >> >> Testing: cds, compiler and jdk > > Yi Yang has updated the pull request incrementally with one additional commit since the last revision: > > AssertionError when expected exception was not thrown test/hotspot/jtreg/compiler/c1/TestCheckIndexC1Intrinsic.java line 86: > 84: // read fields > 85: Preconditions.checkIndex(limit + 1, limit, (s, integers) -> new MyException("Reason:" + s + "::" + integers)); > 86: throw new AssertionError("Expected IndexOutOfBoundsException not thrown"); nit: well maybe here it should be: throw new AssertionError("Expected MyException not thrown"); test/hotspot/jtreg/compiler/c1/TestCheckIndexC1Intrinsic.java line 94: > 92: static void check1(int i) { > 93: try { > 94: Preconditions.checkIndex(i, 9999, (s, integers) -> new RuntimeException("ex")); I believe throw new AssertionError("Expected IndexOutOfBoundsException not thrown"); should be added in `check1`...`check4` as well... ------------- PR: https://git.openjdk.java.net/jdk/pull/3615 From yyang at openjdk.java.net Thu Apr 29 10:16:19 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Thu, 29 Apr 2021 10:16:19 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v5] In-Reply-To: References: Message-ID: > The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. > > In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. > > But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: > > 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. > 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag > > Testing: cds, compiler and jdk Yi Yang has updated the pull request incrementally with one additional commit since the last revision: AssertionError when expected exception was not thrown ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3615/files - new: https://git.openjdk.java.net/jdk/pull/3615/files/bbdf4b9e..1a96be7e Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3615&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3615&range=03-04 Stats: 8 lines in 1 file changed: 5 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3615.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3615/head:pull/3615 PR: https://git.openjdk.java.net/jdk/pull/3615 From mcimadamore at openjdk.java.net Thu Apr 29 10:34:53 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Thu, 29 Apr 2021 10:34:53 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v3] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 23:22:38 GMT, Mandy Chung wrote: > My statement was overly simplified, sorry. If `handle` is invoked in another module B and invoked by a class in module B, which module (the `lookup`'s module or ) do you expect be the caller to check against for native access check? `CLinker::downcallHandle` is not caller-sensitive but its implementation is. > > The method handle of a caller-sensitive method behaves as if it were called from an instruction contained in the lookup class [1]. I think I expect that, with caller sensitive, it is possible from a client in an "enabled" module to obtain a MethodHandle, and then pass it to an unprivileged module, which then calls it, and works ok. This matches my expectation (this trick could be used also to give a client access to an otherwise inaccessible - in the classic sense - method). So, when working with method handles, I'm completely fine working with the lookup class as context (and not with the real caller class). Given all this, do you agree that current checks are not affected by interface-ness of some of the methods involved? ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From yyang at openjdk.java.net Thu Apr 29 10:40:52 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Thu, 29 Apr 2021 10:40:52 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v5] In-Reply-To: References: Message-ID: <8qnpaKEeGr3iGKtcGK8l5YjtxP_Y3cdmwbbDVvr-Y7Y=.d0a0cc48-3027-4b15-982b-88b18c4d0830@github.com> On Thu, 29 Apr 2021 10:13:05 GMT, Daniel Fuchs wrote: >> Yi Yang has updated the pull request incrementally with one additional commit since the last revision: >> >> AssertionError when expected exception was not thrown > > test/hotspot/jtreg/compiler/c1/TestCheckIndexC1Intrinsic.java line 94: > >> 92: static void check1(int i) { >> 93: try { >> 94: Preconditions.checkIndex(i, 9999, (s, integers) -> new RuntimeException("ex")); > > I believe > > throw new AssertionError("Expected IndexOutOfBoundsException not thrown"); > > > should be added in `check1`...`check4` as well... Hmm.. They would throw desired exceptions only if i==9999. ------------- PR: https://git.openjdk.java.net/jdk/pull/3615 From mcimadamore at openjdk.java.net Thu Apr 29 11:15:00 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Thu, 29 Apr 2021 11:15:00 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v4] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/412 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Address review comments: * Fix all link vs. linkplain * Remove `volatile` and other defensive copying in `IllegalNativeAccessChecker` * Added @throws clauses in restricted methods for `IllegalCallerException` ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3699/files - new: https://git.openjdk.java.net/jdk/pull/3699/files/24e554c5..20671853 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=02-03 Stats: 102 lines in 8 files changed: 57 ins; 0 del; 45 mod Patch: https://git.openjdk.java.net/jdk/pull/3699.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3699/head:pull/3699 PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Thu Apr 29 11:15:32 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Thu, 29 Apr 2021 11:15:32 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v3] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 18:33:36 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/412 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Address review comments: > * fix typos in javadoc > * document ISE being thrown in all methods accepting a scope; add more tests for that Here's a new round of javadoc/specdiff with latest changes Javadoc: http://cr.openjdk.java.net/~mcimadamore/JEP-412/v2/javadoc/jdk/incubator/foreign/package-summary.html Specdiff: http://cr.openjdk.java.net/~mcimadamore/JEP-412/v2/specdiff/overview-summary.html ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Thu Apr 29 12:22:41 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Thu, 29 Apr 2021 12:22:41 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v5] In-Reply-To: References: Message-ID: <02OtcjUL7EtDhrlea772aQceCYVBpl3uMAWuwkgK6p0=.46f81497-dcc1-4ab2-b60a-84ee417f1301@github.com> > This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/412 Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision: - Merge branch 'master' into JEP-412 - Address review comments: * Fix all link vs. linkplain * Remove `volatile` and other defensive copying in `IllegalNativeAccessChecker` * Added @throws clauses in restricted methods for `IllegalCallerException` - Address review comments: * fix typos in javadoc * document ISE being thrown in all methods accepting a scope; add more tests for that - Address first batch of review comments - Add linker test excludes for x86 and other platforms which do not support `CLinker` - Integrate latest API tweaks - Fix x86 build - Fix `TestLinkToNativeRBP` - Initial push ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3699/files - new: https://git.openjdk.java.net/jdk/pull/3699/files/20671853..f27db3f3 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=03-04 Stats: 477828 lines in 3664 files changed: 4983 ins; 470775 del; 2070 mod Patch: https://git.openjdk.java.net/jdk/pull/3699.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3699/head:pull/3699 PR: https://git.openjdk.java.net/jdk/pull/3699 From bpb at openjdk.java.net Thu Apr 29 15:36:53 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 29 Apr 2021 15:36:53 GMT Subject: Integrated: 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow In-Reply-To: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> References: <_IpcmMVZxGDatnYm_y5zoKDhVt_ptQRmmulgJF63vGw=.cd2fc3f0-a7de-438e-854f-20c60c3079c1@github.com> Message-ID: On Fri, 23 Apr 2021 19:23:13 GMT, Brian Burkhalter wrote: > Please consider this request to accelerate absolute and relative bulk array transfer on views of heap byte buffers where the element size is greater than one. What currently happens is that the transfer devolves to a ?loopy? element-by-element copy such as > > int end = offset + length; > for (int i = offset, j = index; i < end; i++, j++) > dst[i] = get(j); > > for `get()`, and > > int end = offset + length; > for (int i = offset, j = index; i < end; i++, j++) > this.put(j, src[i]); > > for `put()`. This is of course relatively slow. > > The change proposed hoists the accelerated versions of these methods using the `ScopedMemoryAccess` methods `copyMemory()` and `copySwapMemory()` from `Direct-X-Buffer` to `X-Buffer`. The array bulk transfer methods are removed from `Direct-X-Buffer` itself. The number of lines of code in the templates decreases by 87. > > With this change the throughput of array bulk `put()` and `get()` for heap view buffers is increased by a factor of 6 to 11 compared with the current code. The performance of direct view buffers does not appear to be affected. > > No tests are modified or added as existing tests already cover these methods. All tests in CI tiers 1-3 passed on all platforms. This pull request has now been integrated. Changeset: 6bb71d9e Author: Brian Burkhalter URL: https://git.openjdk.java.net/jdk/commit/6bb71d9e25f04dbde13d010e2e6279c8059c2cdb Stats: 291 lines in 2 files changed: 88 ins; 183 del; 20 mod 8264762: ByteBuffer.byteOrder(BIG_ENDIAN).asXBuffer.put(Xarray) and ByteBuffer.byteOrder(nativeOrder()).asXBuffer.put(Xarray) are slow Reviewed-by: alanb, psandoz, chegar ------------- PR: https://git.openjdk.java.net/jdk/pull/3660 From mchung at openjdk.java.net Thu Apr 29 18:20:53 2021 From: mchung at openjdk.java.net (Mandy Chung) Date: Thu, 29 Apr 2021 18:20:53 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v3] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 10:31:29 GMT, Maurizio Cimadamore wrote: > I think I expect that, with caller sensitive, it is possible from a client in an "enabled" module to obtain a MethodHandle, and then pass it to an unprivileged module, which then calls it, and works ok. This matches my expectation (this trick could be used also to give a client access to an otherwise inaccessible - in the classic sense - method). > > So, when working with method handles, I'm completely fine working with the lookup class as context (and not with the real caller class). Good, this matches the behavior of method handles for CSMs. However, I think the implementation does the opposite since we don't support interface method being CSM. Can you test that out? > Given all this, do you agree that current checks are not affected by interface-ness of some of the methods involved? I think the implementation does not support that. I will also need to look into how this impacts JDK-8266010. As I suggest earlier, I'm fine to do this as a follow up after integration. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From kustos at gmx.net Thu Apr 29 18:47:58 2021 From: kustos at gmx.net (Philippe Marschall) Date: Thu, 29 Apr 2021 20:47:58 +0200 Subject: Missing specialization in CharBufferR.put(String, int, int) Message-ID: <665a62d2-fbfd-d26b-13f5-8c4bf6a42463@gmx.net> Hello CharBufferR.put(String, int, int) seems to be the only put* method in CharBufferR that doesn't unconditionally throw ReadOnlyBufferException but instead copies the superclass implementation. This seems to have been introduced with JDK-8011135. I think this would be relatively easy to fix in Heap-X-Buffer.java.template by means of #if[rw] #else[rw] similar to all the other put* methods. If you agree this is a bug and should be fixed then I can come up with a PR if somebody creates a JIRA. [1] https://bugs.openjdk.java.net/browse/JDK-8011135 Cheers Philippe From mik3hall at gmail.com Thu Apr 29 18:55:17 2021 From: mik3hall at gmail.com (Michael Hall) Date: Thu, 29 Apr 2021 13:55:17 -0500 Subject: FileSystemProvider In-Reply-To: <3D35610B-E2A7-4075-BBED-7C289766C171@gmail.com> References: <526680A4-7424-49B9-AA08-D690D0244126@gmail.com> <3D35610B-E2A7-4075-BBED-7C289766C171@gmail.com> Message-ID: > > I can provide the start of the stack trace if needed? > > MFSP newByteChannel proxy /Library/Java/JavaVirtualMachines/jdk-16.jdk/Contents/Home/lib/security/default.policy > > From mine with debugging messages. It appears to try and get this file. In the process it does another security check before initialized that triggers the loop. My guess as to what it appears to do. Should of been DefaultFileSystemProvider. Submitted a bug report internal review ID : 9070087 -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Apr 29 19:53:36 2021 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 29 Apr 2021 19:53:36 +0000 Subject: Missing specialization in CharBufferR.put(String, int, int) In-Reply-To: <665a62d2-fbfd-d26b-13f5-8c4bf6a42463@gmx.net> References: <665a62d2-fbfd-d26b-13f5-8c4bf6a42463@gmx.net> Message-ID: <9AF8FC3A-F0EA-4F16-A4A2-52D947E7BC27@oracle.com> Hello, I created https://bugs.openjdk.java.net/browse/JDK-8266320 for this. Thanks, Brian On Apr 29, 2021, at 11:47 AM, Philippe Marschall > wrote: Hello CharBufferR.put(String, int, int) seems to be the only put* method in CharBufferR that doesn't unconditionally throw ReadOnlyBufferException but instead copies the superclass implementation. This seems to have been introduced with JDK-8011135. I think this would be relatively easy to fix in Heap-X-Buffer.java.template by means of #if[rw] #else[rw] similar to all the other put* methods. If you agree this is a bug and should be fixed then I can come up with a PR if somebody creates a JIRA. [1] https://bugs.openjdk.java.net/browse/JDK-8011135 Cheers Philippe -------------- next part -------------- An HTML attachment was scrubbed... URL: From yyang at openjdk.java.net Fri Apr 30 02:19:30 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Fri, 30 Apr 2021 02:19:30 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v6] In-Reply-To: References: Message-ID: > The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. > > In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. > > But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: > > 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. > 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag > > Testing: cds, compiler and jdk Yi Yang has updated the pull request incrementally with one additional commit since the last revision: better check1-4 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3615/files - new: https://git.openjdk.java.net/jdk/pull/3615/files/1a96be7e..954abc6e Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3615&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3615&range=04-05 Stats: 32 lines in 1 file changed: 31 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3615.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3615/head:pull/3615 PR: https://git.openjdk.java.net/jdk/pull/3615 From github.com+471021+marschall at openjdk.java.net Fri Apr 30 07:26:11 2021 From: github.com+471021+marschall at openjdk.java.net (Philippe Marschall) Date: Fri, 30 Apr 2021 07:26:11 GMT Subject: RFR: 8266320: (bf) ReadOnlyBufferException in heap buffer put(String, int, int) should not be conditional Message-ID: 8266320: (bf) ReadOnlyBufferException in heap buffer put(String,int,int) should not be conditional ------------- Commit messages: - 8266320: (bf) ReadOnlyBufferException in heap buffer put(String,int,int) should not be conditional Changes: https://git.openjdk.java.net/jdk/pull/3809/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3809&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266320 Stats: 9 lines in 2 files changed: 7 ins; 2 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3809.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3809/head:pull/3809 PR: https://git.openjdk.java.net/jdk/pull/3809 From dfuchs at openjdk.java.net Fri Apr 30 08:59:51 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Fri, 30 Apr 2021 08:59:51 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v6] In-Reply-To: References: Message-ID: <7a4Q1KdYnrj-8iKArQaCHP_s6QHuN58Gz1PpvlSOKzo=.03f760ef-75a4-4882-ac3a-b5a21bc3d73d@github.com> On Fri, 30 Apr 2021 02:19:30 GMT, Yi Yang wrote: >> The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. >> >> In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. >> >> But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: >> >> 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. >> 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag >> >> Testing: cds, compiler and jdk > > Yi Yang has updated the pull request incrementally with one additional commit since the last revision: > > better check1-4 Thanks for taking this feedback into account and updating the test! Note: I only reviewed the test. ------------- Marked as reviewed by dfuchs (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3615 From mcimadamore at openjdk.java.net Fri Apr 30 12:23:45 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Fri, 30 Apr 2021 12:23:45 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v6] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/412 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: * Add @CS to interface methods in the Foreign API * Beef up test for methd handles and restricted methods ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3699/files - new: https://git.openjdk.java.net/jdk/pull/3699/files/f27db3f3..75474a1f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=04-05 Stats: 306 lines in 8 files changed: 306 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3699.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3699/head:pull/3699 PR: https://git.openjdk.java.net/jdk/pull/3699 From mcimadamore at openjdk.java.net Fri Apr 30 12:27:55 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Fri, 30 Apr 2021 12:27:55 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v3] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 18:18:00 GMT, Mandy Chung wrote: > I think the implementation does not support that. I will also need to look into how this impacts JDK-8266010. As I suggest earlier, I'm fine to do this as a follow up after integration. I've added `@CS` in the interface methods too. I've also added a stronger test which creates method handles in one module (which doesn't have native access) and then calls them from another module (which does NOT have native access enabled), and make sure that all call fails as expected (e.g. that caller context is that of the lookup module). Surprisingly, CheckCSMs does NOT fail if both interface and implementation methods are `@CS` annotated - but if only interface is annotated, the test fails. This seems odd - since I can't find any logic in the test which check for overriding. Is the test accidentally passing? ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From bpb at openjdk.java.net Fri Apr 30 15:07:53 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Fri, 30 Apr 2021 15:07:53 GMT Subject: RFR: 8266320: (bf) ReadOnlyBufferException in heap buffer put(String, int, int) should not be conditional In-Reply-To: References: Message-ID: <3vObMZRr7OJ3IJ8FmtzzOCNiI-gSYsGrQgEx1E4DoEU=.cadad8c3-cfdf-46d8-91fa-c43e29dabc23@github.com> On Fri, 30 Apr 2021 07:20:44 GMT, Philippe Marschall wrote: > 8266320: (bf) ReadOnlyBufferException in heap buffer put(String,int,int) should not be conditional The more recent copyright year needs to be 2021 in all files changed. `Basic$Type$.java` will need to be generated from `Basic-X.java.template` using `genBasic.sh`; these files are not automatically generated. test/jdk/java/nio/Buffer/Basic-X.java.template line 1006: > 1004: catchReadOnlyBuffer(b, () -> rb.append(new String(new char[rb.remaining() + 1]))); > 1005: > 1006: // 8266320 The bug ID should be deleted here and appended to the list of bug IDs in Basic.java. ------------- PR: https://git.openjdk.java.net/jdk/pull/3809 From mcimadamore at openjdk.java.net Fri Apr 30 15:20:42 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Fri, 30 Apr 2021 15:20:42 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v7] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-412 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/412 Maurizio Cimadamore has updated the pull request incrementally with two additional commits since the last revision: - Revert bad change in benchmark copyright - Do not apply optimized bound check if accessed offset/length do not fit in an `int` value ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3699/files - new: https://git.openjdk.java.net/jdk/pull/3699/files/75474a1f..793ea5c5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=06 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3699&range=05-06 Stats: 23 lines in 3 files changed: 17 ins; 5 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3699.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3699/head:pull/3699 PR: https://git.openjdk.java.net/jdk/pull/3699 From alanb at openjdk.java.net Fri Apr 30 15:28:52 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 30 Apr 2021 15:28:52 GMT Subject: RFR: 8266320: (bf) ReadOnlyBufferException in heap buffer put(String, int, int) should not be conditional In-Reply-To: References: Message-ID: On Fri, 30 Apr 2021 07:20:44 GMT, Philippe Marschall wrote: > 8266320: (bf) ReadOnlyBufferException in heap buffer put(String,int,int) should not be conditional src/java.base/share/classes/java/nio/Heap-X-Buffer.java.template line 289: > 287: #else[rw] > 288: throw new ReadOnlyBufferException(); > 289: #end[rw] I assume this isn't technically a bug in that it's unspecified which exception is thrown when the buffer is read-only and an index is out of range. The change is okay of course. It would be useful to run the tests without the addition to Basic-X.java.template as I would expect this case to be covered by an existing test already. ------------- PR: https://git.openjdk.java.net/jdk/pull/3809 From Alan.Bateman at oracle.com Fri Apr 30 16:17:23 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 30 Apr 2021 17:17:23 +0100 Subject: FileSystemProvider In-Reply-To: <526680A4-7424-49B9-AA08-D690D0244126@gmail.com> References: <526680A4-7424-49B9-AA08-D690D0244126@gmail.com> Message-ID: On 29/04/2021 01:13, Michael Hall wrote: > : > > The error is if -Djava.security.manager and -Djava.security.policy are > used in invocation?. > > Thanks for the bug report. It's in JBS as JDK-8266345. I think this is another case where the PolicyFile code needs to be very defensive as opening the policy file is a privileged operation. Also interposing on the default file system provider sets up a sandwich where the PolicyFile is going through untrusted code to get to the file system. Getting all these pieces to work together may require the PolicyFile code to use the built-in file system provider directly, needs more investigation. -Alan From mik3hall at gmail.com Fri Apr 30 16:40:53 2021 From: mik3hall at gmail.com (Michael Hall) Date: Fri, 30 Apr 2021 11:40:53 -0500 Subject: FileSystemProvider In-Reply-To: References: <526680A4-7424-49B9-AA08-D690D0244126@gmail.com> Message-ID: <37319CB2-93F0-474D-B433-B18DEAC3881C@gmail.com> > On Apr 30, 2021, at 11:17 AM, Alan Bateman wrote: > > On 29/04/2021 01:13, Michael Hall wrote: >> : >> >> The error is if -Djava.security.manager and -Djava.security.policy are used in invocation?. >> >> > Thanks for the bug report. It's in JBS as JDK-8266345. I think this is another case where the PolicyFile code needs to be very defensive as opening the policy file is a privileged operation. Also interposing on the default file system provider sets up a sandwich where the PolicyFile is going through untrusted code to get to the file system. Getting all these pieces to work together may require the PolicyFile code to use the built-in file system provider directly, needs more investigation. > > -Alan OK, thanks. For now I can test the DefaultFileSystemProvider using jdk12. From mchung at openjdk.java.net Fri Apr 30 17:23:01 2021 From: mchung at openjdk.java.net (Mandy Chung) Date: Fri, 30 Apr 2021 17:23:01 GMT Subject: RFR: 8264774: Implementation of Foreign Function and Memory API (Incubator) [v3] In-Reply-To: References: Message-ID: On Fri, 30 Apr 2021 12:24:38 GMT, Maurizio Cimadamore wrote: > I've added `@CS` in the interface methods too. I've also added a stronger test which creates method handles in one module (which doesn't have native access) and then calls them from another module (which does NOT have native access enabled), and make sure that all call fails as expected (e.g. that caller context is that of the lookup module). Thanks. > Surprisingly, CheckCSMs does NOT fail if both interface and implementation methods are `@CS` annotated - but if only interface is annotated, the test fails. This seems odd - since I can't find any logic in the test which check for overriding. Is the test accidentally passing? I create JDK-8266383 and I will look into that. ------------- PR: https://git.openjdk.java.net/jdk/pull/3699 From alanb at openjdk.java.net Fri Apr 30 17:25:29 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 30 Apr 2021 17:25:29 GMT Subject: RFR: 8266369: (se) Add wepoll based Selector Message-ID: <-hwm9SZitbfLQrsJcdXhUFgEnUrxJHH-i42QZy5ve1Q=.ebec11da-ceba-46dd-8870-554d0b83c45f@github.com> This PR proposes a new Selector implementation for the Windows platform based on the Bert Belder's "wepoll" library. The wepoll library provides a scalable polling interface to the Ancillary Function Driver for Winsock. The interface that it exposes is close to the Linux epoll interface. The main benefit is that the cost of selection operations are a function of the number of sockets that are ready for I/O rather than the number of channels registered with the Selector. The existing Selector on Windows uses the legacy "select" which scales horribly, partly because it's O(N) on the number of registered channels and partly because it is limited to 1024 sockets and so requires a thread per 1024 sockets. The patch brings the wepoll sources into the repo, adds a small shim, and the Selector implementation itself. The new implementation needs to handle OOB data in the same way as the old implementation so the method to discard OOB moves to Net. I've added a microbenchmark to demonstrate the scaling when there is 1, 10, ... 10k connections and one socket ready for I/O. Windows Server 2019 was used for the data here. I've seen improvements with almost everything that I've tried. The only exception is a select after a wakeup where the old implementation skips the selection operation (arguably a long standing bug in the old implementation). Existing select based Selector where the selection operation is almost 1ms when there are 10,000 channels registered. Benchmark (nchannels) Mode Cnt Score Error Units SelectOne.testSelectOne 1 avgt 20 1587.429 ? 4.053 ns/op SelectOne.testSelectOne 10 avgt 20 2515.324 ? 8.141 ns/op SelectOne.testSelectOne 100 avgt 20 10903.445 ? 65.508 ns/op SelectOne.testSelectOne 1000 avgt 20 101651.369 ? 673.781 ns/op SelectOne.testSelectOne 10000 avgt 20 932032.385 ? 59004.246 ns/op New wepoll based Selector: Benchmark (nchannels) Mode Cnt Score Error Units SelectOne.testSelectOne 1 avgt 20 1428.914 ? 6.648 ns/op SelectOne.testSelectOne 10 avgt 20 1429.806 ? 5.537 ns/op SelectOne.testSelectOne 100 avgt 20 1433.371 ? 3.253 ns/op SelectOne.testSelectOne 1000 avgt 20 1435.007 ? 12.420 ns/op SelectOne.testSelectOne 10000 avgt 20 1445.201 ? 7.902 ns/op Replacing the Selector implementation after 19 years of service is a big step. The proposal is to leave the existing implementation until there is confidence with the new implementation. It also serves as a backup in case Windows were to significantly change the AFD interface. The old Selector can be used by running with -Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.WindowsSelectorProvider. A selection of existing tests have been updated to run a second time with the old Selector so that it is continues to be tested. CSR and Release Note planned. ------------- Commit messages: - Initial commit Changes: https://git.openjdk.java.net/jdk/pull/3816/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3816&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266369 Stats: 3271 lines in 23 files changed: 3169 ins; 61 del; 41 mod Patch: https://git.openjdk.java.net/jdk/pull/3816.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3816/head:pull/3816 PR: https://git.openjdk.java.net/jdk/pull/3816 From psandoz at openjdk.java.net Fri Apr 30 17:33:52 2021 From: psandoz at openjdk.java.net (Paul Sandoz) Date: Fri, 30 Apr 2021 17:33:52 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v6] In-Reply-To: References: Message-ID: <9Z_DkUjmqefCjf9mvecHUtoLHhw1qGNWJPxufuwvXI0=.36498a86-d09f-4eea-ab89-74844dd862cf@github.com> On Fri, 30 Apr 2021 02:19:30 GMT, Yi Yang wrote: >> The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. >> >> In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. >> >> But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: >> >> 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. >> 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag >> >> Testing: cds, compiler and jdk > > Yi Yang has updated the pull request incrementally with one additional commit since the last revision: > > better check1-4 It was my hope this would eventually happen when we added `Objects.checkIndex` and the underlying intrinsic. Very good! ------------- PR: https://git.openjdk.java.net/jdk/pull/3615 From iveresov at openjdk.java.net Fri Apr 30 19:23:52 2021 From: iveresov at openjdk.java.net (Igor Veresov) Date: Fri, 30 Apr 2021 19:23:52 GMT Subject: RFR: 8265518: C1: Intrinsic support for Preconditions.checkIndex [v6] In-Reply-To: References: Message-ID: On Fri, 30 Apr 2021 02:19:30 GMT, Yi Yang wrote: >> The JDK codebase re-created many variants of checkIndex(`grep -I -r 'cehckIndex' jdk/`). A notable variant is java.nio.Buffer.checkIndex, which annotated with @IntrinsicCandidate and it only has a corresponding C1 intrinsic version. >> >> In fact, there is an utility method `jdk.internal.util.Preconditions.checkIndex`(wrapped by java.lang.Objects.checkIndex) that behaves the same as these variants of checkIndex, we can replace these re-created variants of checkIndex by Objects.checkIndex, it would significantly reduce duplicated code and enjoys performance improvement because Preconditions.checkIndex is @IntrinsicCandidate and it has a corresponding intrinsic method in HotSpot. >> >> But, the problem is currently HotSpot only implements the C2 version of Preconditions.checkIndex. To reuse it global-widely in JDK code, I think we can firstly implement its C1 counterpart. There are also a few kinds of stuff we can do later: >> >> 1. Replace all variants of checkIndex by Objects.checkIndex in the whole JDK codebase. >> 2. Remove Buffer.checkIndex and obsolete/deprecate InlineNIOCheckIndex flag >> >> Testing: cds, compiler and jdk > > Yi Yang has updated the pull request incrementally with one additional commit since the last revision: > > better check1-4 Looks like now the test fails in the pre-submit tests? ------------- PR: https://git.openjdk.java.net/jdk/pull/3615