From Alan.Bateman at oracle.com  Thu Jul  1 08:43:46 2021
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 1 Jul 2021 09:43:46 +0100
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v2]
In-Reply-To: <f6ba21c5-6d3e-9491-3d3a-a40fee147640@gmail.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <C1tXC_OuVRZJHFI1P0mHAK8Oa95pEPgcsBsfOgebzvg=.35c3a21c-f02a-471c-901a-d22e0ec5a759@github.com>
 <X-L6wgK4tmBlZsu0b5egUts_ckRsn-yjjklqMPO_DoI=.0e1f2557-18c4-478a-b7db-c1e6c8046204@github.com>
 <f6ba21c5-6d3e-9491-3d3a-a40fee147640@gmail.com>
Message-ID: <083bb049-3eb5-fb83-4846-29627c7faf05@oracle.com>

On 30/06/2021 17:15, Jaikiran Pai wrote:
>
> I understand that Alan's suggestion holds good and we should have some 
> logic in place which switches to using a temp file once we notice that 
> the sizes we are dealing with can exceed some threshold, but I guess 
> that is something we need to do separately outside of this PR?

My comment was mostly just to point out that it's only a partial fix and 
it will eventually fail with an OOME once the deflated size is too big 
for the BAOS. I don't have a strong opinion on whether the complete fix 
is done in one, two or many PRs but I think the first step could be to 
use the "useTempFile" path when the entry size is larger than some (10s 
of MB?) threshold.

-Alan

From jai.forums2013 at gmail.com  Thu Jul  1 09:13:35 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Thu, 1 Jul 2021 14:43:35 +0530
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v2]
In-Reply-To: <083bb049-3eb5-fb83-4846-29627c7faf05@oracle.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <C1tXC_OuVRZJHFI1P0mHAK8Oa95pEPgcsBsfOgebzvg=.35c3a21c-f02a-471c-901a-d22e0ec5a759@github.com>
 <X-L6wgK4tmBlZsu0b5egUts_ckRsn-yjjklqMPO_DoI=.0e1f2557-18c4-478a-b7db-c1e6c8046204@github.com>
 <f6ba21c5-6d3e-9491-3d3a-a40fee147640@gmail.com>
 <083bb049-3eb5-fb83-4846-29627c7faf05@oracle.com>
Message-ID: <2c8f9b6e-80d8-95ae-62b0-5ca4d9d3cd22@gmail.com>

Thank you Alan and Lance. I'll update this PR shortly with the proposed 
approach.

-Jaikiran

On 01/07/21 2:13 pm, Alan Bateman wrote:
> On 30/06/2021 17:15, Jaikiran Pai wrote:
>>
>> I understand that Alan's suggestion holds good and we should have 
>> some logic in place which switches to using a temp file once we 
>> notice that the sizes we are dealing with can exceed some threshold, 
>> but I guess that is something we need to do separately outside of 
>> this PR?
>
> My comment was mostly just to point out that it's only a partial fix 
> and it will eventually fail with an OOME once the deflated size is too 
> big for the BAOS. I don't have a strong opinion on whether the 
> complete fix is done in one, two or many PRs but I think the first 
> step could be to use the "useTempFile" path when the entry size is 
> larger than some (10s of MB?) threshold.
>
> -Alan

From alanb at openjdk.java.net  Thu Jul  1 13:07:59 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Thu, 1 Jul 2021 13:07:59 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside
In-Reply-To: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
Message-ID: <ulkuJloqkrTihc2c_FI3Vr9HeBO7XsrkXzzJWzYxdJY=.6fa48d22-b72a-4869-b87a-07bd97465eb5@github.com>

On Sun, 27 Jun 2021 13:13:42 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> Can I please get a review of this proposed fix for https://bugs.openjdk.java.net/browse/JDK-8251329?
> 
> As noted in that issue, if a zip filesystem created on top of a jar containing a "./" entry is "walked" using the `Files.walkFileTree`, it leads to a infinite never ending iteration (which ultimately fails with Java heap space OOM).
> 
> Alan notes in that issue that:
> 
>> This is more likely an issue with the zipfs DirectoryStream implementation. A DirectoryStream is specified to not include elements that for the special links to the current or parent directory. It should be rare. 
> 
> This indeed turned out to be an issue in the `jdk.nio.zipfs.ZipDirectoryStream#iterator()` implementation where it calls the `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` implementation.  The implementation, unlike what the javadoc of `java.nio.file.DirectoryStream` states, was including the "." and ".." paths in its returned iterator:
> 
>> The elements returned by the iterator are in no specific order. Some file
>  systems maintain special links to the directory itself and the directory's
>   parent directory. Entries representing these links are not returned by the
>   iterator.
> 
> 
> The proposed fix in this patch checks the paths for "." and "..", similar to what the `sun.nio.fs.UnixDirectoryStream` does in its `isSelfOrParent` and skips those paths from being added into the returned iterator. The `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` (where this change has been done) is currently only used by `jdk.nio.zipfs.ZipDirectoryStream#iterator()` and has package-private visibility, so this change shouldn't impact any other usage/expectations.
> 
> A new jtreg test has been added to reproduce this issue and verify the fix. Local testing of the `test/jdk/jdk/nio/` (including this new test) went fine without any issues after this change. I will be triggering a `tier1` test locally in a while.

src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java line 1971:

> 1969:         }
> 1970:         return false;
> 1971:     }

It might be a bit clearer if ZipPath define isSelfOrParent(), that would avoid needing to expose the internal representation.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4604

From jpai at openjdk.java.net  Thu Jul  1 13:25:32 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Thu, 1 Jul 2021 13:25:32 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v2]
In-Reply-To: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
Message-ID: <YPCqIo02ztuhCJjbbrGnaF_FomLwH8ZrRu0JGj3Nc40=.05cc10af-9962-425f-8b90-86261619822e@github.com>

> Can I please get a review of this proposed fix for https://bugs.openjdk.java.net/browse/JDK-8251329?
> 
> As noted in that issue, if a zip filesystem created on top of a jar containing a "./" entry is "walked" using the `Files.walkFileTree`, it leads to a infinite never ending iteration (which ultimately fails with Java heap space OOM).
> 
> Alan notes in that issue that:
> 
>> This is more likely an issue with the zipfs DirectoryStream implementation. A DirectoryStream is specified to not include elements that for the special links to the current or parent directory. It should be rare. 
> 
> This indeed turned out to be an issue in the `jdk.nio.zipfs.ZipDirectoryStream#iterator()` implementation where it calls the `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` implementation.  The implementation, unlike what the javadoc of `java.nio.file.DirectoryStream` states, was including the "." and ".." paths in its returned iterator:
> 
>> The elements returned by the iterator are in no specific order. Some file
>  systems maintain special links to the directory itself and the directory's
>   parent directory. Entries representing these links are not returned by the
>   iterator.
> 
> 
> The proposed fix in this patch checks the paths for "." and "..", similar to what the `sun.nio.fs.UnixDirectoryStream` does in its `isSelfOrParent` and skips those paths from being added into the returned iterator. The `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` (where this change has been done) is currently only used by `jdk.nio.zipfs.ZipDirectoryStream#iterator()` and has package-private visibility, so this change shouldn't impact any other usage/expectations.
> 
> A new jtreg test has been added to reproduce this issue and verify the fix. Local testing of the `test/jdk/jdk/nio/` (including this new test) went fine without any issues after this change. I will be triggering a `tier1` test locally in a while.

Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:

  implement review suggestion - move isSelfOrParent to ZipPath class

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4604/files
  - new: https://git.openjdk.java.net/jdk/pull/4604/files/344da6cb..3971ad6f

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4604&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4604&range=00-01

  Stats: 20 lines in 2 files changed: 8 ins; 9 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4604.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4604/head:pull/4604

PR: https://git.openjdk.java.net/jdk/pull/4604

From jpai at openjdk.java.net  Thu Jul  1 13:25:37 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Thu, 1 Jul 2021 13:25:37 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v2]
In-Reply-To: <ulkuJloqkrTihc2c_FI3Vr9HeBO7XsrkXzzJWzYxdJY=.6fa48d22-b72a-4869-b87a-07bd97465eb5@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <ulkuJloqkrTihc2c_FI3Vr9HeBO7XsrkXzzJWzYxdJY=.6fa48d22-b72a-4869-b87a-07bd97465eb5@github.com>
Message-ID: <y3qwFFskkgTk96tsbVS1AKz4GNTgdoFBzqOtujKzvwc=.8ae3da6a-da7a-4a9e-ac86-1fe83357cc86@github.com>

On Thu, 1 Jul 2021 13:05:26 GMT, Alan Bateman <alanb at openjdk.org> wrote:

>> Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   implement review suggestion - move isSelfOrParent to ZipPath class
>
> src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java line 1971:
> 
>> 1969:         }
>> 1970:         return false;
>> 1971:     }
> 
> It might be a bit clearer if ZipPath define isSelfOrParent(), that would avoid needing to expose the internal representation.

Done. I've updated the PR to implement this suggestion.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4604

From github.com+1701815+mkarg at openjdk.java.net  Thu Jul  1 21:59:33 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Thu, 1 Jul 2021 21:59:33 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v2]
In-Reply-To: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
Message-ID: <cZlX2_-SUyC7Ynxi-JQF2FM6qay9v_dbaD5W4Z-0a9Y=.d0bc0712-1645-4f38-967d-3bc2691f1c36@github.com>

> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
> 
> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
> 
> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
> 
> I encourage everybody to discuss this draft:
> * Are there valid arguments for *not* doing this change?
> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
> * How to go on from here: What is missing to get this ready for an actual review?

Markus KARG has updated the pull request incrementally with one additional commit since the last revision:

  Draft: Renaming i and separating code into several methods
  
  Signed-off-by: Markus Karg <markus at headcrashing.eu>

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4263/files
  - new: https://git.openjdk.java.net/jdk/pull/4263/files/ed49098a..7d18ca62

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=00-01

  Stats: 108 lines in 1 file changed: 57 ins; 39 del; 12 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4263.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4263/head:pull/4263

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Fri Jul  2 06:20:29 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Fri, 2 Jul 2021 06:20:29 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v3]
In-Reply-To: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
Message-ID: <aS5i72i7KLnTndIFIA_94OUjEgyuwXiHDHu1H898QtE=.0fcdd01c-2219-4318-86d7-8826941b6f41@github.com>

> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
> 
> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
> 
> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
> 
> I encourage everybody to discuss this draft:
> * Are there valid arguments for *not* doing this change?
> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
> * How to go on from here: What is missing to get this ready for an actual review?

Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision:

  Draft: Renaming i and separating code into several methods
  
  Signed-off-by: Markus Karg <markus at headcrashing.eu>

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4263/files
  - new: https://git.openjdk.java.net/jdk/pull/4263/files/7d18ca62..47ee00a2

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=01-02

  Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4263.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4263/head:pull/4263

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Fri Jul  2 06:20:30 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Fri, 2 Jul 2021 06:20:30 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v2]
In-Reply-To: <cZlX2_-SUyC7Ynxi-JQF2FM6qay9v_dbaD5W4Z-0a9Y=.d0bc0712-1645-4f38-967d-3bc2691f1c36@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <cZlX2_-SUyC7Ynxi-JQF2FM6qay9v_dbaD5W4Z-0a9Y=.d0bc0712-1645-4f38-967d-3bc2691f1c36@github.com>
Message-ID: <hawEb_Wv5KQa7KFHa4isTjiedxPp1g4J1PhFkEGbJSA=.2cfe4407-b8f4-4ece-b41b-ee31de7c87fa@github.com>

On Thu, 1 Jul 2021 21:59:33 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
>> 
>> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
>> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
>> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
>> 
>> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
>> 
>> I encourage everybody to discuss this draft:
>> * Are there valid arguments for *not* doing this change?
>> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
>> * How to go on from here: What is missing to get this ready for an actual review?
>
> Markus KARG has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Draft: Renaming i and separating code into several methods
>   
>   Signed-off-by: Markus Karg <markus at headcrashing.eu>

I have renamed `i` (and other ambiguous variables) and separated the implementation of `transferTo` into multiple methods. I will look into the test examples next to learn what I can reuse when writing tests for all those implementations. Thank you for all the kind input and the good ideas.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From alanb at openjdk.java.net  Fri Jul  2 09:17:03 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Fri, 2 Jul 2021 09:17:03 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v2]
In-Reply-To: <YPCqIo02ztuhCJjbbrGnaF_FomLwH8ZrRu0JGj3Nc40=.05cc10af-9962-425f-8b90-86261619822e@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <YPCqIo02ztuhCJjbbrGnaF_FomLwH8ZrRu0JGj3Nc40=.05cc10af-9962-425f-8b90-86261619822e@github.com>
Message-ID: <vnOKBNmlmQgX_5Q667LTvWpNrmWBBVRdz8L_TyQqC8o=.7a5ff6f9-592f-423e-ab3d-cb1a4e15c8c1@github.com>

On Thu, 1 Jul 2021 13:25:32 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review of this proposed fix for https://bugs.openjdk.java.net/browse/JDK-8251329?
>> 
>> As noted in that issue, if a zip filesystem created on top of a jar containing a "./" entry is "walked" using the `Files.walkFileTree`, it leads to a infinite never ending iteration (which ultimately fails with Java heap space OOM).
>> 
>> Alan notes in that issue that:
>> 
>>> This is more likely an issue with the zipfs DirectoryStream implementation. A DirectoryStream is specified to not include elements that for the special links to the current or parent directory. It should be rare. 
>> 
>> This indeed turned out to be an issue in the `jdk.nio.zipfs.ZipDirectoryStream#iterator()` implementation where it calls the `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` implementation.  The implementation, unlike what the javadoc of `java.nio.file.DirectoryStream` states, was including the "." and ".." paths in its returned iterator:
>> 
>>> The elements returned by the iterator are in no specific order. Some file
>>  systems maintain special links to the directory itself and the directory's
>>   parent directory. Entries representing these links are not returned by the
>>   iterator.
>> 
>> 
>> The proposed fix in this patch checks the paths for "." and "..", similar to what the `sun.nio.fs.UnixDirectoryStream` does in its `isSelfOrParent` and skips those paths from being added into the returned iterator. The `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` (where this change has been done) is currently only used by `jdk.nio.zipfs.ZipDirectoryStream#iterator()` and has package-private visibility, so this change shouldn't impact any other usage/expectations.
>> 
>> A new jtreg test has been added to reproduce this issue and verify the fix. Local testing of the `test/jdk/jdk/nio/` (including this new test) went fine without any issues after this change. I will be triggering a `tier1` test locally in a while.
>
> Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:
> 
>   implement review suggestion - move isSelfOrParent to ZipPath class

The implementation change look good.

I skimmed the test. A helper method that maps a Set<Path> to a modifiable Set<String> would avoid the 10 usages of toString() in the setup, just a suggestion. Also I was puzzled by fs.getRootDirectories().iterator().next() and why this isn't fs.getPath("/").

-------------

PR: https://git.openjdk.java.net/jdk/pull/4604

From jpai at openjdk.java.net  Fri Jul  2 10:25:29 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 2 Jul 2021 10:25:29 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v3]
In-Reply-To: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
Message-ID: <hUKNLIxq2DB8bCyqQxl7r4ZjNWgq8zsx_MdTyFuXk-I=.520ea0e0-7715-4172-8165-2825b8b39b40@github.com>

> Can I please get a review of this proposed fix for https://bugs.openjdk.java.net/browse/JDK-8251329?
> 
> As noted in that issue, if a zip filesystem created on top of a jar containing a "./" entry is "walked" using the `Files.walkFileTree`, it leads to a infinite never ending iteration (which ultimately fails with Java heap space OOM).
> 
> Alan notes in that issue that:
> 
>> This is more likely an issue with the zipfs DirectoryStream implementation. A DirectoryStream is specified to not include elements that for the special links to the current or parent directory. It should be rare. 
> 
> This indeed turned out to be an issue in the `jdk.nio.zipfs.ZipDirectoryStream#iterator()` implementation where it calls the `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` implementation.  The implementation, unlike what the javadoc of `java.nio.file.DirectoryStream` states, was including the "." and ".." paths in its returned iterator:
> 
>> The elements returned by the iterator are in no specific order. Some file
>  systems maintain special links to the directory itself and the directory's
>   parent directory. Entries representing these links are not returned by the
>   iterator.
> 
> 
> The proposed fix in this patch checks the paths for "." and "..", similar to what the `sun.nio.fs.UnixDirectoryStream` does in its `isSelfOrParent` and skips those paths from being added into the returned iterator. The `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` (where this change has been done) is currently only used by `jdk.nio.zipfs.ZipDirectoryStream#iterator()` and has package-private visibility, so this change shouldn't impact any other usage/expectations.
> 
> A new jtreg test has been added to reproduce this issue and verify the fix. Local testing of the `test/jdk/jdk/nio/` (including this new test) went fine without any issues after this change. I will be triggering a `tier1` test locally in a while.

Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:

  implement review suggestions:
   - reduce the toString() calls by creating a helper
   - fs.getPath("/") instead of fs.getRootDirectories().iterator().next()

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4604/files
  - new: https://git.openjdk.java.net/jdk/pull/4604/files/3971ad6f..8e77d289

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4604&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4604&range=01-02

  Stats: 18 lines in 1 file changed: 6 ins; 0 del; 12 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4604.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4604/head:pull/4604

PR: https://git.openjdk.java.net/jdk/pull/4604

From jpai at openjdk.java.net  Fri Jul  2 10:31:00 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 2 Jul 2021 10:31:00 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v3]
In-Reply-To: <hUKNLIxq2DB8bCyqQxl7r4ZjNWgq8zsx_MdTyFuXk-I=.520ea0e0-7715-4172-8165-2825b8b39b40@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <hUKNLIxq2DB8bCyqQxl7r4ZjNWgq8zsx_MdTyFuXk-I=.520ea0e0-7715-4172-8165-2825b8b39b40@github.com>
Message-ID: <KVe0boBSrHhaZJ_OP5uDp5yQsFzIOMVEyqQseJFdxp8=.461f56d7-db55-4b60-ac53-fd40ac3ca79b@github.com>

On Fri, 2 Jul 2021 10:25:29 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review of this proposed fix for https://bugs.openjdk.java.net/browse/JDK-8251329?
>> 
>> As noted in that issue, if a zip filesystem created on top of a jar containing a "./" entry is "walked" using the `Files.walkFileTree`, it leads to a infinite never ending iteration (which ultimately fails with Java heap space OOM).
>> 
>> Alan notes in that issue that:
>> 
>>> This is more likely an issue with the zipfs DirectoryStream implementation. A DirectoryStream is specified to not include elements that for the special links to the current or parent directory. It should be rare. 
>> 
>> This indeed turned out to be an issue in the `jdk.nio.zipfs.ZipDirectoryStream#iterator()` implementation where it calls the `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` implementation.  The implementation, unlike what the javadoc of `java.nio.file.DirectoryStream` states, was including the "." and ".." paths in its returned iterator:
>> 
>>> The elements returned by the iterator are in no specific order. Some file
>>  systems maintain special links to the directory itself and the directory's
>>   parent directory. Entries representing these links are not returned by the
>>   iterator.
>> 
>> 
>> The proposed fix in this patch checks the paths for "." and "..", similar to what the `sun.nio.fs.UnixDirectoryStream` does in its `isSelfOrParent` and skips those paths from being added into the returned iterator. The `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` (where this change has been done) is currently only used by `jdk.nio.zipfs.ZipDirectoryStream#iterator()` and has package-private visibility, so this change shouldn't impact any other usage/expectations.
>> 
>> A new jtreg test has been added to reproduce this issue and verify the fix. Local testing of the `test/jdk/jdk/nio/` (including this new test) went fine without any issues after this change. I will be triggering a `tier1` test locally in a while.
>
> Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:
> 
>   implement review suggestions:
>    - reduce the toString() calls by creating a helper
>    - fs.getPath("/") instead of fs.getRootDirectories().iterator().next()

>I skimmed the test. A helper method that maps a Set to a modifiable Set would avoid the 10 usages of toString() in the setup, just a suggestion.

Done. I introduced a helper in the updated version of the PR.

> Also I was puzzled by fs.getRootDirectories().iterator().next() and why this isn't fs.getPath("/").

When I started with a reproducer and then this test, I used the code that the reporter had attached in the JBS issue. That one had used "fs.getRootDirectories().iterator().next()" to show this issue. I just happened to carry that over into the test. I've updated the PR to now use fs.getPath("/") and also verified that this version of the test continues to fail without the fix proposed in this PR and passes with the changes.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4604

From jpai at openjdk.java.net  Fri Jul  2 11:06:40 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 2 Jul 2021 11:06:40 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
Message-ID: <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>

> Can I please get a review of this proposed fix for https://bugs.openjdk.java.net/browse/JDK-8251329?
> 
> As noted in that issue, if a zip filesystem created on top of a jar containing a "./" entry is "walked" using the `Files.walkFileTree`, it leads to a infinite never ending iteration (which ultimately fails with Java heap space OOM).
> 
> Alan notes in that issue that:
> 
>> This is more likely an issue with the zipfs DirectoryStream implementation. A DirectoryStream is specified to not include elements that for the special links to the current or parent directory. It should be rare. 
> 
> This indeed turned out to be an issue in the `jdk.nio.zipfs.ZipDirectoryStream#iterator()` implementation where it calls the `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` implementation.  The implementation, unlike what the javadoc of `java.nio.file.DirectoryStream` states, was including the "." and ".." paths in its returned iterator:
> 
>> The elements returned by the iterator are in no specific order. Some file
>  systems maintain special links to the directory itself and the directory's
>   parent directory. Entries representing these links are not returned by the
>   iterator.
> 
> 
> The proposed fix in this patch checks the paths for "." and "..", similar to what the `sun.nio.fs.UnixDirectoryStream` does in its `isSelfOrParent` and skips those paths from being added into the returned iterator. The `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` (where this change has been done) is currently only used by `jdk.nio.zipfs.ZipDirectoryStream#iterator()` and has package-private visibility, so this change shouldn't impact any other usage/expectations.
> 
> A new jtreg test has been added to reproduce this issue and verify the fix. Local testing of the `test/jdk/jdk/nio/` (including this new test) went fine without any issues after this change. I will be triggering a `tier1` test locally in a while.

Jaikiran Pai has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:

 - Merge latest from upstream master branch to bring in fixes that might help fix the unrelated tier1 failures in Github Actions job runs
 - implement review suggestions:
    - reduce the toString() calls by creating a helper
    - fs.getPath("/") instead of fs.getRootDirectories().iterator().next()
 - implement review suggestion - move isSelfOrParent to ZipPath class
 - 8251329: (zipfs) Files.walkFileTree walks infinitely if zip has dir named "." inside

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4604/files
  - new: https://git.openjdk.java.net/jdk/pull/4604/files/8e77d289..abae52bb

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4604&range=03
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4604&range=02-03

  Stats: 10322 lines in 336 files changed: 6220 ins; 2802 del; 1300 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4604.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4604/head:pull/4604

PR: https://git.openjdk.java.net/jdk/pull/4604

From lancea at openjdk.java.net  Fri Jul  2 11:12:04 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Fri, 2 Jul 2021 11:12:04 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
Message-ID: <FS4dG8muHl9vXW_eryRBZkDsRiquNnNOCLBlwsNVXeU=.1cecfec8-6a90-4cd4-a796-249c9b8c3110@github.com>

On Fri, 2 Jul 2021 11:06:40 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review of this proposed fix for https://bugs.openjdk.java.net/browse/JDK-8251329?
>> 
>> As noted in that issue, if a zip filesystem created on top of a jar containing a "./" entry is "walked" using the `Files.walkFileTree`, it leads to a infinite never ending iteration (which ultimately fails with Java heap space OOM).
>> 
>> Alan notes in that issue that:
>> 
>>> This is more likely an issue with the zipfs DirectoryStream implementation. A DirectoryStream is specified to not include elements that for the special links to the current or parent directory. It should be rare. 
>> 
>> This indeed turned out to be an issue in the `jdk.nio.zipfs.ZipDirectoryStream#iterator()` implementation where it calls the `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` implementation.  The implementation, unlike what the javadoc of `java.nio.file.DirectoryStream` states, was including the "." and ".." paths in its returned iterator:
>> 
>>> The elements returned by the iterator are in no specific order. Some file
>>  systems maintain special links to the directory itself and the directory's
>>   parent directory. Entries representing these links are not returned by the
>>   iterator.
>> 
>> 
>> The proposed fix in this patch checks the paths for "." and "..", similar to what the `sun.nio.fs.UnixDirectoryStream` does in its `isSelfOrParent` and skips those paths from being added into the returned iterator. The `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` (where this change has been done) is currently only used by `jdk.nio.zipfs.ZipDirectoryStream#iterator()` and has package-private visibility, so this change shouldn't impact any other usage/expectations.
>> 
>> A new jtreg test has been added to reproduce this issue and verify the fix. Local testing of the `test/jdk/jdk/nio/` (including this new test) went fine without any issues after this change. I will be triggering a `tier1` test locally in a while.
>
> Jaikiran Pai has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
> 
>  - Merge latest from upstream master branch to bring in fixes that might help fix the unrelated tier1 failures in Github Actions job runs
>  - implement review suggestions:
>     - reduce the toString() calls by creating a helper
>     - fs.getPath("/") instead of fs.getRootDirectories().iterator().next()
>  - implement review suggestion - move isSelfOrParent to ZipPath class
>  - 8251329: (zipfs) Files.walkFileTree walks infinitely if zip has dir named "." inside

Hi Jaikiran,

Consider:


        try (var os = Files.newOutputStream(ZIPFILE);
             ZipOutputStream zos = new ZipOutputStream(os)) {
            zos.putNextEntry(new ZipEntry("../Hello.txt"));
            zos.write("Hello World".getBytes(StandardCharsets.UTF_8));
        }


With your proposed fix, you will only return "/" when you walk the the Zip file and we should also return "/Hello.txt"

If. you resolve the path when the Inode entry is created:

           ` name(new ZipPath(null, normalize(name), true).getResolvedPath());`

That should address the issue and also allow you to access the entry.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4604

From jai.forums2013 at gmail.com  Fri Jul  2 12:08:43 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Fri, 2 Jul 2021 17:38:43 +0530
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <FS4dG8muHl9vXW_eryRBZkDsRiquNnNOCLBlwsNVXeU=.1cecfec8-6a90-4cd4-a796-249c9b8c3110@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
 <FS4dG8muHl9vXW_eryRBZkDsRiquNnNOCLBlwsNVXeU=.1cecfec8-6a90-4cd4-a796-249c9b8c3110@github.com>
Message-ID: <3d068121-9800-972c-dbd6-cd46b25f6dde@gmail.com>

Hello Lance,

On 02/07/21 4:42 pm, Lance Andersen wrote:
> Hi Jaikiran,
>
> Consider:
>
>
>          try (var os = Files.newOutputStream(ZIPFILE);
>               ZipOutputStream zos = new ZipOutputStream(os)) {
>              zos.putNextEntry(new ZipEntry("../Hello.txt"));
>              zos.write("Hello World".getBytes(StandardCharsets.UTF_8));
>          }
>
>
> With your proposed fix, you will only return "/" when you walk the the Zip file and we should also return "/Hello.txt"

Thank you for noticing this issue in my change and bringing this up. I 
have a question around this use case. Please consider a small variation 
to your example as below:

try (var os = Files.newOutputStream(ZIPFILE);
 ???????????? ZipOutputStream zos = new ZipOutputStream(os)) {
 ??????????? zos.putNextEntry(new ZipEntry("../Hello.txt"));
 ??????????? zos.write("Hello World".getBytes(StandardCharsets.UTF_8));
 ??????????? zos.closeEntry();
 ??????????? zos.putNextEntry(new ZipEntry("/Hello.txt"));
 ??? ??? ??? zos.write("Bye bye".getBytes(StandardCharsets.UTF_8));
 ??? ??? ??? zos.closeEntry();

 ??????? }

Notice that I now have a zip/jar which has 2 differently named entries 
"../Hello.txt" and "/Hello.txt". This creates the archive without any 
issues and those 2 entries are noted in its listing. Now assuming 
someone walks over this jar using the ZipFileSystem, starting at root 
("/"), what should be the expected output? The path(s) returned will end 
up being "/" and "/Hello.txt" but which resource is expected to be 
served in this case? The one which has "Hello World" in it or the one 
which has "Bye bye"? This example can be extended a bit more by 
introducing a "./Hello.txt", in the jar, with yet another different 
content in that entry. Is there some specification for this that I can 
check and adapt the test case accordingly?

-Jaikiran




From Alan.Bateman at oracle.com  Fri Jul  2 13:37:15 2021
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 2 Jul 2021 14:37:15 +0100
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <3d068121-9800-972c-dbd6-cd46b25f6dde@gmail.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
 <FS4dG8muHl9vXW_eryRBZkDsRiquNnNOCLBlwsNVXeU=.1cecfec8-6a90-4cd4-a796-249c9b8c3110@github.com>
 <3d068121-9800-972c-dbd6-cd46b25f6dde@gmail.com>
Message-ID: <f8a57ce5-4f67-ad37-ef4f-66777bb79257@oracle.com>

On 02/07/2021 13:08, Jaikiran Pai wrote:
>
> Thank you for noticing this issue in my change and bringing this up. I 
> have a question around this use case. Please consider a small 
> variation to your example as below:
>
> try (var os = Files.newOutputStream(ZIPFILE);
> ???????????? ZipOutputStream zos = new ZipOutputStream(os)) {
> ??????????? zos.putNextEntry(new ZipEntry("../Hello.txt"));
> ??????????? zos.write("Hello World".getBytes(StandardCharsets.UTF_8));
> ??????????? zos.closeEntry();
> ??????????? zos.putNextEntry(new ZipEntry("/Hello.txt"));
> ??? ??? ??? zos.write("Bye bye".getBytes(StandardCharsets.UTF_8));
> ??? ??? ??? zos.closeEntry();
>
> ??????? }
>
> Notice that I now have a zip/jar which has 2 differently named entries 
> "../Hello.txt" and "/Hello.txt". This creates the archive without any 
> issues and those 2 entries are noted in its listing. Now assuming 
> someone walks over this jar using the ZipFileSystem, starting at root 
> ("/"), what should be the expected output? The path(s) returned will 
> end up being "/" and "/Hello.txt" but which resource is expected to be 
> served in this case? The one which has "Hello World" in it or the one 
> which has "Bye bye"? This example can be extended a bit more by 
> introducing a "./Hello.txt", in the jar, with yet another different 
> content in that entry. Is there some specification for this that I can 
> check and adapt the test case accordingly?

This is an area where treating a zip file as a virtual file system 
doesn't quite work. We may have to look at zipfs ignoring entries that 
contain "." or ".." name elements or have it throw an exception if they 
are encountered. I think the main thing is that the APIs and access 
behaves consistently.

-Alan

From jpai at openjdk.java.net  Fri Jul  2 14:08:17 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 2 Jul 2021 14:08:17 GMT
Subject: RFR: 8233020: (fs) UnixFileSystemProvider should use
 StaticProperty.userDir().
Message-ID: <7jQFZOFRheEkl5XeEof4GWJ1tKvhcFIIPYQ_S7T5vJk=.f7b2a312-b999-4324-9f75-983cce410641@github.com>

Can I please get a review for this change which addresses https://bugs.openjdk.java.net/browse/JDK-8233020?

The commit in this PR updates `sun.nio.fs.UnixFileSystemProvider` to use the `jdk.internal.util.StaticProperty`, which was introduced in https://bugs.openjdk.java.net/browse/JDK-8066709, to get the value for the "user.dir" system property. Given the nature of this change, no new jtreg test has been added.

Alan notes in JDK-8233020:

> The changes in JDK-8066709 were okay to leave out the Unix file system provider because it was initialized very early in the startup at the time.

So, with input from Roger, I generated the classloading logs of a trivial helloworld application, before and after this change. I used `-verbose:class` to generate them.


public class Test {
	public static void main(final String[] args) throws Exception {
		System.out.println("Hello world");
	}
}


The classloading logs showed that the order, both before and after the change, has the `jdk.internal.util.StaticProperty` loaded way before the `sun.nio.fs.UnixFileSystemProvider`. Given the way `jdk.internal.util.StaticProperty` class gets loaded in phase 1 of java.lang.System initialization[1], this classloading order also implies that `jdk.internal.util.StaticProperty` is initialized before any instance of `sun.nio.fs.UnixFileSystemProvider` gets created. I also checked a few versions of JDKs - 11, 15, 16 and even 8 (which seems to have this backported) and none of them show the `sun.nio.fs.UnixFileSystemProvider` loading before the `StaticProperty`. I wasn't able to build the version of a JDK (due to build tool incompatibilities with my local system) before the change in JDK-8066709 was introduced. So I don't have any version of JDK to compare where `UnixFileSystemProvider` was perhaps being loaded way too early to cause any issue. So in short, on the ordering front, IMO this cha
 nge shouldn't cause any issues that I can think of. 

For the sake of reference, a snippet of the classloading logs before and after this change is provided at the end of this comment.

One other aspect to consider is the usage of "StaticProperty" itself. The javadoc of this class states:

>  <strong>{@link SecurityManager#checkPropertyAccess} is NOT checked
>  in these access methods. The caller of these methods should take care to ensure
>  that the returned property is not made accessible to untrusted code.</strong>
> 


In this specific case, the `sun.nio.fs.UnixFileSystemProvider` is a public class. It now uses this `StaticProperty` to get the "user.dir" system property value and passes it on to the `newFileSystem(String dir)` method of its sub-classes. However, the `newFileSystem` method that's being called has package-private access, so no application specific sub-classes or any code outside of the JDK can override it and get access to the passed "user.dir" system property value. So the absence of security manager check for the property access in this case won't pose an issue, IMO.

Following are the classloading logs:

Before:


line 252: [0.031s][info][class,load] jdk.internal.util.StaticProperty source: shared objects file
[0.031s][info][class,load] java.io.FileInputStream source: shared objects file
[0.031s][info][class,load] java.io.FileDescriptor source: shared objects file
[0.031s][info][class,load] jdk.internal.access.JavaIOFileDescriptorAccess source: shared objects file
....
[0.074s][info][class,load] java.lang.module.ModuleReader source: shared objects file
[0.074s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemModuleReader source: shared objects file
[0.074s][info][class,load] jdk.internal.module.ModulePatcher$PatchedModuleReader source: jrt:/java.base
[0.074s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemImage source: shared objects file
[0.074s][info][class,load] jdk.internal.jimage.ImageReaderFactory source: shared objects file
[0.074s][info][class,load] java.nio.file.Paths source: shared objects file
[0.074s][info][class,load] java.nio.file.FileSystems source: shared objects file
[0.074s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder source: shared objects file
[0.074s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder$1 source: shared objects file
[0.074s][info][class,load] sun.nio.fs.DefaultFileSystemProvider source: shared objects file
[0.074s][info][class,load] java.nio.file.spi.FileSystemProvider source: shared objects file
[0.074s][info][class,load] sun.nio.fs.AbstractFileSystemProvider source: shared objects file
line 579: [0.074s][info][class,load] sun.nio.fs.UnixFileSystemProvider source: shared objects file


After:


line 252: [0.037s][info][class,load] jdk.internal.util.StaticProperty source: shared objects file
[0.037s][info][class,load] java.io.FileInputStream source: shared objects file
[0.037s][info][class,load] java.io.FileDescriptor source: shared objects file
...
[0.081s][info][class,load] jdk.internal.module.ModulePatcher$PatchedModuleReader source: jrt:/java.base
[0.081s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemImage source: shared objects file
[0.081s][info][class,load] jdk.internal.jimage.ImageReaderFactory source: shared objects file
[0.081s][info][class,load] java.nio.file.Paths source: shared objects file
[0.081s][info][class,load] java.nio.file.FileSystems source: shared objects file
[0.081s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder source: shared objects file
[0.081s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder$1 source: shared objects file
[0.082s][info][class,load] sun.nio.fs.DefaultFileSystemProvider source: shared objects file
[0.082s][info][class,load] java.nio.file.spi.FileSystemProvider source: shared objects file
[0.082s][info][class,load] sun.nio.fs.AbstractFileSystemProvider source: shared objects file
line 579: [0.082s][info][class,load] sun.nio.fs.UnixFileSystemProvider source: shared objects file



[1] https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/lang/System.java#L2089

-------------

Commit messages:
 - 8233020: (fs) UnixFileSystemProvider should use StaticProperty.userDir()

Changes: https://git.openjdk.java.net/jdk/pull/4668/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4668&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8233020
  Stats: 4 lines in 1 file changed: 1 ins; 2 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4668.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4668/head:pull/4668

PR: https://git.openjdk.java.net/jdk/pull/4668

From lance.andersen at oracle.com  Fri Jul  2 16:13:36 2021
From: lance.andersen at oracle.com (Lance Andersen)
Date: Fri, 2 Jul 2021 16:13:36 +0000
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <3d068121-9800-972c-dbd6-cd46b25f6dde@gmail.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
 <FS4dG8muHl9vXW_eryRBZkDsRiquNnNOCLBlwsNVXeU=.1cecfec8-6a90-4cd4-a796-249c9b8c3110@github.com>
 <3d068121-9800-972c-dbd6-cd46b25f6dde@gmail.com>
Message-ID: <C30F6CF9-02F3-4604-8976-2425B8568326@oracle.com>



On Jul 2, 2021, at 8:08 AM, Jaikiran Pai <jai.forums2013 at gmail.com<mailto:jai.forums2013 at gmail.com>> wrote:

Hello Lance,

On 02/07/21 4:42 pm, Lance Andersen wrote:
Hi Jaikiran,

Consider:


        try (var os = Files.newOutputStream(ZIPFILE);
             ZipOutputStream zos = new ZipOutputStream(os)) {
            zos.putNextEntry(new ZipEntry("../Hello.txt"));
            zos.write("Hello World".getBytes(StandardCharsets.UTF_8));
        }


With your proposed fix, you will only return "/" when you walk the the Zip file and we should also return "/Hello.txt"

Thank you for noticing this issue in my change and bringing this up. I have a question around this use case. Please consider a small variation to your example as below:

try (var os = Files.newOutputStream(ZIPFILE);
             ZipOutputStream zos = new ZipOutputStream(os)) {
            zos.putNextEntry(new ZipEntry("../Hello.txt"));
            zos.write("Hello World".getBytes(StandardCharsets.UTF_8));
            zos.closeEntry();
            zos.putNextEntry(new ZipEntry("/Hello.txt"));
            zos.write("Bye bye".getBytes(StandardCharsets.UTF_8));
            zos.closeEntry();

        }

Notice that I now have a zip/jar which has 2 differently named entries "../Hello.txt" and "/Hello.txt". This creates the archive without any issues and those 2 entries are noted in its listing. Now assuming someone walks over this jar using the ZipFileSystem, starting at root ("/"), what should be the expected output? The path(s) returned will end up being "/" and "/Hello.txt" but which resource is expected to be served in this case? The one which has "Hello World" in it or the one which has "Bye bye"? This example can be extended a bit more by introducing a "./Hello.txt", in the jar, with yet another different content in that entry. Is there some specification for this that I can check and adapt the test case accordingly?


Consider Hello.zip created via:


try (var os = Files.newOutputStream(Path.of("Hello.zip"));
     ZipOutputStream zos = new ZipOutputStream(os)) {
    zos.putNextEntry(new ZipEntry("../Hello.txt"));
    zos.write("Hello".getBytes(StandardCharsets.UTF_8));
    zos.putNextEntry(new ZipEntry("Hello.txt"));
    zos.write("Another Hello".getBytes(StandardCharsets.UTF_8));

}

Winzip does not handle this case well.

InfoZip will :

-------------
unzip ../Hello.zip
Archive:  ../Hello.zip
warning:  skipped "../" path component(s) in ../Hello.txt
  inflating: Hello.txt
replace Hello.txt? [y]es, [n]o, [A]ll, [N]one, [r]ename: r
new name: foo.txt
  inflating: foo.txt
% ls
Hello.txt       foo.txt
 % cat Hello.txt
Hello
 % cat foo.txt
Another Hello
%
????????

This scenario is not well covered in the Zip spec, so we can do what we think is best.

 Personally, I am OK with resolving the path.  The odds of having multiple relative paths to the same file would be a mistake in creating the Zip.

That being said, if we want to follow Alan?s suggestion and throw an Exception, I am OK with that as well.

Either way, we currently cannot access the file via Zip FS due to the call to ZipPath::getResolvedPath() for all access and the path is only normalized when the Inodes are created.

Alan, do you have a specific preference?


-Jaikiran




[cid:E1C4E2F0-ECD0-4C9D-ADB4-B16CA7BCB7FC at home]



Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering
1 Network Drive
Burlington, MA 01803
Lance.Andersen at oracle.com<mailto:Lance.Andersen at oracle.com>



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210702/99c8e356/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: oracle_sig_logo.gif
Type: image/gif
Size: 658 bytes
Desc: oracle_sig_logo.gif
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210702/99c8e356/oracle_sig_logo.gif>

From lance.andersen at oracle.com  Fri Jul  2 17:16:39 2021
From: lance.andersen at oracle.com (Lance Andersen)
Date: Fri, 2 Jul 2021 17:16:39 +0000
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <C30F6CF9-02F3-4604-8976-2425B8568326@oracle.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
 <FS4dG8muHl9vXW_eryRBZkDsRiquNnNOCLBlwsNVXeU=.1cecfec8-6a90-4cd4-a796-249c9b8c3110@github.com>
 <3d068121-9800-972c-dbd6-cd46b25f6dde@gmail.com>
 <C30F6CF9-02F3-4604-8976-2425B8568326@oracle.com>
Message-ID: <A7AB0C56-5B42-44E3-89DD-C425F46FF84B@oracle.com>



On Jul 2, 2021, at 12:13 PM, Lance Andersen <lance.andersen at oracle.com<mailto:lance.andersen at oracle.com>> wrote:



On Jul 2, 2021, at 8:08 AM, Jaikiran Pai <jai.forums2013 at gmail.com<mailto:jai.forums2013 at gmail.com>> wrote:

Hello Lance,

On 02/07/21 4:42 pm, Lance Andersen wrote:
Hi Jaikiran,

Consider:


        try (var os = Files.newOutputStream(ZIPFILE);
             ZipOutputStream zos = new ZipOutputStream(os)) {
            zos.putNextEntry(new ZipEntry("../Hello.txt"));
            zos.write("Hello World".getBytes(StandardCharsets.UTF_8));
        }


With your proposed fix, you will only return "/" when you walk the the Zip file and we should also return "/Hello.txt"

Thank you for noticing this issue in my change and bringing this up. I have a question around this use case. Please consider a small variation to your example as below:

try (var os = Files.newOutputStream(ZIPFILE);
             ZipOutputStream zos = new ZipOutputStream(os)) {
            zos.putNextEntry(new ZipEntry("../Hello.txt"));
            zos.write("Hello World".getBytes(StandardCharsets.UTF_8));
            zos.closeEntry();
            zos.putNextEntry(new ZipEntry("/Hello.txt"));
            zos.write("Bye bye".getBytes(StandardCharsets.UTF_8));
            zos.closeEntry();

        }

Notice that I now have a zip/jar which has 2 differently named entries "../Hello.txt" and "/Hello.txt". This creates the archive without any issues and those 2 entries are noted in its listing. Now assuming someone walks over this jar using the ZipFileSystem, starting at root ("/"), what should be the expected output? The path(s) returned will end up being "/" and "/Hello.txt" but which resource is expected to be served in this case? The one which has "Hello World" in it or the one which has "Bye bye"? This example can be extended a bit more by introducing a "./Hello.txt", in the jar, with yet another different content in that entry. Is there some specification for this that I can check and adapt the test case accordingly?


Consider Hello.zip created via:


try (var os = Files.newOutputStream(Path.of("Hello.zip"));
     ZipOutputStream zos = new ZipOutputStream(os)) {
    zos.putNextEntry(new ZipEntry("../Hello.txt"));
    zos.write("Hello".getBytes(StandardCharsets.UTF_8));
    zos.putNextEntry(new ZipEntry("Hello.txt"));
    zos.write("Another Hello".getBytes(StandardCharsets.UTF_8));

}

Winzip does not handle this case well.

InfoZip will :

-------------
unzip ../Hello.zip
Archive:  ../Hello.zip
warning:  skipped "../" path component(s) in ../Hello.txt
  inflating: Hello.txt
replace Hello.txt? [y]es, [n]o, [A]ll, [N]one, [r]ename: r
new name: foo.txt
  inflating: foo.txt
% ls
Hello.txt       foo.txt
 % cat Hello.txt
Hello
 % cat foo.txt
Another Hello
%
????????

This scenario is not well covered in the Zip spec, so we can do what we think is best.

 Personally, I am OK with resolving the path.  The odds of having multiple relative paths to the same file would be a mistake in creating the Zip.

That being said, if we want to follow Alan?s suggestion and throw an Exception, I am OK with that as well.

Either way, we currently cannot access the file via Zip FS due to the call to ZipPath::getResolvedPath() for all access and the path is only normalized when the Inodes are created.

Alan, do you have a specific preference?

One more datapoint

Consider:


try (FileSystem zipfs =
             FileSystems.newFileSystem(Path.of("ZipFSHello.zip"), Map.of("create", "true"))) {
    Files.writeString(zipfs.getPath("HelloZipfs.txt"), "Hello");
    Files.writeString(zipfs.getPath("../HelloZipfs.txt"), "A ZipFS Hello");

}


The above will result in a Zip with a single entry of ?HelloZipfs.txt? and a value of ?A ZipFS Hello?

As mentioned in my previous comment, this is due to the use of ZipPath::getResolvedPath() which brings me back to the suggestion of doing the same when we create the Inodes given this is an a virtual FS.

HTH

Best
Lance


-Jaikiran




<oracle_sig_logo.gif>



Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering
1 Network Drive
Burlington, MA 01803
Lance.Andersen at oracle.com<mailto:Lance.Andersen at oracle.com>

[cid:E1C4E2F0-ECD0-4C9D-ADB4-B16CA7BCB7FC at home]



Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering
1 Network Drive
Burlington, MA 01803
Lance.Andersen at oracle.com<mailto:Lance.Andersen at oracle.com>



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210702/387bc602/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: oracle_sig_logo.gif
Type: image/gif
Size: 658 bytes
Desc: oracle_sig_logo.gif
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210702/387bc602/oracle_sig_logo-0001.gif>

From Alan.Bateman at oracle.com  Fri Jul  2 18:29:19 2021
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 2 Jul 2021 19:29:19 +0100
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <A7AB0C56-5B42-44E3-89DD-C425F46FF84B@oracle.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
 <FS4dG8muHl9vXW_eryRBZkDsRiquNnNOCLBlwsNVXeU=.1cecfec8-6a90-4cd4-a796-249c9b8c3110@github.com>
 <3d068121-9800-972c-dbd6-cd46b25f6dde@gmail.com>
 <C30F6CF9-02F3-4604-8976-2425B8568326@oracle.com>
 <A7AB0C56-5B42-44E3-89DD-C425F46FF84B@oracle.com>
Message-ID: <d3b474af-b9d4-b8c0-79aa-993e55d679f1@oracle.com>

On 02/07/2021 18:16, Lance Andersen wrote:
> :
>
> That being said, if we want to follow Alan?s suggestion and throw an Exception, I am OK with that as well.
>
> Either way, we currently cannot access the file via Zip FS due to the call to ZipPath::getResolvedPath() for all access and the path is only normalized when the Inodes are created.
>
> Alan, do you have a specific preference?
>
Not yet but if you tolerate these hostile entries then it means that all 
access with relative paths containing a "." or ".." element will be 
ambiguous. It will break relativize, normalize, and maybe other path 
operations. I assume it will break copy/move operations when the source 
is in a zip file and the target is in a non-zip file. I suspect it would 
also need to encode file names when open zip file for read/write access 
because the native file system is used for caching.

-Alan

From alanb at openjdk.java.net  Sat Jul  3 07:28:52 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Sat, 3 Jul 2021 07:28:52 GMT
Subject: RFR: 8233020: (fs) UnixFileSystemProvider should use
 StaticProperty.userDir().
In-Reply-To: <7jQFZOFRheEkl5XeEof4GWJ1tKvhcFIIPYQ_S7T5vJk=.f7b2a312-b999-4324-9f75-983cce410641@github.com>
References: <7jQFZOFRheEkl5XeEof4GWJ1tKvhcFIIPYQ_S7T5vJk=.f7b2a312-b999-4324-9f75-983cce410641@github.com>
Message-ID: <W7palUxVF8Ts6XVngXsYH_8eA4UT4f2hOJNlMnsWjLo=.535b3683-39da-4392-9822-ac3325308f7a@github.com>

On Fri, 2 Jul 2021 13:50:20 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> Can I please get a review for this change which addresses https://bugs.openjdk.java.net/browse/JDK-8233020?
> 
> The commit in this PR updates `sun.nio.fs.UnixFileSystemProvider` to use the `jdk.internal.util.StaticProperty`, which was introduced in https://bugs.openjdk.java.net/browse/JDK-8066709, to get the value for the "user.dir" system property. Given the nature of this change, no new jtreg test has been added.
> 
> Alan notes in JDK-8233020:
> 
>> The changes in JDK-8066709 were okay to leave out the Unix file system provider because it was initialized very early in the startup at the time.
> 
> So, with input from Roger, I generated the classloading logs of a trivial helloworld application, before and after this change. I used `-verbose:class` to generate them.
> 
> 
> public class Test {
> 	public static void main(final String[] args) throws Exception {
> 		System.out.println("Hello world");
> 	}
> }
> 
> 
> The classloading logs showed that the order, both before and after the change, has the `jdk.internal.util.StaticProperty` loaded way before the `sun.nio.fs.UnixFileSystemProvider`. Given the way `jdk.internal.util.StaticProperty` class gets loaded in phase 1 of java.lang.System initialization[1], this classloading order also implies that `jdk.internal.util.StaticProperty` is initialized before any instance of `sun.nio.fs.UnixFileSystemProvider` gets created. I also checked a few versions of JDKs - 11, 15, 16 and even 8 (which seems to have this backported) and none of them show the `sun.nio.fs.UnixFileSystemProvider` loading before the `StaticProperty`. I wasn't able to build the version of a JDK (due to build tool incompatibilities with my local system) before the change in JDK-8066709 was introduced. So I don't have any version of JDK to compare where `UnixFileSystemProvider` was perhaps being loaded way too early to cause any issue. So in short, on the ordering front, IMO this c
 hange shouldn't cause any issues that I can think of. 
> 
> For the sake of reference, a snippet of the classloading logs before and after this change is provided at the end of this comment.
> 
> One other aspect to consider is the usage of "StaticProperty" itself. The javadoc of this class states:
> 
>>  <strong>{@link SecurityManager#checkPropertyAccess} is NOT checked
>>  in these access methods. The caller of these methods should take care to ensure
>>  that the returned property is not made accessible to untrusted code.</strong>
>> 
> 
> 
> In this specific case, the `sun.nio.fs.UnixFileSystemProvider` is a public class. It now uses this `StaticProperty` to get the "user.dir" system property value and passes it on to the `newFileSystem(String dir)` method of its sub-classes. However, the `newFileSystem` method that's being called has package-private access, so no application specific sub-classes or any code outside of the JDK can override it and get access to the passed "user.dir" system property value. So the absence of security manager check for the property access in this case won't pose an issue, IMO.
> 
> Following are the classloading logs:
> 
> Before:
> 
> 
> line 252: [0.031s][info][class,load] jdk.internal.util.StaticProperty source: shared objects file
> [0.031s][info][class,load] java.io.FileInputStream source: shared objects file
> [0.031s][info][class,load] java.io.FileDescriptor source: shared objects file
> [0.031s][info][class,load] jdk.internal.access.JavaIOFileDescriptorAccess source: shared objects file
> ....
> [0.074s][info][class,load] java.lang.module.ModuleReader source: shared objects file
> [0.074s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemModuleReader source: shared objects file
> [0.074s][info][class,load] jdk.internal.module.ModulePatcher$PatchedModuleReader source: jrt:/java.base
> [0.074s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemImage source: shared objects file
> [0.074s][info][class,load] jdk.internal.jimage.ImageReaderFactory source: shared objects file
> [0.074s][info][class,load] java.nio.file.Paths source: shared objects file
> [0.074s][info][class,load] java.nio.file.FileSystems source: shared objects file
> [0.074s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder source: shared objects file
> [0.074s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder$1 source: shared objects file
> [0.074s][info][class,load] sun.nio.fs.DefaultFileSystemProvider source: shared objects file
> [0.074s][info][class,load] java.nio.file.spi.FileSystemProvider source: shared objects file
> [0.074s][info][class,load] sun.nio.fs.AbstractFileSystemProvider source: shared objects file
> line 579: [0.074s][info][class,load] sun.nio.fs.UnixFileSystemProvider source: shared objects file
> 
> 
> After:
> 
> 
> line 252: [0.037s][info][class,load] jdk.internal.util.StaticProperty source: shared objects file
> [0.037s][info][class,load] java.io.FileInputStream source: shared objects file
> [0.037s][info][class,load] java.io.FileDescriptor source: shared objects file
> ...
> [0.081s][info][class,load] jdk.internal.module.ModulePatcher$PatchedModuleReader source: jrt:/java.base
> [0.081s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemImage source: shared objects file
> [0.081s][info][class,load] jdk.internal.jimage.ImageReaderFactory source: shared objects file
> [0.081s][info][class,load] java.nio.file.Paths source: shared objects file
> [0.081s][info][class,load] java.nio.file.FileSystems source: shared objects file
> [0.081s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder source: shared objects file
> [0.081s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder$1 source: shared objects file
> [0.082s][info][class,load] sun.nio.fs.DefaultFileSystemProvider source: shared objects file
> [0.082s][info][class,load] java.nio.file.spi.FileSystemProvider source: shared objects file
> [0.082s][info][class,load] sun.nio.fs.AbstractFileSystemProvider source: shared objects file
> line 579: [0.082s][info][class,load] sun.nio.fs.UnixFileSystemProvider source: shared objects file
> 
> 
> 
> [1] https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/lang/System.java#L2089

Marked as reviewed by alanb (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk/pull/4668

From jpai at openjdk.java.net  Sat Jul  3 16:19:23 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Sat, 3 Jul 2021 16:19:23 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v4]
In-Reply-To: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
Message-ID: <DngsxlE20UMkL9y5TnLw25-JNl_59vHt5lDATmImLXU=.444fba1b-d80b-4002-956b-4bd41dcb4c43@github.com>

> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
> 
> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
> 
> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146

Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:

  Introduce a way to allow outputstream returned by ZipOutputStream to rollover into a temporary file during writes, if the data size exceeds a threshold

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4607/files
  - new: https://git.openjdk.java.net/jdk/pull/4607/files/f519aa47..5c5f2694

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=03
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=02-03

  Stats: 362 lines in 4 files changed: 330 ins; 18 del; 14 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4607.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4607/head:pull/4607

PR: https://git.openjdk.java.net/jdk/pull/4607

From jpai at openjdk.java.net  Sat Jul  3 16:19:25 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Sat, 3 Jul 2021 16:19:25 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v3]
In-Reply-To: <zbLFjaCUglo5tfRWa1pl1CvwRiorO0jIpbvdwp52I04=.98e2a64e-b9d7-4e02-b1bb-73795e95e501@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <zbLFjaCUglo5tfRWa1pl1CvwRiorO0jIpbvdwp52I04=.98e2a64e-b9d7-4e02-b1bb-73795e95e501@github.com>
Message-ID: <gUbGHIfxWL0JgKqLVXyX-hRcbBRPtYCMr2PMZy7_S4E=.4a9eb7d0-db93-4754-98a3-43fba7f0d3e9@github.com>

On Wed, 30 Jun 2021 16:05:40 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
>> 
>> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
>> 
>> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146
>
> Jaikiran Pai has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - an initial proposed test for testing compressed size entry greater than 2gb
>  - minor summary update on test

Based on the inputs received, I've now updated this PR to enhance the ZipFS to allow for rolling over the outputstream data, into a temporary file after reaching a threshold. Details follow:

- A new "tempFileThreshold" property has been introduced for ZipFileSystem. This property can be passed like other existing properties when creating the filesystem. The value of this property is expected to be a size in bytes and represents the threshold which will be used to decide whether and when to use a temporary file for outputstreams of zip file entries returned by the ZipFileSystem.

- The "tempFileThreshold" property is optional and if not set to any explicit value will default to 10MB. In other words, this feature is by default enabled, without the user having to do any specific configuration (not even the existing "useTempFile" property is requried to be set).

- To disable the threshold based temp file creation feature, a value of 0 or a negative value can be passed.

- A new (internal) `FileRolloverOutputStream` has been introduced in `ZipFileSystem` and forms the core of this enhancement. It extends the `ByteArrayOutputStream` and its write methods have the additional ability to rollover the current and subsequently written data into a temporary file representing that zip file entry.

- A new `ZipFSOutputStreamTest` has been added with the sole focus of verifying the usage of this new "tempFileThreshold" property.

- The previously added `LargeEntrySizeTest` and the manual `LargeCompressedEntrySizeTest` continue to stay and are mainly there to test the large file sizes and deflated sizes of the zip entries and verify that the original reported JBS issue is fixed by this enhancement. One important thing to note about these tests is that I've now removed the explicit "@requires" (memory) requirements, since after this enhancement (with "tempFileThreshold" by default enabled as in those tests), it no longer should require any specific high memory systems to run these tests.

There are still some decisions to be made:

1. Should we introduce this new property or should we enhance the existing "useTempFile" property to allow a size to be passed? Specifically, can we/should we use that existing property to allow users to set the following values:

	- "true", this would imply the temp file feature is enabled always irrespective of the zip entry size. This is how the value of "true" is currently handled before the patch in this PR. So no change in behaviour.

	- a byte size, represented as a `String` or an integer. This would imply that the user wants to enable the temp file feature, but only when the size or compressed size of the zip entry reaches a threshold specified by this value. This would mean that for sizes lesser than this the ZipFS implementation would use a ByteArrayOutputStream and would only rollover to a temp file when the threshold is reached.

	- "false", this would disable the temp file feature completely and outputstreams for zip entries of the ZipFS instance will always use ByteArrayOutputStream

	- value of "0" or "-1". This would be the same as specifying "false" value.

Using the existing property and just one property to control the temp file creation semantics will help avoid having to deal with 2 different properties ("useTempFile" and the "tempFileThreshold"). Plus it would also prevent any potentially conflicting user specified values for them. For example, we won't have to decide what to do when a user sets useTempFile=false and tempFileThreshold=1234.

If we do decide to introduce this new property then some small amount of additional work needs to be done in this implementation to make sure semantically conflicting values for these 2 properties are handled correctly. I decided to skip that part in this round of the PR till we reached a decision about the properties.

2. Given that this is a new enhancement, I believe this requires a CSR.

3. Should this PR be now linked to https://bugs.openjdk.java.net/browse/JDK-8011146 instead of https://bugs.openjdk.java.net/browse/JDK-8190753?

4. I've never previously created a manual test case. The `LargeCompressedEntrySizeTest` in this PR is expected to be a manual test case (given how long it might take to run on various different systems). The only difference between this test case and other jtreg automated tests is the absence of a `@test` on this one. Is this how manual tests are written or is there some other way?

-------------

PR: https://git.openjdk.java.net/jdk/pull/4607

From jpai at openjdk.java.net  Sun Jul  4 07:17:44 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Sun, 4 Jul 2021 07:17:44 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v5]
In-Reply-To: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
Message-ID: <Q42bQKkj0HlPbz6BtWxgD0m6x89OmBO8i6V27l9qStw=.b2b56b19-19b5-41cf-af8f-b7c2b79b8dde@github.com>

> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
> 
> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
> 
> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146

Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:

  propagate back the original checked IOException to the callers

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4607/files
  - new: https://git.openjdk.java.net/jdk/pull/4607/files/5c5f2694..a9dfd8cd

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=04
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=03-04

  Stats: 19 lines in 1 file changed: 12 ins; 0 del; 7 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4607.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4607/head:pull/4607

PR: https://git.openjdk.java.net/jdk/pull/4607

From Alan.Bateman at oracle.com  Sun Jul  4 15:17:26 2021
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Sun, 4 Jul 2021 16:17:26 +0100
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v3]
In-Reply-To: <gUbGHIfxWL0JgKqLVXyX-hRcbBRPtYCMr2PMZy7_S4E=.4a9eb7d0-db93-4754-98a3-43fba7f0d3e9@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <zbLFjaCUglo5tfRWa1pl1CvwRiorO0jIpbvdwp52I04=.98e2a64e-b9d7-4e02-b1bb-73795e95e501@github.com>
 <gUbGHIfxWL0JgKqLVXyX-hRcbBRPtYCMr2PMZy7_S4E=.4a9eb7d0-db93-4754-98a3-43fba7f0d3e9@github.com>
Message-ID: <d0d8e38b-2bbe-ae22-add9-644fa2339e3d@oracle.com>

On 03/07/2021 17:19, Jaikiran Pai wrote:
> :
>
>
> There are still some decisions to be made:
>
> 1. Should we introduce this new property or should we enhance the existing "useTempFile" property to allow a size to be passed?
It's an implementation tuning knob and highly unlikely to be in the 
environment specified to newFileSystem. So I think the simplest is to 
just leave it out, it can always be added at a later time if there is a 
new real need. The default that you've chosen (10MB) looks a reasonable 
value to start with.

BTW: The documented properties that can specified in the environment are 
in jdk.zipfs's module description. The useTempFile property is not 
listed here as it was never a documented/supported property that dates 
back to early iterations of the zip file system when it was a demo.

As the PR has now expanded to do JDK-8011146 (and it doesn't matter if 
you use that JBS issue or continue with JDK-8190753) then I think the 
priority should be to be confident with the security.

> 4. I've never previously created a manual test case. The `LargeCompressedEntrySizeTest` in this PR is expected to be a manual test case (given how long it might take to run on various different systems). The only difference between this test case and other jtreg automated tests is the absence of a `@test` on this one. Is this how manual tests are written or is there some other way?
>
We avoid manual tests as there is no guarantee that they will be run. So 
maybe we'll need to explore the scenario a bit further to see if there 
is some way to come up with an automated test. The jtreg foo for manual 
tests is `@run main/manual LargeCompressedEntrySizeTest`. You'll see a 
few examples in the test suite but I don't know if they are ever run.

-Alan

From jpai at openjdk.java.net  Mon Jul  5 04:30:27 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Mon, 5 Jul 2021 04:30:27 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v6]
In-Reply-To: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
Message-ID: <gwRe2sf5P1rr17Ue63VGQPMG2TvLPNvUlHCBg1jUFyA=.fefe46bd-fd6f-4dd6-84d1-1b32ee385884@github.com>

> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
> 
> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
> 
> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146

Jaikiran Pai has updated the pull request incrementally with three additional commits since the last revision:

 - remove no longer used constant
 - use jtreg's construct of manual test
 - Implement Alan's review suggestion - don't expose the threshold as a configuration property, instead set it internally to a specific value

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4607/files
  - new: https://git.openjdk.java.net/jdk/pull/4607/files/a9dfd8cd..50e64d74

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=05
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=04-05

  Stats: 33 lines in 2 files changed: 1 ins; 25 del; 7 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4607.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4607/head:pull/4607

PR: https://git.openjdk.java.net/jdk/pull/4607

From jpai at openjdk.java.net  Mon Jul  5 04:37:47 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Mon, 5 Jul 2021 04:37:47 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v7]
In-Reply-To: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
Message-ID: <g-lSUDwcKWnS0sjsg7eeeII_nuMvhes_WpDdiYijhK4=.3f9de433-95de-4b17-a49e-ff5a3ec222e0@github.com>

> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
> 
> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
> 
> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146

Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:

  minor update to comment on FileRolloverOutputStream class

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4607/files
  - new: https://git.openjdk.java.net/jdk/pull/4607/files/50e64d74..0fdbcea6

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=06
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=05-06

  Stats: 3 lines in 1 file changed: 0 ins; 3 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4607.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4607/head:pull/4607

PR: https://git.openjdk.java.net/jdk/pull/4607

From jai.forums2013 at gmail.com  Mon Jul  5 05:22:29 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Mon, 5 Jul 2021 10:52:29 +0530
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v3]
In-Reply-To: <d0d8e38b-2bbe-ae22-add9-644fa2339e3d@oracle.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <zbLFjaCUglo5tfRWa1pl1CvwRiorO0jIpbvdwp52I04=.98e2a64e-b9d7-4e02-b1bb-73795e95e501@github.com>
 <gUbGHIfxWL0JgKqLVXyX-hRcbBRPtYCMr2PMZy7_S4E=.4a9eb7d0-db93-4754-98a3-43fba7f0d3e9@github.com>
 <d0d8e38b-2bbe-ae22-add9-644fa2339e3d@oracle.com>
Message-ID: <d8759650-0932-e04a-857f-a902e3e16f2d@gmail.com>

Hello Alan,

On 04/07/21 8:47 pm, Alan Bateman wrote:
> On 03/07/2021 17:19, Jaikiran Pai wrote:
>> :
>>
>>
>> There are still some decisions to be made:
>>
>> 1. Should we introduce this new property or should we enhance the 
>> existing "useTempFile" property to allow a size to be passed?
> It's an implementation tuning knob and highly unlikely to be in the 
> environment specified to newFileSystem. So I think the simplest is to 
> just leave it out, it can always be added at a later time if there is 
> a new real need. The default that you've chosen (10MB) looks a 
> reasonable value to start with.
>
> BTW: The documented properties that can specified in the environment 
> are in jdk.zipfs's module description. The useTempFile property is not 
> listed here as it was never a documented/supported property that dates 
> back to early iterations of the zip file system when it was a demo.

I've updated the PR to make this an internal implementation detail/value 
and not expect it to be a configuration passed during the creation of 
the file system. This also allowed me to simplify some of this new code 
to no longer check for 0 or negative values for this property and other 
related semantics.


>
> As the PR has now expanded to do JDK-8011146 (and it doesn't matter if 
> you use that JBS issue or continue with JDK-8190753) then I think the 
> priority should be to be confident with the security.
>
Couple of things that I can think of when it comes to security is making 
sure the temp file is created in the right location and right set of 
permissions and making sure that the temp file isn't left behind after 
the filesystem is closed or after the zip entry is no longer relevant. 
When it comes to temp file creation, the code in this PR uses existing 
constructs/code that's present in this jdk.nio.zipfs.ZipFileSystem 
class. I will take a detailed look at this part once I get to my 
workstation later today. Are there any other security aspects that we 
need to think about?


>> 4. I've never previously created a manual test case. The 
>> `LargeCompressedEntrySizeTest` in this PR is expected to be a manual 
>> test case (given how long it might take to run on various different 
>> systems). The only difference between this test case and other jtreg 
>> automated tests is the absence of a `@test` on this one. Is this how 
>> manual tests are written or is there some other way?
>>
> We avoid manual tests as there is no guarantee that they will be run. 
> So maybe we'll need to explore the scenario a bit further to see if 
> there is some way to come up with an automated test. The jtreg foo for 
> manual tests is `@run main/manual LargeCompressedEntrySizeTest`. 
> You'll see a few examples in the test suite but I don't know if they 
> are ever run.

I have updated the PR to use jtreg's construct of @run testng/manual to 
mark this as a manual test. I will post the timing of this test case 
later today after I run the latest version locally and see how long it's 
taking.

-Jaikiran




From jpai at openjdk.java.net  Mon Jul  5 07:42:26 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Mon, 5 Jul 2021 07:42:26 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v8]
In-Reply-To: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
Message-ID: <ehuksZQILsWLXcuvxgtRGIq7Lqwl146QQpIVHanMoIs=.9cad7f15-24b8-4bf6-99db-cbf3ef3cccbf@github.com>

> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
> 
> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
> 
> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146

Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:

  reorganize the tests now that the temp file creation threshold isn't exposed as a user configurable value

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4607/files
  - new: https://git.openjdk.java.net/jdk/pull/4607/files/0fdbcea6..c1ec9e12

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=07
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=06-07

  Stats: 200 lines in 3 files changed: 1 ins; 184 del; 15 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4607.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4607/head:pull/4607

PR: https://git.openjdk.java.net/jdk/pull/4607

From jpai at openjdk.java.net  Mon Jul  5 13:57:53 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Mon, 5 Jul 2021 13:57:53 GMT
Subject: RFR: 8233020: (fs) UnixFileSystemProvider should use
 StaticProperty.userDir().
In-Reply-To: <7jQFZOFRheEkl5XeEof4GWJ1tKvhcFIIPYQ_S7T5vJk=.f7b2a312-b999-4324-9f75-983cce410641@github.com>
References: <7jQFZOFRheEkl5XeEof4GWJ1tKvhcFIIPYQ_S7T5vJk=.f7b2a312-b999-4324-9f75-983cce410641@github.com>
Message-ID: <_uXYtR8NAD5Vv22-nCRhxG_9N0iY9BQBjFsmZQhMwf8=.f0e69c3c-f144-4f42-9fa1-c2b2a743cfe7@github.com>

On Fri, 2 Jul 2021 13:50:20 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> Can I please get a review for this change which addresses https://bugs.openjdk.java.net/browse/JDK-8233020?
> 
> The commit in this PR updates `sun.nio.fs.UnixFileSystemProvider` to use the `jdk.internal.util.StaticProperty`, which was introduced in https://bugs.openjdk.java.net/browse/JDK-8066709, to get the value for the "user.dir" system property. Given the nature of this change, no new jtreg test has been added.
> 
> Alan notes in JDK-8233020:
> 
>> The changes in JDK-8066709 were okay to leave out the Unix file system provider because it was initialized very early in the startup at the time.
> 
> So, with input from Roger, I generated the classloading logs of a trivial helloworld application, before and after this change. I used `-verbose:class` to generate them.
> 
> 
> public class Test {
> 	public static void main(final String[] args) throws Exception {
> 		System.out.println("Hello world");
> 	}
> }
> 
> 
> The classloading logs showed that the order, both before and after the change, has the `jdk.internal.util.StaticProperty` loaded way before the `sun.nio.fs.UnixFileSystemProvider`. Given the way `jdk.internal.util.StaticProperty` class gets loaded in phase 1 of java.lang.System initialization[1], this classloading order also implies that `jdk.internal.util.StaticProperty` is initialized before any instance of `sun.nio.fs.UnixFileSystemProvider` gets created. I also checked a few versions of JDKs - 11, 15, 16 and even 8 (which seems to have this backported) and none of them show the `sun.nio.fs.UnixFileSystemProvider` loading before the `StaticProperty`. I wasn't able to build the version of a JDK (due to build tool incompatibilities with my local system) before the change in JDK-8066709 was introduced. So I don't have any version of JDK to compare where `UnixFileSystemProvider` was perhaps being loaded way too early to cause any issue. So in short, on the ordering front, IMO this c
 hange shouldn't cause any issues that I can think of. 
> 
> For the sake of reference, a snippet of the classloading logs before and after this change is provided at the end of this comment.
> 
> One other aspect to consider is the usage of "StaticProperty" itself. The javadoc of this class states:
> 
>>  <strong>{@link SecurityManager#checkPropertyAccess} is NOT checked
>>  in these access methods. The caller of these methods should take care to ensure
>>  that the returned property is not made accessible to untrusted code.</strong>
>> 
> 
> 
> In this specific case, the `sun.nio.fs.UnixFileSystemProvider` is a public class. It now uses this `StaticProperty` to get the "user.dir" system property value and passes it on to the `newFileSystem(String dir)` method of its sub-classes. However, the `newFileSystem` method that's being called has package-private access, so no application specific sub-classes or any code outside of the JDK can override it and get access to the passed "user.dir" system property value. So the absence of security manager check for the property access in this case won't pose an issue, IMO.
> 
> Following are the classloading logs:
> 
> Before:
> 
> 
> line 252: [0.031s][info][class,load] jdk.internal.util.StaticProperty source: shared objects file
> [0.031s][info][class,load] java.io.FileInputStream source: shared objects file
> [0.031s][info][class,load] java.io.FileDescriptor source: shared objects file
> [0.031s][info][class,load] jdk.internal.access.JavaIOFileDescriptorAccess source: shared objects file
> ....
> [0.074s][info][class,load] java.lang.module.ModuleReader source: shared objects file
> [0.074s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemModuleReader source: shared objects file
> [0.074s][info][class,load] jdk.internal.module.ModulePatcher$PatchedModuleReader source: jrt:/java.base
> [0.074s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemImage source: shared objects file
> [0.074s][info][class,load] jdk.internal.jimage.ImageReaderFactory source: shared objects file
> [0.074s][info][class,load] java.nio.file.Paths source: shared objects file
> [0.074s][info][class,load] java.nio.file.FileSystems source: shared objects file
> [0.074s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder source: shared objects file
> [0.074s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder$1 source: shared objects file
> [0.074s][info][class,load] sun.nio.fs.DefaultFileSystemProvider source: shared objects file
> [0.074s][info][class,load] java.nio.file.spi.FileSystemProvider source: shared objects file
> [0.074s][info][class,load] sun.nio.fs.AbstractFileSystemProvider source: shared objects file
> line 579: [0.074s][info][class,load] sun.nio.fs.UnixFileSystemProvider source: shared objects file
> 
> 
> After:
> 
> 
> line 252: [0.037s][info][class,load] jdk.internal.util.StaticProperty source: shared objects file
> [0.037s][info][class,load] java.io.FileInputStream source: shared objects file
> [0.037s][info][class,load] java.io.FileDescriptor source: shared objects file
> ...
> [0.081s][info][class,load] jdk.internal.module.ModulePatcher$PatchedModuleReader source: jrt:/java.base
> [0.081s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemImage source: shared objects file
> [0.081s][info][class,load] jdk.internal.jimage.ImageReaderFactory source: shared objects file
> [0.081s][info][class,load] java.nio.file.Paths source: shared objects file
> [0.081s][info][class,load] java.nio.file.FileSystems source: shared objects file
> [0.081s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder source: shared objects file
> [0.081s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder$1 source: shared objects file
> [0.082s][info][class,load] sun.nio.fs.DefaultFileSystemProvider source: shared objects file
> [0.082s][info][class,load] java.nio.file.spi.FileSystemProvider source: shared objects file
> [0.082s][info][class,load] sun.nio.fs.AbstractFileSystemProvider source: shared objects file
> line 579: [0.082s][info][class,load] sun.nio.fs.UnixFileSystemProvider source: shared objects file
> 
> 
> 
> [1] https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/lang/System.java#L2089

Thank you for the review, Alan.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4668

From jpai at openjdk.java.net  Mon Jul  5 13:57:53 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Mon, 5 Jul 2021 13:57:53 GMT
Subject: Integrated: 8233020: (fs) UnixFileSystemProvider should use
 StaticProperty.userDir().
In-Reply-To: <7jQFZOFRheEkl5XeEof4GWJ1tKvhcFIIPYQ_S7T5vJk=.f7b2a312-b999-4324-9f75-983cce410641@github.com>
References: <7jQFZOFRheEkl5XeEof4GWJ1tKvhcFIIPYQ_S7T5vJk=.f7b2a312-b999-4324-9f75-983cce410641@github.com>
Message-ID: <8rpMKayl0nyLIH0p8QDLXLAWmMVCEdN6fwY4RTrWAj8=.7c7ee8e8-f8f1-459d-8b5c-24b46a213479@github.com>

On Fri, 2 Jul 2021 13:50:20 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> Can I please get a review for this change which addresses https://bugs.openjdk.java.net/browse/JDK-8233020?
> 
> The commit in this PR updates `sun.nio.fs.UnixFileSystemProvider` to use the `jdk.internal.util.StaticProperty`, which was introduced in https://bugs.openjdk.java.net/browse/JDK-8066709, to get the value for the "user.dir" system property. Given the nature of this change, no new jtreg test has been added.
> 
> Alan notes in JDK-8233020:
> 
>> The changes in JDK-8066709 were okay to leave out the Unix file system provider because it was initialized very early in the startup at the time.
> 
> So, with input from Roger, I generated the classloading logs of a trivial helloworld application, before and after this change. I used `-verbose:class` to generate them.
> 
> 
> public class Test {
> 	public static void main(final String[] args) throws Exception {
> 		System.out.println("Hello world");
> 	}
> }
> 
> 
> The classloading logs showed that the order, both before and after the change, has the `jdk.internal.util.StaticProperty` loaded way before the `sun.nio.fs.UnixFileSystemProvider`. Given the way `jdk.internal.util.StaticProperty` class gets loaded in phase 1 of java.lang.System initialization[1], this classloading order also implies that `jdk.internal.util.StaticProperty` is initialized before any instance of `sun.nio.fs.UnixFileSystemProvider` gets created. I also checked a few versions of JDKs - 11, 15, 16 and even 8 (which seems to have this backported) and none of them show the `sun.nio.fs.UnixFileSystemProvider` loading before the `StaticProperty`. I wasn't able to build the version of a JDK (due to build tool incompatibilities with my local system) before the change in JDK-8066709 was introduced. So I don't have any version of JDK to compare where `UnixFileSystemProvider` was perhaps being loaded way too early to cause any issue. So in short, on the ordering front, IMO this c
 hange shouldn't cause any issues that I can think of. 
> 
> For the sake of reference, a snippet of the classloading logs before and after this change is provided at the end of this comment.
> 
> One other aspect to consider is the usage of "StaticProperty" itself. The javadoc of this class states:
> 
>>  <strong>{@link SecurityManager#checkPropertyAccess} is NOT checked
>>  in these access methods. The caller of these methods should take care to ensure
>>  that the returned property is not made accessible to untrusted code.</strong>
>> 
> 
> 
> In this specific case, the `sun.nio.fs.UnixFileSystemProvider` is a public class. It now uses this `StaticProperty` to get the "user.dir" system property value and passes it on to the `newFileSystem(String dir)` method of its sub-classes. However, the `newFileSystem` method that's being called has package-private access, so no application specific sub-classes or any code outside of the JDK can override it and get access to the passed "user.dir" system property value. So the absence of security manager check for the property access in this case won't pose an issue, IMO.
> 
> Following are the classloading logs:
> 
> Before:
> 
> 
> line 252: [0.031s][info][class,load] jdk.internal.util.StaticProperty source: shared objects file
> [0.031s][info][class,load] java.io.FileInputStream source: shared objects file
> [0.031s][info][class,load] java.io.FileDescriptor source: shared objects file
> [0.031s][info][class,load] jdk.internal.access.JavaIOFileDescriptorAccess source: shared objects file
> ....
> [0.074s][info][class,load] java.lang.module.ModuleReader source: shared objects file
> [0.074s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemModuleReader source: shared objects file
> [0.074s][info][class,load] jdk.internal.module.ModulePatcher$PatchedModuleReader source: jrt:/java.base
> [0.074s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemImage source: shared objects file
> [0.074s][info][class,load] jdk.internal.jimage.ImageReaderFactory source: shared objects file
> [0.074s][info][class,load] java.nio.file.Paths source: shared objects file
> [0.074s][info][class,load] java.nio.file.FileSystems source: shared objects file
> [0.074s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder source: shared objects file
> [0.074s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder$1 source: shared objects file
> [0.074s][info][class,load] sun.nio.fs.DefaultFileSystemProvider source: shared objects file
> [0.074s][info][class,load] java.nio.file.spi.FileSystemProvider source: shared objects file
> [0.074s][info][class,load] sun.nio.fs.AbstractFileSystemProvider source: shared objects file
> line 579: [0.074s][info][class,load] sun.nio.fs.UnixFileSystemProvider source: shared objects file
> 
> 
> After:
> 
> 
> line 252: [0.037s][info][class,load] jdk.internal.util.StaticProperty source: shared objects file
> [0.037s][info][class,load] java.io.FileInputStream source: shared objects file
> [0.037s][info][class,load] java.io.FileDescriptor source: shared objects file
> ...
> [0.081s][info][class,load] jdk.internal.module.ModulePatcher$PatchedModuleReader source: jrt:/java.base
> [0.081s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemImage source: shared objects file
> [0.081s][info][class,load] jdk.internal.jimage.ImageReaderFactory source: shared objects file
> [0.081s][info][class,load] java.nio.file.Paths source: shared objects file
> [0.081s][info][class,load] java.nio.file.FileSystems source: shared objects file
> [0.081s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder source: shared objects file
> [0.081s][info][class,load] java.nio.file.FileSystems$DefaultFileSystemHolder$1 source: shared objects file
> [0.082s][info][class,load] sun.nio.fs.DefaultFileSystemProvider source: shared objects file
> [0.082s][info][class,load] java.nio.file.spi.FileSystemProvider source: shared objects file
> [0.082s][info][class,load] sun.nio.fs.AbstractFileSystemProvider source: shared objects file
> line 579: [0.082s][info][class,load] sun.nio.fs.UnixFileSystemProvider source: shared objects file
> 
> 
> 
> [1] https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/lang/System.java#L2089

This pull request has now been integrated.

Changeset: fd4de1ed
Author:    Jaikiran Pai <jpai at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/fd4de1ed404640ee0e744f022bbfa89db200ef05
Stats:     4 lines in 1 file changed: 1 ins; 2 del; 1 mod

8233020: (fs) UnixFileSystemProvider should use StaticProperty.userDir().

Reviewed-by: alanb

-------------

PR: https://git.openjdk.java.net/jdk/pull/4668

From pconcannon at openjdk.java.net  Tue Jul  6 10:24:29 2021
From: pconcannon at openjdk.java.net (Patrick Concannon)
Date: Tue, 6 Jul 2021 10:24:29 GMT
Subject: RFR: 8253119: Remove the legacy PlainSocketImpl and
 PlainDatagramSocketImpl implementation [v6]
In-Reply-To: <ZceXxvb4Mw2_kmjYCpLlkgbQLqXek4zbfLy4rpkYXnA=.a38aec80-8338-4556-8786-12370864be59@github.com>
References: <ZceXxvb4Mw2_kmjYCpLlkgbQLqXek4zbfLy4rpkYXnA=.a38aec80-8338-4556-8786-12370864be59@github.com>
Message-ID: <xBVt4ToIsZo0iLBheZlh6ilATN_PcdmiMcl5KDZ5Ru4=.e5784c9d-7817-4316-b546-bafaaa53e1a8@github.com>

> Hi,
> 
> Could someone please review my changes for the removal of the legacy `PlainSocketImpl` and `PlainDatagramSocketImpl` implementations?
> 
> In JDK 13, JEP 353 provided a drop in replacement for the legacy `PlainSocketImpl` implementation. Since JDK 13, the `PlainSocketImpl` implementation was no longer used but included a mitigation mechanism to reduce compatibility risks in the form of a JDK-specific property `jdk.net.usePlainSocketImpl` allowing to switch back to the old implementation. 
> Similarly, in JDK 15, JEP 373 provided a new implementation for `DatagramSocket` and `MulticastSocket`, with a JDK-specific property `jdk.net.usePlainDatagramSocketImpl` also allowing the user to switch back to the old implementation in case of compatibility issue.
> 
> As these implementations (and the mechanisms they use to enable them to mitigate compatibility issues) have been deemed no longer necessary, they now represent a maintenance burden. This patch looks at removing them from the JDK.
> 
> Kind regards,
> Patrick

Patrick Concannon has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:

 - Merge remote-tracking branch 'origin/master' into JDK-8253119
 - 8253119: Fixed typo
 - 8253119: Added message to null check in java/net/DatagramSocket
 - Merge remote-tracking branch 'origin/master' into JDK-8253119
 - 8253119: Removed redundant comments and USE_PLAINDATAGRAMSOCKET from java/net/DatagramSocket
 - Merge remote-tracking branch 'origin/master' into JDK-8253119
 - 8253119: Remove the legacy PlainSocketImpl and PlainDatagramSocketImpl implementation

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4574/files
  - new: https://git.openjdk.java.net/jdk/pull/4574/files/686b4369..538b145e

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4574&range=05
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4574&range=04-05

  Stats: 12821 lines in 436 files changed: 7154 ins; 3504 del; 2163 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4574.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4574/head:pull/4574

PR: https://git.openjdk.java.net/jdk/pull/4574

From pconcannon at openjdk.java.net  Tue Jul  6 13:50:53 2021
From: pconcannon at openjdk.java.net (Patrick Concannon)
Date: Tue, 6 Jul 2021 13:50:53 GMT
Subject: Integrated: 8253119: Remove the legacy PlainSocketImpl and
 PlainDatagramSocketImpl implementation
In-Reply-To: <ZceXxvb4Mw2_kmjYCpLlkgbQLqXek4zbfLy4rpkYXnA=.a38aec80-8338-4556-8786-12370864be59@github.com>
References: <ZceXxvb4Mw2_kmjYCpLlkgbQLqXek4zbfLy4rpkYXnA=.a38aec80-8338-4556-8786-12370864be59@github.com>
Message-ID: <Zk8-H_v4_lAPs7eLhDTQ_0NSTYUZDuBwXWjErPR6pnQ=.a85bb9cb-db9b-4b6b-ade3-3d2fafe5abfe@github.com>

On Wed, 23 Jun 2021 12:06:41 GMT, Patrick Concannon <pconcannon at openjdk.org> wrote:

> Hi,
> 
> Could someone please review my changes for the removal of the legacy `PlainSocketImpl` and `PlainDatagramSocketImpl` implementations?
> 
> In JDK 13, JEP 353 provided a drop in replacement for the legacy `PlainSocketImpl` implementation. Since JDK 13, the `PlainSocketImpl` implementation was no longer used but included a mitigation mechanism to reduce compatibility risks in the form of a JDK-specific property `jdk.net.usePlainSocketImpl` allowing to switch back to the old implementation. 
> Similarly, in JDK 15, JEP 373 provided a new implementation for `DatagramSocket` and `MulticastSocket`, with a JDK-specific property `jdk.net.usePlainDatagramSocketImpl` also allowing the user to switch back to the old implementation in case of compatibility issue.
> 
> As these implementations (and the mechanisms they use to enable them to mitigate compatibility issues) have been deemed no longer necessary, they now represent a maintenance burden. This patch looks at removing them from the JDK.
> 
> Kind regards,
> Patrick

This pull request has now been integrated.

Changeset: 326b2e13
Author:    Patrick Concannon <pconcannon at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/326b2e13447d734f84271942cc8154e30486fa7d
Stats:     11059 lines in 78 files changed: 136 ins; 10853 del; 70 mod

8253119: Remove the legacy PlainSocketImpl and PlainDatagramSocketImpl implementation

Reviewed-by: alanb, dfuchs, chegar

-------------

PR: https://git.openjdk.java.net/jdk/pull/4574

From chegar at openjdk.java.net  Tue Jul  6 14:51:06 2021
From: chegar at openjdk.java.net (Chris Hegarty)
Date: Tue, 6 Jul 2021 14:51:06 GMT
Subject: RFR: 8269917: Insert missing commas in copyrights in java.net
In-Reply-To: <Wo1CPYsCE8OEzSvxLYUShKm3ms1yCsKz5slDulKdvhg=.eb30b10a-46c2-4e22-831a-77cedfaceab2@github.com>
References: <Wo1CPYsCE8OEzSvxLYUShKm3ms1yCsKz5slDulKdvhg=.eb30b10a-46c2-4e22-831a-77cedfaceab2@github.com>
Message-ID: <lAxM_v9VB0TwalX0jx_y61fmVAPwsIPZHCHw4hr6NsA=.c76d5211-30f4-4c2d-bc21-752c00c134b9@github.com>

On Tue, 6 Jul 2021 14:41:18 GMT, Patrick Concannon <pconcannon at openjdk.org> wrote:

> Hi,
> 
> Could someone please review my patch for Inserting missing commas in copyright headers from files in java.net. These commas were mistakenly left out of PR https://github.com/openjdk/jdk/pull/4574.
> 
> Kind regards,
> 
> Patrick

Marked as reviewed by chegar (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk/pull/4690

From pconcannon at openjdk.java.net  Tue Jul  6 14:51:05 2021
From: pconcannon at openjdk.java.net (Patrick Concannon)
Date: Tue, 6 Jul 2021 14:51:05 GMT
Subject: RFR: 8269917: Insert missing commas in copyrights in java.net
Message-ID: <Wo1CPYsCE8OEzSvxLYUShKm3ms1yCsKz5slDulKdvhg=.eb30b10a-46c2-4e22-831a-77cedfaceab2@github.com>

Hi,

Could someone please review my patch for Inserting missing commas in copyright headers from files in java.net. These commas were mistakenly left out of PR https://github.com/openjdk/jdk/pull/4574.

Kind regards,

Patrick

-------------

Commit messages:
 - 8269917: Insert missing commas in copyrights in java.net

Changes: https://git.openjdk.java.net/jdk/pull/4690/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4690&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8269917
  Stats: 20 lines in 20 files changed: 0 ins; 0 del; 20 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4690.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4690/head:pull/4690

PR: https://git.openjdk.java.net/jdk/pull/4690

From dfuchs at openjdk.java.net  Tue Jul  6 14:51:06 2021
From: dfuchs at openjdk.java.net (Daniel Fuchs)
Date: Tue, 6 Jul 2021 14:51:06 GMT
Subject: RFR: 8269917: Insert missing commas in copyrights in java.net
In-Reply-To: <Wo1CPYsCE8OEzSvxLYUShKm3ms1yCsKz5slDulKdvhg=.eb30b10a-46c2-4e22-831a-77cedfaceab2@github.com>
References: <Wo1CPYsCE8OEzSvxLYUShKm3ms1yCsKz5slDulKdvhg=.eb30b10a-46c2-4e22-831a-77cedfaceab2@github.com>
Message-ID: <ZhZz9Kas6VrlJ-u72EHbQtSGEIR6u3Px2yk2UrPdooU=.a0a72c69-29f2-43c7-a152-f11da3c1026e@github.com>

On Tue, 6 Jul 2021 14:41:18 GMT, Patrick Concannon <pconcannon at openjdk.org> wrote:

> Hi,
> 
> Could someone please review my patch for Inserting missing commas in copyright headers from files in java.net. These commas were mistakenly left out of PR https://github.com/openjdk/jdk/pull/4574.
> 
> Kind regards,
> 
> Patrick

LGTM

-------------

Marked as reviewed by dfuchs (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/4690

From pconcannon at openjdk.java.net  Tue Jul  6 15:13:49 2021
From: pconcannon at openjdk.java.net (Patrick Concannon)
Date: Tue, 6 Jul 2021 15:13:49 GMT
Subject: Integrated: 8269917: Insert missing commas in copyrights in java.net
In-Reply-To: <Wo1CPYsCE8OEzSvxLYUShKm3ms1yCsKz5slDulKdvhg=.eb30b10a-46c2-4e22-831a-77cedfaceab2@github.com>
References: <Wo1CPYsCE8OEzSvxLYUShKm3ms1yCsKz5slDulKdvhg=.eb30b10a-46c2-4e22-831a-77cedfaceab2@github.com>
Message-ID: <EQY2b78CFc9jrwmzUuuwyomnJ1Yp4lsjkMNzpEZ_VcU=.67605d99-6363-41c2-ba77-9f260522fa3e@github.com>

On Tue, 6 Jul 2021 14:41:18 GMT, Patrick Concannon <pconcannon at openjdk.org> wrote:

> Hi,
> 
> Could someone please review my patch for Inserting missing commas in copyright headers from files in java.net. These commas were mistakenly left out of PR https://github.com/openjdk/jdk/pull/4574.
> 
> Kind regards,
> 
> Patrick

This pull request has now been integrated.

Changeset: 1578979d
Author:    Patrick Concannon <pconcannon at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/1578979df706ae9f4324931e1fc0d58265762c79
Stats:     20 lines in 20 files changed: 0 ins; 0 del; 20 mod

8269917: Insert missing commas in copyrights in java.net

Reviewed-by: chegar, dfuchs

-------------

PR: https://git.openjdk.java.net/jdk/pull/4690

From github.com+10835776+stsypanov at openjdk.java.net  Thu Jul  8 09:32:02 2021
From: github.com+10835776+stsypanov at openjdk.java.net (=?UTF-8?B?0KHQtdGA0LPQtdC5?= =?UTF-8?B?IA==?= =?UTF-8?B?0KbRi9C/0LDQvdC+0LI=?=)
Date: Thu, 8 Jul 2021 09:32:02 GMT
Subject: RFR: 8263561: Re-examine uses of LinkedList [v4]
In-Reply-To: <eCQ8oFDjAJgFR3SG0K7GlaM1lxivFOkQGkIi-i7I7Xw=.bd379708-2bb4-4d52-81e3-175fb7a9c46b@github.com>
References: <eCQ8oFDjAJgFR3SG0K7GlaM1lxivFOkQGkIi-i7I7Xw=.bd379708-2bb4-4d52-81e3-175fb7a9c46b@github.com>
Message-ID: <HKWDbXjH2i_rbOzyRaCheEGZCop9nB5hkfdOUlGMCQU=.72d4db7e-f057-4151-913b-f836da3262ce@github.com>

> After I've renamed remove branch GitHub for some reason has closed original https://github.com/openjdk/jdk/pull/2744, so I've decided to recreate it.

?????? ??????? has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits:

 - Merge branch 'master' into 8263561
 - Merge branch 'master' into 8263561
 - Merge branch 'master' into 8263561
 - Merge branch 'master' into 8263561
   
   # Conflicts:
   #	src/java.base/unix/classes/sun/net/dns/ResolverConfigurationImpl.java
 - Merge branch 'master' into purge-linked-list
 - 8263561: Use sized constructor where reasonable
 - 8263561: Use interface List instead of particular type where possible
 - 8263561: Rename requestList -> requests
 - 8263561: Re-examine uses of LinkedList

-------------

Changes: https://git.openjdk.java.net/jdk/pull/4304/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4304&range=03
  Stats: 48 lines in 9 files changed: 0 ins; 2 del; 46 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4304.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4304/head:pull/4304

PR: https://git.openjdk.java.net/jdk/pull/4304

From sebastian.stenzel at gmail.com  Thu Jul  8 14:54:29 2021
From: sebastian.stenzel at gmail.com (Sebastian Stenzel)
Date: Thu, 8 Jul 2021 16:54:29 +0200
Subject: Make java.nio.ByteBuffer a sealed class
Message-ID: <EFF59E15-951F-428D-8905-BF4743FD534B@gmail.com>

A while back, when working on JDK-8264110, we had a discussion about the nature of ByteBuffers, where I asked whether we could eventually make this a sealed class. [1]

The idea is, that basically there are only two kinds of ByteBuffers:
- HeapByteBuffers
- MappedByteBuffer ? DirectByteBuffer

While the former is backed by an array, the latter isn't, which leads to branching such as `if (buffer.hasArray())`, which could be simplified, if we knew for sure to deal with either of the two kinds.

Now, that sealed classes are stable with JDK 17, I'd like to ask for your approval to make ByteBuffer a sealed class, so we can rely on an exhaustive list of ByteBuffer "flavours" at compile time.

This is a requirement for follow-up tasks, I'd like to work on, such as JDK-8264341.


[1] https://github.com/openjdk/jdk/pull/3217#discussion_r602528233

From mullan at openjdk.java.net  Thu Jul  8 18:28:12 2021
From: mullan at openjdk.java.net (Sean Mullan)
Date: Thu, 8 Jul 2021 18:28:12 GMT
Subject: [jdk17] RFR: 8266345: (fs) Custom DefaultFileSystemProvider security
 related loops
Message-ID: <V4RIZ2QAVoDuK-mc4hnsFn6j5HkfFj0lfxaFqAFf34s=.68b08383-7cd1-4af9-bd8e-657293351338@github.com>

Please review this fix to use the platform's default file system to avoid recursive policy initialization
issues when a SecurityManager is enabled and the VM is configured to use a custom file system provider.

-------------

Commit messages:
 - 8266345: (fs) Custom DefaultFileSystemProvider security related loops

Changes: https://git.openjdk.java.net/jdk17/pull/232/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk17&pr=232&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8266345
  Stats: 30 lines in 3 files changed: 28 ins; 0 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk17/pull/232.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk17 pull/232/head:pull/232

PR: https://git.openjdk.java.net/jdk17/pull/232

From bpb at openjdk.java.net  Thu Jul  8 18:39:54 2021
From: bpb at openjdk.java.net (Brian Burkhalter)
Date: Thu, 8 Jul 2021 18:39:54 GMT
Subject: [jdk17] RFR: 8266345: (fs) Custom DefaultFileSystemProvider
 security related loops
In-Reply-To: <V4RIZ2QAVoDuK-mc4hnsFn6j5HkfFj0lfxaFqAFf34s=.68b08383-7cd1-4af9-bd8e-657293351338@github.com>
References: <V4RIZ2QAVoDuK-mc4hnsFn6j5HkfFj0lfxaFqAFf34s=.68b08383-7cd1-4af9-bd8e-657293351338@github.com>
Message-ID: <rafD_e7oU7y1_4N0E13hB8iYBHJkjq41vncTB4PMXHQ=.e21d0d4f-c654-4130-8ec1-ade1142b1e25@github.com>

On Thu, 8 Jul 2021 18:21:31 GMT, Sean Mullan <mullan at openjdk.org> wrote:

> Please review this fix to use the platform's default file system to avoid recursive policy initialization
> issues when a SecurityManager is enabled and the VM is configured to use a custom file system provider.

Looks good.

-------------

Marked as reviewed by bpb (Reviewer).

PR: https://git.openjdk.java.net/jdk17/pull/232

From jai.forums2013 at gmail.com  Fri Jul  9 12:43:44 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Fri, 9 Jul 2021 18:13:44 +0530
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v3]
In-Reply-To: <d8759650-0932-e04a-857f-a902e3e16f2d@gmail.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <zbLFjaCUglo5tfRWa1pl1CvwRiorO0jIpbvdwp52I04=.98e2a64e-b9d7-4e02-b1bb-73795e95e501@github.com>
 <gUbGHIfxWL0JgKqLVXyX-hRcbBRPtYCMr2PMZy7_S4E=.4a9eb7d0-db93-4754-98a3-43fba7f0d3e9@github.com>
 <d0d8e38b-2bbe-ae22-add9-644fa2339e3d@oracle.com>
 <d8759650-0932-e04a-857f-a902e3e16f2d@gmail.com>
Message-ID: <ec5c75c3-93fe-9f2a-a31f-ff2e8be477d9@gmail.com>


On 05/07/21 10:52 am, Jaikiran Pai wrote:
>
>>> 4. I've never previously created a manual test case. The 
>>> `LargeCompressedEntrySizeTest` in this PR is expected to be a manual 
>>> test case (given how long it might take to run on various different 
>>> systems). The only difference between this test case and other jtreg 
>>> automated tests is the absence of a `@test` on this one. Is this how 
>>> manual tests are written or is there some other way?
>>>
>> We avoid manual tests as there is no guarantee that they will be run. 
>> So maybe we'll need to explore the scenario a bit further to see if 
>> there is some way to come up with an automated test. The jtreg foo 
>> for manual tests is `@run main/manual LargeCompressedEntrySizeTest`. 
>> You'll see a few examples in the test suite but I don't know if they 
>> are ever run.
>
> I have updated the PR to use jtreg's construct of @run testng/manual 
> to mark this as a manual test. I will post the timing of this test 
> case later today after I run the latest version locally and see how 
> long it's taking.
>
On my local setup, the LargeCompressedEntrySizeTest (latest version of 
this PR) consistently takes between 205 to 215 seconds to complete (so 
between 3 to 4 minutes). Is that something that will allow it to be 
added as part of automated tests or is it too long for automated tests?

-Jaikiran


From lance.andersen at oracle.com  Fri Jul  9 16:11:48 2021
From: lance.andersen at oracle.com (Lance Andersen)
Date: Fri, 9 Jul 2021 16:11:48 +0000
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v3]
In-Reply-To: <ec5c75c3-93fe-9f2a-a31f-ff2e8be477d9@gmail.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <zbLFjaCUglo5tfRWa1pl1CvwRiorO0jIpbvdwp52I04=.98e2a64e-b9d7-4e02-b1bb-73795e95e501@github.com>
 <gUbGHIfxWL0JgKqLVXyX-hRcbBRPtYCMr2PMZy7_S4E=.4a9eb7d0-db93-4754-98a3-43fba7f0d3e9@github.com>
 <d0d8e38b-2bbe-ae22-add9-644fa2339e3d@oracle.com>
 <d8759650-0932-e04a-857f-a902e3e16f2d@gmail.com>
 <ec5c75c3-93fe-9f2a-a31f-ff2e8be477d9@gmail.com>
Message-ID: <0FDCCD5F-4D35-46E8-B5E1-DEE83CC663A8@oracle.com>

Hi Jaikiran,

On Jul 9, 2021, at 8:43 AM, Jaikiran Pai <jai.forums2013 at gmail.com<mailto:jai.forums2013 at gmail.com>> wrote:


On 05/07/21 10:52 am, Jaikiran Pai wrote:

4. I've never previously created a manual test case. The `LargeCompressedEntrySizeTest` in this PR is expected to be a manual test case (given how long it might take to run on various different systems). The only difference between this test case and other jtreg automated tests is the absence of a `@test` on this one. Is this how manual tests are written or is there some other way?

We avoid manual tests as there is no guarantee that they will be run. So maybe we'll need to explore the scenario a bit further to see if there is some way to come up with an automated test. The jtreg foo for manual tests is `@run main/manual LargeCompressedEntrySizeTest`. You'll see a few examples in the test suite but I don't know if they are ever run.

I have updated the PR to use jtreg's construct of @run testng/manual to mark this as a manual test. I will post the timing of this test case later today after I run the latest version locally and see how long it's taking.

On my local setup, the LargeCompressedEntrySizeTest (latest version of this PR) consistently takes between 205 to 215 seconds to complete (so between 3 to 4 minutes). Is that something that will allow it to be added as part of automated tests or is it too long for automated tests?

Thank you for the info.  This test needs to remain a manual test.

I have a full test run going and will finish my review later today or tomorrow

Best
Lance

-Jaikiran

[cid:E1C4E2F0-ECD0-4C9D-ADB4-B16CA7BCB7FC at home]



Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering
1 Network Drive
Burlington, MA 01803
Lance.Andersen at oracle.com<mailto:Lance.Andersen at oracle.com>



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210709/736d0918/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: oracle_sig_logo.gif
Type: image/gif
Size: 658 bytes
Desc: oracle_sig_logo.gif
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210709/736d0918/oracle_sig_logo.gif>

From lancea at openjdk.java.net  Sun Jul 11 20:38:55 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Sun, 11 Jul 2021 20:38:55 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v8]
In-Reply-To: <ehuksZQILsWLXcuvxgtRGIq7Lqwl146QQpIVHanMoIs=.9cad7f15-24b8-4bf6-99db-cbf3ef3cccbf@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <ehuksZQILsWLXcuvxgtRGIq7Lqwl146QQpIVHanMoIs=.9cad7f15-24b8-4bf6-99db-cbf3ef3cccbf@github.com>
Message-ID: <vhRP_xqkq0g3topmOPn1JdTpaXtEDTKuI72fdNd6hSw=.86d4f9c6-197a-48de-94a5-0cbe54ad41ff@github.com>

On Mon, 5 Jul 2021 07:42:26 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
>> 
>> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
>> 
>> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146
>
> Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:
> 
>   reorganize the tests now that the temp file creation threshold isn't exposed as a user configurable value

I think the updates made to Zip FS look better.   Alan is on vacation so I would prefer to wait until he gets back and give him a chance to provide any last thoughts on the change to Zip FS.

The manual test looks OK and is a good addition

-------------

Marked as reviewed by lancea (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/4607

From jai.forums2013 at gmail.com  Mon Jul 12 01:19:47 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Mon, 12 Jul 2021 06:49:47 +0530
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v8]
In-Reply-To: <vhRP_xqkq0g3topmOPn1JdTpaXtEDTKuI72fdNd6hSw=.86d4f9c6-197a-48de-94a5-0cbe54ad41ff@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <ehuksZQILsWLXcuvxgtRGIq7Lqwl146QQpIVHanMoIs=.9cad7f15-24b8-4bf6-99db-cbf3ef3cccbf@github.com>
 <vhRP_xqkq0g3topmOPn1JdTpaXtEDTKuI72fdNd6hSw=.86d4f9c6-197a-48de-94a5-0cbe54ad41ff@github.com>
Message-ID: <9b331e83-3352-fbed-265b-38c6d2f22139@gmail.com>


On 12/07/21 2:08 am, Lance Andersen wrote:
> On Mon, 5 Jul 2021 07:42:26 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:
>
>>> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
>>>
>>> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
>>>
>>> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146
>> Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:
>>
>>    reorganize the tests now that the temp file creation threshold isn't exposed as a user configurable value
> I think the updates made to Zip FS look better.   Alan is on vacation so I would prefer to wait until he gets back and give him a chance to provide any last thoughts on the change to Zip FS.
>
> The manual test looks OK and is a good addition


Thank you for the reviews and running the tests, Lance. I'll wait for 
Alan to be back for his reviews.

-Jaikiran


From chegar at openjdk.java.net  Mon Jul 12 14:42:52 2021
From: chegar at openjdk.java.net (Chris Hegarty)
Date: Mon, 12 Jul 2021 14:42:52 GMT
Subject: [jdk17] RFR: 8266345: (fs) Custom DefaultFileSystemProvider
 security related loops
In-Reply-To: <V4RIZ2QAVoDuK-mc4hnsFn6j5HkfFj0lfxaFqAFf34s=.68b08383-7cd1-4af9-bd8e-657293351338@github.com>
References: <V4RIZ2QAVoDuK-mc4hnsFn6j5HkfFj0lfxaFqAFf34s=.68b08383-7cd1-4af9-bd8e-657293351338@github.com>
Message-ID: <XCWkV-I0UaU1pYXlPYRKsiOT6tqSibp6kEhbob4QfJ0=.1b967978-123c-44b2-8dfc-a35ea1323a9e@github.com>

On Thu, 8 Jul 2021 18:21:31 GMT, Sean Mullan <mullan at openjdk.org> wrote:

> Please review this fix to use the platform's default file system to avoid recursive policy initialization
> issues when a SecurityManager is enabled and the VM is configured to use a custom file system provider.

Marked as reviewed by chegar (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk17/pull/232

From mullan at openjdk.java.net  Mon Jul 12 14:57:58 2021
From: mullan at openjdk.java.net (Sean Mullan)
Date: Mon, 12 Jul 2021 14:57:58 GMT
Subject: [jdk17] Integrated: 8266345: (fs) Custom DefaultFileSystemProvider
 security related loops
In-Reply-To: <V4RIZ2QAVoDuK-mc4hnsFn6j5HkfFj0lfxaFqAFf34s=.68b08383-7cd1-4af9-bd8e-657293351338@github.com>
References: <V4RIZ2QAVoDuK-mc4hnsFn6j5HkfFj0lfxaFqAFf34s=.68b08383-7cd1-4af9-bd8e-657293351338@github.com>
Message-ID: <qWOpurec6FES8J2GmpWc8BdWSkF6KdFYNHV9MjKFpnE=.303c1e34-4c2b-43da-ac80-5f99bddfcc72@github.com>

On Thu, 8 Jul 2021 18:21:31 GMT, Sean Mullan <mullan at openjdk.org> wrote:

> Please review this fix to use the platform's default file system to avoid recursive policy initialization
> issues when a SecurityManager is enabled and the VM is configured to use a custom file system provider.

This pull request has now been integrated.

Changeset: 4fc3180f
Author:    Sean Mullan <mullan at openjdk.org>
URL:       https://git.openjdk.java.net/jdk17/commit/4fc3180f75e1cea4ebd613f8253be205d95f830c
Stats:     30 lines in 3 files changed: 28 ins; 0 del; 2 mod

8266345: (fs) Custom DefaultFileSystemProvider security related loops

Co-authored-by: Brian Burkhalter <bpb at openjdk.org>
Reviewed-by: bpb, chegar

-------------

PR: https://git.openjdk.java.net/jdk17/pull/232

From jwilhelm at openjdk.java.net  Mon Jul 12 23:21:35 2021
From: jwilhelm at openjdk.java.net (Jesper Wilhelmsson)
Date: Mon, 12 Jul 2021 23:21:35 GMT
Subject: RFR: Merge jdk17
Message-ID: <7zQieFkCIJDip3Y4qUzNkzmvs5CWEaF2L09fXDMbLkc=.2b0d08b5-5ed6-4d63-bc76-fccad123bdb6@github.com>

Forwardport JDK 17 -> JDK 18

-------------

Commit messages:
 - Merge
 - 8266345: (fs) Custom DefaultFileSystemProvider security related loops
 - 8269873: serviceability/sa/Clhsdb tests are using a C2 specific VMStruct field
 - 8268965: TCP Connection Reset when connecting simple socket to SSL server
 - 8269558: fix of JDK-8252657 missed to update history at the end of JVM TI spec
 - 8270216: [macOS] Update named used for Java run loop mode

The webrevs contain the adjustments done while merging with regards to each parent branch:
 - master: https://webrevs.openjdk.java.net/?repo=jdk&pr=4760&range=00.0
 - jdk17: https://webrevs.openjdk.java.net/?repo=jdk&pr=4760&range=00.1

Changes: https://git.openjdk.java.net/jdk/pull/4760/files
  Stats: 39 lines in 7 files changed: 33 ins; 2 del; 4 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4760.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4760/head:pull/4760

PR: https://git.openjdk.java.net/jdk/pull/4760

From jwilhelm at openjdk.java.net  Tue Jul 13 10:54:07 2021
From: jwilhelm at openjdk.java.net (Jesper Wilhelmsson)
Date: Tue, 13 Jul 2021 10:54:07 GMT
Subject: Integrated: Merge jdk17
In-Reply-To: <7zQieFkCIJDip3Y4qUzNkzmvs5CWEaF2L09fXDMbLkc=.2b0d08b5-5ed6-4d63-bc76-fccad123bdb6@github.com>
References: <7zQieFkCIJDip3Y4qUzNkzmvs5CWEaF2L09fXDMbLkc=.2b0d08b5-5ed6-4d63-bc76-fccad123bdb6@github.com>
Message-ID: <XC7_NxNXzknqIq8paB3h30KQkYqOppf0NgK02-xgEhU=.15d63b64-b013-400c-b722-27be4f11e5df@github.com>

On Mon, 12 Jul 2021 23:12:29 GMT, Jesper Wilhelmsson <jwilhelm at openjdk.org> wrote:

> Forwardport JDK 17 -> JDK 18

This pull request has now been integrated.

Changeset: 6b123b05
Author:    Jesper Wilhelmsson <jwilhelm at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/6b123b059136b0c1efa62a23824b9aa253e6a519
Stats:     39 lines in 7 files changed: 33 ins; 2 del; 4 mod

Merge

-------------

PR: https://git.openjdk.java.net/jdk/pull/4760

From github.com+1701815+mkarg at openjdk.java.net  Wed Jul 14 17:48:17 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Wed, 14 Jul 2021 17:48:17 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v3]
In-Reply-To: <aS5i72i7KLnTndIFIA_94OUjEgyuwXiHDHu1H898QtE=.0fcdd01c-2219-4318-86d7-8826941b6f41@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <aS5i72i7KLnTndIFIA_94OUjEgyuwXiHDHu1H898QtE=.0fcdd01c-2219-4318-86d7-8826941b6f41@github.com>
Message-ID: <8kSiVkKwbP0EbOfpVHO9HLMPHM58ySG-ZLwZjMdCpUk=.ff8a4d7b-085d-487d-ba47-f072a439ce6a@github.com>

On Fri, 2 Jul 2021 06:20:29 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
>> 
>> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
>> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
>> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
>> 
>> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
>> 
>> I encourage everybody to discuss this draft:
>> * Are there valid arguments for *not* doing this change?
>> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
>> * How to go on from here: What is missing to get this ready for an actual review?
>
> Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR.

I cloned the `InputStream/TransferTo` test in a way which uses providers, but when started to implement the providers I noticed that a huge amount of *empty* source code is needed just to make the compiler happy: `FileChannel` already has dozens of abstract methods I have to override but they never will be used by any of the tests... Having that said, it would be good if I would be allowed to spare us thousands of useless codelines:
* May I just implement the `content()` test or do you insist on me really implementing *all* the tests found in `InputStream/TransferTo` for *each* of the overloaded `transferTo` implementations?
* May I use a mocking framework to actually just provide *partial* implementations of the channels, or do you insist on me actually overloading *all* methods of *all* channels in actual Java source code just for the sake of making the compiler happy?

While I understand the necessity of tests, and while I am convinced that the performance benefit outweighs the weeks of work this would imply just for adding all those test code, I actually like to propose that I only cover the `content()` tests plus using Mockito, so we would have 80% of the benefit for 20% of the coding costs.

WDYT?

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From bpb at openjdk.java.net  Wed Jul 14 18:38:15 2021
From: bpb at openjdk.java.net (Brian Burkhalter)
Date: Wed, 14 Jul 2021 18:38:15 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v3]
In-Reply-To: <aS5i72i7KLnTndIFIA_94OUjEgyuwXiHDHu1H898QtE=.0fcdd01c-2219-4318-86d7-8826941b6f41@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <aS5i72i7KLnTndIFIA_94OUjEgyuwXiHDHu1H898QtE=.0fcdd01c-2219-4318-86d7-8826941b6f41@github.com>
Message-ID: <xBg2xYkt8KnJCohvSQW13aIyNAEhQIG9PVzLUAV1lQM=.771005fb-080f-4828-acc4-1ebf63a42a21@github.com>

On Fri, 2 Jul 2021 06:20:29 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
>> 
>> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
>> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
>> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
>> 
>> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
>> 
>> I encourage everybody to discuss this draft:
>> * Are there valid arguments for *not* doing this change?
>> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
>> * How to go on from here: What is missing to get this ready for an actual review?
>
> Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision:
> 
>   Draft: Renaming i and separating code into several methods
>   
>   Signed-off-by: Markus Karg <markus at headcrashing.eu>

Mockito is not approved for distribution and/or hosting by Java (Oracle JDK / OpenJDK).

I understand not wanting to have a lot of useless code. Have you looked into other tests which have implemented Providers?

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Sat Jul 17 08:37:52 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Sat, 17 Jul 2021 08:37:52 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v3]
In-Reply-To: <xBg2xYkt8KnJCohvSQW13aIyNAEhQIG9PVzLUAV1lQM=.771005fb-080f-4828-acc4-1ebf63a42a21@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <aS5i72i7KLnTndIFIA_94OUjEgyuwXiHDHu1H898QtE=.0fcdd01c-2219-4318-86d7-8826941b6f41@github.com>
 <xBg2xYkt8KnJCohvSQW13aIyNAEhQIG9PVzLUAV1lQM=.771005fb-080f-4828-acc4-1ebf63a42a21@github.com>
Message-ID: <p2EzAoH2D4K_NLO3V5PssMeZ1ZjPQ7LR7e20Iz6sdnQ=.24a97456-2286-4629-a90b-13ae4e956d71@github.com>

On Wed, 14 Jul 2021 18:35:15 GMT, Brian Burkhalter <bpb at openjdk.org> wrote:

> Have you looked into other tests which have implemented Providers?

I have not found any test code containing the word `new FileChannel() {...}` or `extends FileChannel`, so I apparently there exists no mock of `FileChannel`. Unfortunately, as long as it is mandatory to copy *all* the tests of `InputStream/TransferTo` this means that I *must* implement *all* methods of `FileChannel` (even if they are empty) just for the sake of throwing an exception after one, two, or three transferred bytes (which, BTW, is not part of the API but looks like just a personal decision of the original test author). It is not really helpful that OpenJDK *neither* uses mocking *nor* lets me reduce the number of tests to the *essential* minimum, as this means, I wrote approx. 100 LoC for the optimized implementations and now I have to write approx. 1000 LoC just to perform *exactly the same tests* (to be clear: I have no problem with testing `content()` which certainly *has* to be tested, but which does *not* imply writing a `FileChannel` mock).

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Thu Jul 22 05:54:51 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Thu, 22 Jul 2021 05:54:51 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v3]
In-Reply-To: <aS5i72i7KLnTndIFIA_94OUjEgyuwXiHDHu1H898QtE=.0fcdd01c-2219-4318-86d7-8826941b6f41@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <aS5i72i7KLnTndIFIA_94OUjEgyuwXiHDHu1H898QtE=.0fcdd01c-2219-4318-86d7-8826941b6f41@github.com>
Message-ID: <5JT_y4R4xZAA8CgrqeCsxrlbDkn2Q8817kyRymvn4F8=.e200a0b0-67e4-4742-97df-2c71000bc0c2@github.com>

On Fri, 2 Jul 2021 06:20:29 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
>> 
>> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
>> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
>> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
>> 
>> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
>> 
>> I encourage everybody to discuss this draft:
>> * Are there valid arguments for *not* doing this change?
>> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
>> * How to go on from here: What is missing to get this ready for an actual review?
>
> Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision:
> 
>   Draft: Renaming i and separating code into several methods
>   
>   Signed-off-by: Markus Karg <markus at headcrashing.eu>

Understood. Work is in progress. Stay tuned.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From jpai at openjdk.java.net  Thu Jul 22 12:55:46 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Thu, 22 Jul 2021 12:55:46 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v8]
In-Reply-To: <qvx7-anm29qNq_xfUfYTP0_8Me5pKoFnXSceloKS-9c=.d12bf1e3-5ead-45b9-bd6a-908a9b798e2f@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <ehuksZQILsWLXcuvxgtRGIq7Lqwl146QQpIVHanMoIs=.9cad7f15-24b8-4bf6-99db-cbf3ef3cccbf@github.com>
 <ConvQJ7XdR_kfz0T9BDkYlZ4l05w65o8i_bulHcdAxM=.ccd7e23a-dea4-44c0-a183-aa98ec31f955@github.com>
 <tYUfVXrSQR82OGRtYohP_1ieRkMgLrcniUksThnWFc0=.f709ca9d-9918-4de0-a5c4-1b011fd07f89@github.com>
 <qvx7-anm29qNq_xfUfYTP0_8Me5pKoFnXSceloKS-9c=.d12bf1e3-5ead-45b9-bd6a-908a9b798e2f@github.com>
Message-ID: <uG2wyFCNIlm-GSa3kAEEEj_MamETPOwZS9oSilafOw8=.09d1b5d4-17c2-468a-bec3-2d1853c717c1@github.com>

On Wed, 21 Jul 2021 04:09:23 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> > > For some context - the new `FileRolloverOutputStream` extends `ByteArrayOutputStream` and hence cannot have a `throws IOException` in its overridden `write` methods.
> > 
> > 
> > Have you tried wrapping a BAOS rather than extending, that might allow the exception wrapping/unwapping to go away.
> 
> Hello Alan,
> 
> I did experiment with it earlier, before going with the current approach in this PR. The disadvantage, as I see it, with wrapping a `ByteArrayOutputStream` instead of extending it is that when trying to rollover the contents to a file, you don't have access to the (inner protected) byte array of the ByteArrayOutputStream.
> 
> The rollover code would then look something like:
> 
> ```
> private void transferToFile() throws IOException {
>     // create a tempfile
>     entry.file = getTempPathForEntry(null);
>     // transfer the already written data from the byte array buffer into this tempfile
>     try (OutputStream os = new BufferedOutputStream(Files.newOutputStream(entry.file))) {
>         new ByteArrayInputStream(baos.toByteArray(), 0, baos.size()).transferTo(os);
>     }
>     // release the underlying byte array
>     baos = null;
>     // append any further data to the file with buffering enabled
>     tmpFileOS = new BufferedOutputStream(Files.newOutputStream(entry.file, APPEND));
> }
> ```
> 
> So although you can transfer the contents to the file without requiring the access to the byte array, you end up creating a new copy of that array (through the use of `baos.toByteArray()`), which can be at most 10MB in size. I thought avoiding a new copy of that (potentially 10MB) array during this transfer would be a good save and hence decided to settle on extending `ByteArrayOutputStream` instead of wrapping it.
> 
> The use of `extends` of course now means dealing with the `UncheckedIOException` as done in this PR. But if you think that the array copy isn't a concern and wrapping the `ByteArrayOutputStream` is a better way, then I'll go ahead and update this PR accordingly.

Now that the mailing lists integration seems to be back to normal, just adding this dummy comment to bring to attention the latest comments in this PR.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4607

From ecki at zusammenkunft.net  Thu Jul 22 15:24:39 2021
From: ecki at zusammenkunft.net (Bernd Eckenfels)
Date: Thu, 22 Jul 2021 15:24:39 +0000
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v8]
In-Reply-To: <uG2wyFCNIlm-GSa3kAEEEj_MamETPOwZS9oSilafOw8=.09d1b5d4-17c2-468a-bec3-2d1853c717c1@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <ehuksZQILsWLXcuvxgtRGIq7Lqwl146QQpIVHanMoIs=.9cad7f15-24b8-4bf6-99db-cbf3ef3cccbf@github.com>
 <ConvQJ7XdR_kfz0T9BDkYlZ4l05w65o8i_bulHcdAxM=.ccd7e23a-dea4-44c0-a183-aa98ec31f955@github.com>
 <tYUfVXrSQR82OGRtYohP_1ieRkMgLrcniUksThnWFc0=.f709ca9d-9918-4de0-a5c4-1b011fd07f89@github.com>
 <qvx7-anm29qNq_xfUfYTP0_8Me5pKoFnXSceloKS-9c=.d12bf1e3-5ead-45b9-bd6a-908a9b798e2f@github.com>,
 <uG2wyFCNIlm-GSa3kAEEEj_MamETPOwZS9oSilafOw8=.09d1b5d4-17c2-468a-bec3-2d1853c717c1@github.com>
Message-ID: <AM0PR03MB4145C179362F192CF419E8E8FFE49@AM0PR03MB4145.eurprd03.prod.outlook.com>

Hello,

> So although you can transfer the contents to the file without requiring the access
> to the byte array, you end up creating a new copy of that array (through the use
> of `baos.toByteArray()`)

You can avoid the copy and the additional buffer with baos.writeTo() I think.

try (OutputStream os = Files.newOutputStream(entry.file)) { // maybe append?
    baos.writeTo(os);
}
// release the underlying byte array
baos = null;
// append any further data to the file with buffering enabled
tmpFileOS = new BufferedOutputStream(Files.newOutputStream(entry.file, APPEND));


--
http://bernd.eckenfels.net
________________________________
Von: nio-dev <nio-dev-retn at openjdk.java.net> im Auftrag von Jaikiran Pai <jpai at openjdk.java.net>
Gesendet: Thursday, July 22, 2021 2:55:46 PM
An: core-libs-dev at openjdk.java.net <core-libs-dev at openjdk.java.net>; nio-dev at openjdk.java.net <nio-dev at openjdk.java.net>
Betreff: Re: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes) leads to a negative initial size for ByteArrayOutputStream [v8]

On Wed, 21 Jul 2021 04:09:23 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> > > For some context - the new `FileRolloverOutputStream` extends `ByteArrayOutputStream` and hence cannot have a `throws IOException` in its overridden `write` methods.
> >
> >
> > Have you tried wrapping a BAOS rather than extending, that might allow the exception wrapping/unwapping to go away.
>
> Hello Alan,
>
> I did experiment with it earlier, before going with the current approach in this PR. The disadvantage, as I see it, with wrapping a `ByteArrayOutputStream` instead of extending it is that when trying to rollover the contents to a file, you don't have access to the (inner protected) byte array of the ByteArrayOutputStream.
>
> The rollover code would then look something like:
>
> ```
> private void transferToFile() throws IOException {
>     // create a tempfile
>     entry.file = getTempPathForEntry(null);
>     // transfer the already written data from the byte array buffer into this tempfile
>     try (OutputStream os = new BufferedOutputStream(Files.newOutputStream(entry.file))) {
>         new ByteArrayInputStream(baos.toByteArray(), 0, baos.size()).transferTo(os);
>     }
>     // release the underlying byte array
>     baos = null;
>     // append any further data to the file with buffering enabled
>     tmpFileOS = new BufferedOutputStream(Files.newOutputStream(entry.file, APPEND));
> }
> ```
>
> So although you can transfer the contents to the file without requiring the access to the byte array, you end up creating a new copy of that array (through the use of `baos.toByteArray()`), which can be at most 10MB in size. I thought avoiding a new copy of that (potentially 10MB) array during this transfer would be a good save and hence decided to settle on extending `ByteArrayOutputStream` instead of wrapping it.
>
> The use of `extends` of course now means dealing with the `UncheckedIOException` as done in this PR. But if you think that the array copy isn't a concern and wrapping the `ByteArrayOutputStream` is a better way, then I'll go ahead and update this PR accordingly.

Now that the mailing lists integration seems to be back to normal, just adding this dummy comment to bring to attention the latest comments in this PR.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4607
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210722/ec04ac6b/attachment.htm>

From jai.forums2013 at gmail.com  Thu Jul 22 16:01:46 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Thu, 22 Jul 2021 21:31:46 +0530
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v8]
In-Reply-To: <AM0PR03MB4145C179362F192CF419E8E8FFE49@AM0PR03MB4145.eurprd03.prod.outlook.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <ehuksZQILsWLXcuvxgtRGIq7Lqwl146QQpIVHanMoIs=.9cad7f15-24b8-4bf6-99db-cbf3ef3cccbf@github.com>
 <ConvQJ7XdR_kfz0T9BDkYlZ4l05w65o8i_bulHcdAxM=.ccd7e23a-dea4-44c0-a183-aa98ec31f955@github.com>
 <tYUfVXrSQR82OGRtYohP_1ieRkMgLrcniUksThnWFc0=.f709ca9d-9918-4de0-a5c4-1b011fd07f89@github.com>
 <qvx7-anm29qNq_xfUfYTP0_8Me5pKoFnXSceloKS-9c=.d12bf1e3-5ead-45b9-bd6a-908a9b798e2f@github.com>
 <uG2wyFCNIlm-GSa3kAEEEj_MamETPOwZS9oSilafOw8=.09d1b5d4-17c2-468a-bec3-2d1853c717c1@github.com>
 <AM0PR03MB4145C179362F192CF419E8E8FFE49@AM0PR03MB4145.eurprd03.prod.outlook.com>
Message-ID: <a6d0412b-5e62-6f49-aefc-1d21f13c8155@gmail.com>

Hello Bernd,

On 22/07/21 8:54 pm, Bernd Eckenfels wrote:
> Hello,
>
>> So although you can transfer the contents to the file without requiring the access
>> to the byte array, you end up creating a new copy of that array (through the use
>> of `baos.toByteArray()`)
> You can avoid the copy and the additional buffer with baos.writeTo() I think.
>
> try (OutputStream os = Files.newOutputStream(entry.file)) { // maybe append?
>      baos.writeTo(os);
> }

You are absolutely right. I hadn't noticed ByteArrayOutputStream had 
this writeTo() method. Thank you for this input.

This was the only concern I had when it came to wrapping the 
ByteArrayOutputStream and now with your input it no longer is a concern. 
I have updated this PR to go ahead with the wrapping approach which also 
does away with the necessity of UncheckedIOException. Existing and the 
new tests continue to pass with this change.

-Jaikiran



From jpai at openjdk.java.net  Thu Jul 22 16:01:30 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Thu, 22 Jul 2021 16:01:30 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v9]
In-Reply-To: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
Message-ID: <xzX9pwzVnXSp-9Dp5BOlDRQNWZRL4mWFGeWSHTufRME=.a568319a-5c6d-451c-a023-8ea0519deae2@github.com>

> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
> 
> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
> 
> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146

Jaikiran Pai has updated the pull request incrementally with two additional commits since the last revision:

 - remove no longer necessary javadoc comment on test
 - review suggestion - wrap ByteArrayOutputStream instead of extending it

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4607/files
  - new: https://git.openjdk.java.net/jdk/pull/4607/files/c1ec9e12..90101d45

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=08
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=07-08

  Stats: 72 lines in 2 files changed: 19 ins; 35 del; 18 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4607.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4607/head:pull/4607

PR: https://git.openjdk.java.net/jdk/pull/4607

From jpai at openjdk.java.net  Fri Jul 23 03:54:13 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 23 Jul 2021 03:54:13 GMT
Subject: RFR: 8271147: java/nio/file/Path.java javadoc typo
Message-ID: <IzS3dg2GJ-tX_WHTAifilNFdJyjQsx7xN7FMUS8c6ok=.8db84393-43cd-4eda-b572-720fb0e34c55@github.com>

Can I please get a review for this change which fixes the typo noted in https://bugs.openjdk.java.net/browse/JDK-8271147? `make docs-image` worked fine and the generated javadoc looks fine.

-------------

Commit messages:
 - 8271147: java/nio/file/Path.java javadoc typo

Changes: https://git.openjdk.java.net/jdk/pull/4884/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4884&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8271147
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4884.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4884/head:pull/4884

PR: https://git.openjdk.java.net/jdk/pull/4884

From iris at openjdk.java.net  Fri Jul 23 04:01:06 2021
From: iris at openjdk.java.net (Iris Clark)
Date: Fri, 23 Jul 2021 04:01:06 GMT
Subject: RFR: 8271147: java/nio/file/Path.java javadoc typo
In-Reply-To: <IzS3dg2GJ-tX_WHTAifilNFdJyjQsx7xN7FMUS8c6ok=.8db84393-43cd-4eda-b572-720fb0e34c55@github.com>
References: <IzS3dg2GJ-tX_WHTAifilNFdJyjQsx7xN7FMUS8c6ok=.8db84393-43cd-4eda-b572-720fb0e34c55@github.com>
Message-ID: <qZXvMfFXBwYPYN2Vqie1g7Sd6pw0gsi6FUirmoesde0=.49aa6981-92c6-47be-a81c-30d3a54a87cd@github.com>

On Fri, 23 Jul 2021 03:37:44 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> Can I please get a review for this change which fixes the typo noted in https://bugs.openjdk.java.net/browse/JDK-8271147? `make docs-image` worked fine and the generated javadoc looks fine.

Marked as reviewed by iris (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk/pull/4884

From jpai at openjdk.java.net  Fri Jul 23 04:10:09 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 23 Jul 2021 04:10:09 GMT
Subject: Integrated: 8271147: java/nio/file/Path.java javadoc typo
In-Reply-To: <IzS3dg2GJ-tX_WHTAifilNFdJyjQsx7xN7FMUS8c6ok=.8db84393-43cd-4eda-b572-720fb0e34c55@github.com>
References: <IzS3dg2GJ-tX_WHTAifilNFdJyjQsx7xN7FMUS8c6ok=.8db84393-43cd-4eda-b572-720fb0e34c55@github.com>
Message-ID: <GaOK3kVWdRyJojbZWqEGmXVk6WtrnIJuqS3TCsomwuc=.b9706fdf-e8fc-4dff-b464-8f16ee287a8a@github.com>

On Fri, 23 Jul 2021 03:37:44 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> Can I please get a review for this change which fixes the typo noted in https://bugs.openjdk.java.net/browse/JDK-8271147? `make docs-image` worked fine and the generated javadoc looks fine.

This pull request has now been integrated.

Changeset: 8156ff60
Author:    Jaikiran Pai <jpai at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/8156ff609b27316f31ba89d9eb8ca752f4027c2b
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

8271147: java/nio/file/Path.java javadoc typo

Reviewed-by: iris

-------------

PR: https://git.openjdk.java.net/jdk/pull/4884

From jpai at openjdk.java.net  Fri Jul 23 04:10:08 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 23 Jul 2021 04:10:08 GMT
Subject: RFR: 8271147: java/nio/file/Path.java javadoc typo
In-Reply-To: <IzS3dg2GJ-tX_WHTAifilNFdJyjQsx7xN7FMUS8c6ok=.8db84393-43cd-4eda-b572-720fb0e34c55@github.com>
References: <IzS3dg2GJ-tX_WHTAifilNFdJyjQsx7xN7FMUS8c6ok=.8db84393-43cd-4eda-b572-720fb0e34c55@github.com>
Message-ID: <TSRVQCf-7JlAueOj6bucPtXXBueZDLcdr2PoL8jfj1k=.3f3d4622-baa7-4a36-b665-bc39f98a71a6@github.com>

On Fri, 23 Jul 2021 03:37:44 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> Can I please get a review for this change which fixes the typo noted in https://bugs.openjdk.java.net/browse/JDK-8271147? `make docs-image` worked fine and the generated javadoc looks fine.

Thank you for the review, Iris.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4884

From alanb at openjdk.java.net  Fri Jul 23 08:37:06 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Fri, 23 Jul 2021 08:37:06 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <FS4dG8muHl9vXW_eryRBZkDsRiquNnNOCLBlwsNVXeU=.1cecfec8-6a90-4cd4-a796-249c9b8c3110@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
 <FS4dG8muHl9vXW_eryRBZkDsRiquNnNOCLBlwsNVXeU=.1cecfec8-6a90-4cd4-a796-249c9b8c3110@github.com>
Message-ID: <RbqToCLJL51Pe1uYIwCEpfTby8QSOwhgqsF6knjTT84=.de3b6c66-3367-4b4f-a7b8-4e30bd7a0dca@github.com>

On Fri, 2 Jul 2021 11:08:43 GMT, Lance Andersen <lancea at openjdk.org> wrote:

>> Jaikiran Pai has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
>> 
>>  - Merge latest from upstream master branch to bring in fixes that might help fix the unrelated tier1 failures in Github Actions job runs
>>  - implement review suggestions:
>>     - reduce the toString() calls by creating a helper
>>     - fs.getPath("/") instead of fs.getRootDirectories().iterator().next()
>>  - implement review suggestion - move isSelfOrParent to ZipPath class
>>  - 8251329: (zipfs) Files.walkFileTree walks infinitely if zip has dir named "." inside
>
> Hi Jaikiran,
> 
> Consider:
> 
> 
>         try (var os = Files.newOutputStream(ZIPFILE);
>              ZipOutputStream zos = new ZipOutputStream(os)) {
>             zos.putNextEntry(new ZipEntry("../Hello.txt"));
>             zos.write("Hello World".getBytes(StandardCharsets.UTF_8));
>         }
> 
> 
> With your proposed fix, you will only return "/" when you walk the the Zip file and we should also return "/Hello.txt"
> 
> If. you resolve the path when the Inode entry is created:
> 
>            ` name(new ZipPath(null, normalize(name), true).getResolvedPath());`
> 
> That should address the issue and also allow you to access the entry.

I discussed this issue with @LanceAndersen and I think we are in agreement that a zip file system needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  The other options on the table (including Lance's origin prototype fix and the proposal in this PR) lead to anomalies. So I think we should close this PR and restart JDK-8251329.  @jaikiran Are you okay this this? JDK-8251329 is still assigned to JDK-8251329 and has a patch with the changes that we think are the right way for newFileSystem to reject ZIP files that have entries in the CEN that can't be used for files in a file system.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4604

From jpai at openjdk.java.net  Fri Jul 23 08:51:09 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 23 Jul 2021 08:51:09 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
Message-ID: <JpH4SJfNWNzVHuX8rsxpiZa6z996faubJkUexd7Mwwk=.fdef89e0-8e7e-491e-9957-53a0d4b03852@github.com>

On Fri, 2 Jul 2021 11:06:40 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review of this proposed fix for https://bugs.openjdk.java.net/browse/JDK-8251329?
>> 
>> As noted in that issue, if a zip filesystem created on top of a jar containing a "./" entry is "walked" using the `Files.walkFileTree`, it leads to a infinite never ending iteration (which ultimately fails with Java heap space OOM).
>> 
>> Alan notes in that issue that:
>> 
>>> This is more likely an issue with the zipfs DirectoryStream implementation. A DirectoryStream is specified to not include elements that for the special links to the current or parent directory. It should be rare. 
>> 
>> This indeed turned out to be an issue in the `jdk.nio.zipfs.ZipDirectoryStream#iterator()` implementation where it calls the `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` implementation.  The implementation, unlike what the javadoc of `java.nio.file.DirectoryStream` states, was including the "." and ".." paths in its returned iterator:
>> 
>>> The elements returned by the iterator are in no specific order. Some file
>>  systems maintain special links to the directory itself and the directory's
>>   parent directory. Entries representing these links are not returned by the
>>   iterator.
>> 
>> 
>> The proposed fix in this patch checks the paths for "." and "..", similar to what the `sun.nio.fs.UnixDirectoryStream` does in its `isSelfOrParent` and skips those paths from being added into the returned iterator. The `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` (where this change has been done) is currently only used by `jdk.nio.zipfs.ZipDirectoryStream#iterator()` and has package-private visibility, so this change shouldn't impact any other usage/expectations.
>> 
>> A new jtreg test has been added to reproduce this issue and verify the fix. Local testing of the `test/jdk/jdk/nio/` (including this new test) went fine without any issues after this change. I will be triggering a `tier1` test locally in a while.
>
> Jaikiran Pai has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
> 
>  - Merge latest from upstream master branch to bring in fixes that might help fix the unrelated tier1 failures in Github Actions job runs
>  - implement review suggestions:
>     - reduce the toString() calls by creating a helper
>     - fs.getPath("/") instead of fs.getRootDirectories().iterator().next()
>  - implement review suggestion - move isSelfOrParent to ZipPath class
>  - 8251329: (zipfs) Files.walkFileTree walks infinitely if zip has dir named "." inside

Hello Alan,

> So I think we should close this PR and restart JDK-8251329. @jaikiran Are you okay this this?

Yes, this sounds fine to me. I don't have any objection in closing this PR and I'll do it shortly.

> JDK-8251329 is still assigned to JDK-8251329 and has a patch with the changes that we think are the right way for newFileSystem to reject ZIP files 

This part I didn't understand. Did you mean to refer some other JBS issue? Because from what I see in https://bugs.openjdk.java.net/browse/JDK-8251329 there's no patch attached to it (unless of course it's restricted to specific JBS user roles?).

-------------

PR: https://git.openjdk.java.net/jdk/pull/4604

From jpai at openjdk.java.net  Fri Jul 23 09:03:13 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 23 Jul 2021 09:03:13 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
Message-ID: <7-Xlx5esvz244se9IG2KZDmLwscSh2SKk2dwSU1-l_0=.1a5b3fcb-a586-4c02-93fd-d45ef54fd78e@github.com>

On Fri, 2 Jul 2021 11:06:40 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review of this proposed fix for https://bugs.openjdk.java.net/browse/JDK-8251329?
>> 
>> As noted in that issue, if a zip filesystem created on top of a jar containing a "./" entry is "walked" using the `Files.walkFileTree`, it leads to a infinite never ending iteration (which ultimately fails with Java heap space OOM).
>> 
>> Alan notes in that issue that:
>> 
>>> This is more likely an issue with the zipfs DirectoryStream implementation. A DirectoryStream is specified to not include elements that for the special links to the current or parent directory. It should be rare. 
>> 
>> This indeed turned out to be an issue in the `jdk.nio.zipfs.ZipDirectoryStream#iterator()` implementation where it calls the `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` implementation.  The implementation, unlike what the javadoc of `java.nio.file.DirectoryStream` states, was including the "." and ".." paths in its returned iterator:
>> 
>>> The elements returned by the iterator are in no specific order. Some file
>>  systems maintain special links to the directory itself and the directory's
>>   parent directory. Entries representing these links are not returned by the
>>   iterator.
>> 
>> 
>> The proposed fix in this patch checks the paths for "." and "..", similar to what the `sun.nio.fs.UnixDirectoryStream` does in its `isSelfOrParent` and skips those paths from being added into the returned iterator. The `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` (where this change has been done) is currently only used by `jdk.nio.zipfs.ZipDirectoryStream#iterator()` and has package-private visibility, so this change shouldn't impact any other usage/expectations.
>> 
>> A new jtreg test has been added to reproduce this issue and verify the fix. Local testing of the `test/jdk/jdk/nio/` (including this new test) went fine without any issues after this change. I will be triggering a `tier1` test locally in a while.
>
> Jaikiran Pai has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
> 
>  - Merge latest from upstream master branch to bring in fixes that might help fix the unrelated tier1 failures in Github Actions job runs
>  - implement review suggestions:
>     - reduce the toString() calls by creating a helper
>     - fs.getPath("/") instead of fs.getRootDirectories().iterator().next()
>  - implement review suggestion - move isSelfOrParent to ZipPath class
>  - 8251329: (zipfs) Files.walkFileTree walks infinitely if zip has dir named "." inside

Closing this PR based on the above discussion.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4604

From jpai at openjdk.java.net  Fri Jul 23 09:03:13 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 23 Jul 2021 09:03:13 GMT
Subject: Withdrawn: 8251329: (zipfs) Files.walkFileTree walks infinitely if
 zip has dir named "." inside
In-Reply-To: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
Message-ID: <p2jXteophrfv2BG0aUQOEmyfaZ3YCfMpcDsk05Hez7Y=.1525f10b-dde2-419b-b407-970093e55122@github.com>

On Sun, 27 Jun 2021 13:13:42 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> Can I please get a review of this proposed fix for https://bugs.openjdk.java.net/browse/JDK-8251329?
> 
> As noted in that issue, if a zip filesystem created on top of a jar containing a "./" entry is "walked" using the `Files.walkFileTree`, it leads to a infinite never ending iteration (which ultimately fails with Java heap space OOM).
> 
> Alan notes in that issue that:
> 
>> This is more likely an issue with the zipfs DirectoryStream implementation. A DirectoryStream is specified to not include elements that for the special links to the current or parent directory. It should be rare. 
> 
> This indeed turned out to be an issue in the `jdk.nio.zipfs.ZipDirectoryStream#iterator()` implementation where it calls the `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` implementation.  The implementation, unlike what the javadoc of `java.nio.file.DirectoryStream` states, was including the "." and ".." paths in its returned iterator:
> 
>> The elements returned by the iterator are in no specific order. Some file
>  systems maintain special links to the directory itself and the directory's
>   parent directory. Entries representing these links are not returned by the
>   iterator.
> 
> 
> The proposed fix in this patch checks the paths for "." and "..", similar to what the `sun.nio.fs.UnixDirectoryStream` does in its `isSelfOrParent` and skips those paths from being added into the returned iterator. The `jdk.nio.zipfs.ZipFileSystem#iteratorOf(...)` (where this change has been done) is currently only used by `jdk.nio.zipfs.ZipDirectoryStream#iterator()` and has package-private visibility, so this change shouldn't impact any other usage/expectations.
> 
> A new jtreg test has been added to reproduce this issue and verify the fix. Local testing of the `test/jdk/jdk/nio/` (including this new test) went fine without any issues after this change. I will be triggering a `tier1` test locally in a while.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4604

From alanb at openjdk.java.net  Fri Jul 23 10:31:04 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Fri, 23 Jul 2021 10:31:04 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v9]
In-Reply-To: <xzX9pwzVnXSp-9Dp5BOlDRQNWZRL4mWFGeWSHTufRME=.a568319a-5c6d-451c-a023-8ea0519deae2@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <xzX9pwzVnXSp-9Dp5BOlDRQNWZRL4mWFGeWSHTufRME=.a568319a-5c6d-451c-a023-8ea0519deae2@github.com>
Message-ID: <woyZSONLQ0DEtRRSVD6kjlyXqlYYUROeN9k-DgWmeMg=.6205b3e1-c2e2-4861-a00d-4aec82e7d154@github.com>

On Thu, 22 Jul 2021 16:01:30 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
>> 
>> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
>> 
>> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146
>
> Jaikiran Pai has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - remove no longer necessary javadoc comment on test
>  - review suggestion - wrap ByteArrayOutputStream instead of extending it

Thanks for changing it to wrap the BOAS rather than existing it, that avoids the annoying wrapping/unwrapping of exceptions.

So I think the approach looks good but I think the synchronization needs to be re-checked it is not obvious that is correct or needed. Are there any cases where FileRolloverOutputStream is returned to user code? I don't think so, instead users of the zip file system will get an EntryOutputStream that wraps the FileRolloverOutputStream. The EntryOutputStream methods are synchronized so I assume that FileRolloverOutputStream does not need to it and that would avoid the inconsistency between the write methods and the flush/close methods.

One other thing to point out is that transferToFile shouldn't need to open the file twice, instead it should be able to open the tmp file for writing once.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4607

From alanb at openjdk.java.net  Fri Jul 23 10:40:09 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Fri, 23 Jul 2021 10:40:09 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <JpH4SJfNWNzVHuX8rsxpiZa6z996faubJkUexd7Mwwk=.fdef89e0-8e7e-491e-9957-53a0d4b03852@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
 <JpH4SJfNWNzVHuX8rsxpiZa6z996faubJkUexd7Mwwk=.fdef89e0-8e7e-491e-9957-53a0d4b03852@github.com>
Message-ID: <C-LUWPR74y0Tpi2CDCyyoWkt8m2uH_IhYomlyO0IoAc=.6ff84ced-ca68-42a3-b839-9fe4af2d492a@github.com>

On Fri, 23 Jul 2021 08:47:46 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> This part I didn't understand. Did you mean to refer some other JBS issue? Because from what I see in https://bugs.openjdk.java.net/browse/JDK-8251329 there's no patch attached to it (unless of course it's restricted to specific JBS user roles?).

I wasn't suggesting there is a patch attached to that issue. Rather I was just pointing out that JDK-8251329 was being worked on already before this PR was created. Lance's initial patch for JDK-8251329 changed ZipPath::getResolvedPath when creating the inode entry but that approach leads to anomalies. Overall I think the discussion has been useful but I don't think it's feasible to have a file system view over entries that look like links to the current or parent directory.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4604

From sebastian.stenzel at gmail.com  Fri Jul 23 10:59:57 2021
From: sebastian.stenzel at gmail.com (Sebastian Stenzel)
Date: Fri, 23 Jul 2021 12:59:57 +0200
Subject: Make java.nio.ByteBuffer a sealed class
In-Reply-To: <EFF59E15-951F-428D-8905-BF4743FD534B@gmail.com>
References: <EFF59E15-951F-428D-8905-BF4743FD534B@gmail.com>
Message-ID: <1CC33B30-7CB8-4499-90A6-6749C37B3C6E@gmail.com>

Any thoughts on this?

> On 8. Jul 2021, at 16:54, Sebastian Stenzel <sebastian.stenzel at gmail.com> wrote:
> 
> A while back, when working on JDK-8264110, we had a discussion about the nature of ByteBuffers, where I asked whether we could eventually make this a sealed class. [1]
> 
> The idea is, that basically there are only two kinds of ByteBuffers:
> - HeapByteBuffers
> - MappedByteBuffer ? DirectByteBuffer
> 
> While the former is backed by an array, the latter isn't, which leads to branching such as `if (buffer.hasArray())`, which could be simplified, if we knew for sure to deal with either of the two kinds.
> 
> Now, that sealed classes are stable with JDK 17, I'd like to ask for your approval to make ByteBuffer a sealed class, so we can rely on an exhaustive list of ByteBuffer "flavours" at compile time.
> 
> This is a requirement for follow-up tasks, I'd like to work on, such as JDK-8264341.
> 
> 
> [1] https://github.com/openjdk/jdk/pull/3217#discussion_r602528233


From jpai at openjdk.java.net  Fri Jul 23 11:52:47 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 23 Jul 2021 11:52:47 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v10]
In-Reply-To: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
Message-ID: <gK-f_jNlcnG0wS8IYmCs_Dz0fzesq0y_28ax8w79Ffg=.1da279b7-2a9d-4d3e-b347-81f3632da0f2@github.com>

> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
> 
> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
> 
> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146

Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision:

  Implement review suggestions:
   - remove unnecessary "synchronized"
   - no need to open the temp file twice

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4607/files
  - new: https://git.openjdk.java.net/jdk/pull/4607/files/90101d45..9e78ba06

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=09
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=08-09

  Stats: 12 lines in 1 file changed: 5 ins; 4 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4607.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4607/head:pull/4607

PR: https://git.openjdk.java.net/jdk/pull/4607

From jpai at openjdk.java.net  Fri Jul 23 12:00:08 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 23 Jul 2021 12:00:08 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v9]
In-Reply-To: <woyZSONLQ0DEtRRSVD6kjlyXqlYYUROeN9k-DgWmeMg=.6205b3e1-c2e2-4861-a00d-4aec82e7d154@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <xzX9pwzVnXSp-9Dp5BOlDRQNWZRL4mWFGeWSHTufRME=.a568319a-5c6d-451c-a023-8ea0519deae2@github.com>
 <woyZSONLQ0DEtRRSVD6kjlyXqlYYUROeN9k-DgWmeMg=.6205b3e1-c2e2-4861-a00d-4aec82e7d154@github.com>
Message-ID: <czBvY_lC-NcKS07rwYvYxv2SZAkznRoNSqf2uIzMm0M=.8a12ec0a-6e10-4ba6-a0b4-fc4a049149ac@github.com>

On Fri, 23 Jul 2021 10:28:02 GMT, Alan Bateman <alanb at openjdk.org> wrote:

> So I think the approach looks good but I think the synchronization needs to be re-checked it is not obvious that is correct or needed. Are there any cases where FileRolloverOutputStream is returned to user code? I don't think so, instead users of the zip file system will get an EntryOutputStream that wraps the FileRolloverOutputStream. The EntryOutputStream methods are synchronized so I assume that FileRolloverOutputStream does not need to it and that would avoid the inconsistency between the write methods and the flush/close methods.

I hadn't paid any thoughts on the "synchronized" part. You are right - the new `FileRolloverOutputStream` doesn't get sent back to the callers directly and instead either the `EntryOutputStream` or the `DeflatingEntryOutputStream` get returned. Both of them have the necessary syncrhonizations in place for `write` and `close` operations. The `flush` of the `FileRolloverOutputStream` calls the `flush` on the `BufferedOutputStream` which already has the necessary synchronization. I've updated the PR to remove the use of `synchronized` from this new class and added a brief note about this for future maintainers, just like the existing `EntryOutputStreamDef` has.

> One other thing to point out is that transferToFile shouldn't need to open the file twice, instead it should be able to open the tmp file for writing once.

The updated version of this PR now fixes this part to open it just once. I had reviewed this `transferTo` multiple times before, but clearly I overlooked this part of the implementation. 

Thank you for these inputs. The updated PR continues to pass the new tests and the existing ones in `test/jdk/jdk/nio/zipfs/`.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4607

From jpai at openjdk.java.net  Fri Jul 23 12:09:10 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 23 Jul 2021 12:09:10 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <C-LUWPR74y0Tpi2CDCyyoWkt8m2uH_IhYomlyO0IoAc=.6ff84ced-ca68-42a3-b839-9fe4af2d492a@github.com>
References: <H5Dy39I78mG6FE2N9U5vZnEhGI2zcIaxGvJ8uZBCGM0=.f3ba4976-f985-4002-87f3-da234b3c402d@github.com>
 <FWKKv49jcPmyN0fr_GVK0Sz3DceDPXRF724C45esjF8=.c86aa8c4-8fbd-4b76-90fa-a906323aae8a@github.com>
 <JpH4SJfNWNzVHuX8rsxpiZa6z996faubJkUexd7Mwwk=.fdef89e0-8e7e-491e-9957-53a0d4b03852@github.com>
 <C-LUWPR74y0Tpi2CDCyyoWkt8m2uH_IhYomlyO0IoAc=.6ff84ced-ca68-42a3-b839-9fe4af2d492a@github.com>
Message-ID: <qtX6PYmeMyrE1wEEfQPql8noL121rWThxuj6Wu7FWvs=.4bacb27a-4072-44c8-ae74-dae1af30b210@github.com>

On Fri, 23 Jul 2021 10:36:51 GMT, Alan Bateman <alanb at openjdk.org> wrote:

> I wasn't suggesting there is a patch attached to that issue. Rather I was just pointing out that JDK-8251329 was being worked on already before this PR was created.

Ok, that makes sense. Thank you for the details.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4604

From jpai at openjdk.java.net  Fri Jul 23 14:58:01 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 23 Jul 2021 14:58:01 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v11]
In-Reply-To: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
Message-ID: <eglX2CJeKvHWYuASSDER6ClGtBUbYk3YVD9cpensiDY=.2450e7d6-9c0e-4de6-a907-d878f21a0e6e@github.com>

> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
> 
> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
> 
> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146

Jaikiran Pai has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 15 additional commits since the last revision:

 - Merge latest from master branch
 - Implement review suggestions:
    - remove unnecessary "synchronized"
    - no need to open the temp file twice
 - remove no longer necessary javadoc comment on test
 - review suggestion - wrap ByteArrayOutputStream instead of extending it
 - reorganize the tests now that the temp file creation threshold isn't exposed as a user configurable value
 - minor update to comment on FileRolloverOutputStream class
 - remove no longer used constant
 - use jtreg's construct of manual test
 - Implement Alan's review suggestion - don't expose the threshold as a configuration property, instead set it internally to a specific value
 - propagate back the original checked IOException to the callers
 - ... and 5 more: https://git.openjdk.java.net/jdk/compare/d783ad9d...991de6b9

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4607/files
  - new: https://git.openjdk.java.net/jdk/pull/4607/files/9e78ba06..991de6b9

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=10
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4607&range=09-10

  Stats: 43946 lines in 1112 files changed: 20903 ins; 17871 del; 5172 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4607.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4607/head:pull/4607

PR: https://git.openjdk.java.net/jdk/pull/4607

From alanb at openjdk.java.net  Fri Jul 23 15:18:06 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Fri, 23 Jul 2021 15:18:06 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v9]
In-Reply-To: <czBvY_lC-NcKS07rwYvYxv2SZAkznRoNSqf2uIzMm0M=.8a12ec0a-6e10-4ba6-a0b4-fc4a049149ac@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <xzX9pwzVnXSp-9Dp5BOlDRQNWZRL4mWFGeWSHTufRME=.a568319a-5c6d-451c-a023-8ea0519deae2@github.com>
 <woyZSONLQ0DEtRRSVD6kjlyXqlYYUROeN9k-DgWmeMg=.6205b3e1-c2e2-4861-a00d-4aec82e7d154@github.com>
 <czBvY_lC-NcKS07rwYvYxv2SZAkznRoNSqf2uIzMm0M=.8a12ec0a-6e10-4ba6-a0b4-fc4a049149ac@github.com>
Message-ID: <NE22Zjblp8nkZWJ3bQ-jfBpXhl-UGlbKNbgLkR49Kko=.81a6e830-d0ff-42c1-9b7c-9b91926d044b@github.com>

On Fri, 23 Jul 2021 11:57:34 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> The updated version of this PR now fixes this part to open it just once. I had reviewed this `transferTo` multiple times before, but clearly I overlooked this part of the implementation.
> 
> Thank you for these inputs. The updated PR continues to pass the new tests and the existing ones in `test/jdk/jdk/nio/zipfs/`.

The updated implementation looks okay, I don't think I have any more questions.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4607

From jpai at openjdk.java.net  Fri Jul 23 15:26:19 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 23 Jul 2021 15:26:19 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v11]
In-Reply-To: <eglX2CJeKvHWYuASSDER6ClGtBUbYk3YVD9cpensiDY=.2450e7d6-9c0e-4de6-a907-d878f21a0e6e@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <eglX2CJeKvHWYuASSDER6ClGtBUbYk3YVD9cpensiDY=.2450e7d6-9c0e-4de6-a907-d878f21a0e6e@github.com>
Message-ID: <2ttMZAIqSLz7XfsDi0CQjsaApF19Vk68hNNgh_WDIw0=.a4b275b9-3ce9-4101-9dfd-0db920386eb1@github.com>

On Fri, 23 Jul 2021 14:58:01 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
>> 
>> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
>> 
>> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146
>
> Jaikiran Pai has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 15 additional commits since the last revision:
> 
>  - Merge latest from master branch
>  - Implement review suggestions:
>     - remove unnecessary "synchronized"
>     - no need to open the temp file twice
>  - remove no longer necessary javadoc comment on test
>  - review suggestion - wrap ByteArrayOutputStream instead of extending it
>  - reorganize the tests now that the temp file creation threshold isn't exposed as a user configurable value
>  - minor update to comment on FileRolloverOutputStream class
>  - remove no longer used constant
>  - use jtreg's construct of manual test
>  - Implement Alan's review suggestion - don't expose the threshold as a configuration property, instead set it internally to a specific value
>  - propagate back the original checked IOException to the callers
>  - ... and 5 more: https://git.openjdk.java.net/jdk/compare/e1196067...991de6b9

Thank you for the review Alan.

@LanceAndersen, I've run the tier1 tests locally with the latest PR and they have passed without any regressions. Given that we changed the implementation to wrap ByteArrayOutputStream instead of extending it, would you want to rerun some of your other tests that you had previously run, before I integrate this?

-------------

PR: https://git.openjdk.java.net/jdk/pull/4607

From paul.sandoz at oracle.com  Fri Jul 23 15:46:25 2021
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Fri, 23 Jul 2021 15:46:25 +0000
Subject: Make java.nio.ByteBuffer a sealed class
In-Reply-To: <1CC33B30-7CB8-4499-90A6-6749C37B3C6E@gmail.com>
References: <EFF59E15-951F-428D-8905-BF4743FD534B@gmail.com>
 <1CC33B30-7CB8-4499-90A6-6749C37B3C6E@gmail.com>
Message-ID: <11344009-DD3A-411F-93E5-0121DC95180D@oracle.com>

I don?t see much advantage to this. Most of the (byte) buffer hierarchy is package private. Further, the hierarchy is set up for convenience: DirectByteBuffer extends MappedByteBuffer [*], even though not all DirectByteBuffer instances are mapped.

Internally we can assume if a ByteBuffer instance is also an instance sun.nio.ch.DirectBuffer (which in turn implies that ByteBuffer.isDirect() == true) then it?s a direct buffer. Otherwise, its a heap buffer and if hasArray() == true then the underlying array is accessible.

In the referenced code there is a precondition that the byte buffer is not read-only. Therefore, we can assume for a heap buffer the src array is accessible (use an assert if you wish).

My hope is this kind of code can improve when we can use MemorySegment.

Paul.

[*]
public abstract class MappedByteBuffer
    extends ByteBuffer
{

    // This is a little bit backwards: By rights MappedByteBuffer should be a
    // subclass of DirectByteBuffer, but to keep the spec clear and simple, and
    // for optimization purposes, it's easier to do it the other way around.
    // This works because DirectByteBuffer is a package-private class.

> On Jul 23, 2021, at 3:59 AM, Sebastian Stenzel <sebastian.stenzel at gmail.com> wrote:
> 
> Any thoughts on this?
> 
>> On 8. Jul 2021, at 16:54, Sebastian Stenzel <sebastian.stenzel at gmail.com> wrote:
>> 
>> A while back, when working on JDK-8264110, we had a discussion about the nature of ByteBuffers, where I asked whether we could eventually make this a sealed class. [1]
>> 
>> The idea is, that basically there are only two kinds of ByteBuffers:
>> - HeapByteBuffers
>> - MappedByteBuffer ? DirectByteBuffer
>> 
>> While the former is backed by an array, the latter isn't, which leads to branching such as `if (buffer.hasArray())`, which could be simplified, if we knew for sure to deal with either of the two kinds.
>> 
>> Now, that sealed classes are stable with JDK 17, I'd like to ask for your approval to make ByteBuffer a sealed class, so we can rely on an exhaustive list of ByteBuffer "flavours" at compile time.
>> 
>> This is a requirement for follow-up tasks, I'd like to work on, such as JDK-8264341.
>> 
>> 
>> [1] https://github.com/openjdk/jdk/pull/3217#discussion_r602528233
> 


From lancea at openjdk.java.net  Fri Jul 23 16:25:03 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Fri, 23 Jul 2021 16:25:03 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v11]
In-Reply-To: <2ttMZAIqSLz7XfsDi0CQjsaApF19Vk68hNNgh_WDIw0=.a4b275b9-3ce9-4101-9dfd-0db920386eb1@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <eglX2CJeKvHWYuASSDER6ClGtBUbYk3YVD9cpensiDY=.2450e7d6-9c0e-4de6-a907-d878f21a0e6e@github.com>
 <2ttMZAIqSLz7XfsDi0CQjsaApF19Vk68hNNgh_WDIw0=.a4b275b9-3ce9-4101-9dfd-0db920386eb1@github.com>
Message-ID: <27fDIfBum2vSv7DmFs_PH8Z27NAYMVeMUmAa6OOJgfI=.b125d8c7-83a0-48d5-8342-d0da696fdbff@github.com>

On Fri, 23 Jul 2021 15:23:07 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> Thank you for the review Alan.
> 
> @LanceAndersen, I've run the tier1 tests locally with the latest PR and they have passed without any regressions. Given that we changed the implementation to wrap ByteArrayOutputStream instead of extending it, would you want to rerun some of your other tests that you had previously run, before I integrate this?

Yes, I will run additional tests and report back after they complete

-------------

PR: https://git.openjdk.java.net/jdk/pull/4607

From sebastian.stenzel at gmail.com  Fri Jul 23 21:48:31 2021
From: sebastian.stenzel at gmail.com (Sebastian Stenzel)
Date: Fri, 23 Jul 2021 23:48:31 +0200
Subject: Make java.nio.ByteBuffer a sealed class
In-Reply-To: <11344009-DD3A-411F-93E5-0121DC95180D@oracle.com>
References: <11344009-DD3A-411F-93E5-0121DC95180D@oracle.com>
Message-ID: <F15BD1D3-DB39-4DEA-AE1A-093AB71BD72E@gmail.com>

Sure, we can just assert that any ByteBuffer that isn't direct must be a HeapByteBuffer. But this assertion is based on our knowlege. The point is, that we could use a mechanism which would instead cause a compile time error if this ever changes. In my opinion this is the stronger tool to ensure correctness than relying on every contributor to know that only these existing implementations of ByteBuffers are allowed.

This change has no direct functional effect, it merely helps to reduce branches that are technically dead code and just exist to satisfy ByteBuffer's contract. In other words: There is no functional advantage. This is about reducing technical debt.

> Am 23.07.2021 um 17:46 schrieb Paul Sandoz <paul.sandoz at oracle.com>:
> 
> ?I don?t see much advantage to this. Most of the (byte) buffer hierarchy is package private. Further, the hierarchy is set up for convenience: DirectByteBuffer extends MappedByteBuffer [*], even though not all DirectByteBuffer instances are mapped.
> 
> Internally we can assume if a ByteBuffer instance is also an instance sun.nio.ch.DirectBuffer (which in turn implies that ByteBuffer.isDirect() == true) then it?s a direct buffer. Otherwise, its a heap buffer and if hasArray() == true then the underlying array is accessible.
> 
> In the referenced code there is a precondition that the byte buffer is not read-only. Therefore, we can assume for a heap buffer the src array is accessible (use an assert if you wish).
> 
> My hope is this kind of code can improve when we can use MemorySegment.
> 
> Paul.
> 
> [*]
> public abstract class MappedByteBuffer
>    extends ByteBuffer
> {
> 
>    // This is a little bit backwards: By rights MappedByteBuffer should be a
>    // subclass of DirectByteBuffer, but to keep the spec clear and simple, and
>    // for optimization purposes, it's easier to do it the other way around.
>    // This works because DirectByteBuffer is a package-private class.
> 
>> On Jul 23, 2021, at 3:59 AM, Sebastian Stenzel <sebastian.stenzel at gmail.com> wrote:
>> 
>> Any thoughts on this?
>> 
>>>> On 8. Jul 2021, at 16:54, Sebastian Stenzel <sebastian.stenzel at gmail.com> wrote:
>>> 
>>> A while back, when working on JDK-8264110, we had a discussion about the nature of ByteBuffers, where I asked whether we could eventually make this a sealed class. [1]
>>> 
>>> The idea is, that basically there are only two kinds of ByteBuffers:
>>> - HeapByteBuffers
>>> - MappedByteBuffer ? DirectByteBuffer
>>> 
>>> While the former is backed by an array, the latter isn't, which leads to branching such as `if (buffer.hasArray())`, which could be simplified, if we knew for sure to deal with either of the two kinds.
>>> 
>>> Now, that sealed classes are stable with JDK 17, I'd like to ask for your approval to make ByteBuffer a sealed class, so we can rely on an exhaustive list of ByteBuffer "flavours" at compile time.
>>> 
>>> This is a requirement for follow-up tasks, I'd like to work on, such as JDK-8264341.
>>> 
>>> 
>>> [1] https://github.com/openjdk/jdk/pull/3217#discussion_r602528233
>> 
> 

From lancea at openjdk.java.net  Sun Jul 25 22:04:40 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Sun, 25 Jul 2021 22:04:40 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip has
 dir named "." inside
Message-ID: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>

Hi,

As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS

Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.


Mach5 tiers 1 through 3 have been run without any errors encountered .

Best,
Lance

-------------

Commit messages:
 - Address missing linefeed  after package name
 - Address overzelous intellij import update
 - Patch to address JDK-8251329

Changes: https://git.openjdk.java.net/jdk/pull/4900/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4900&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8251329
  Stats: 174 lines in 2 files changed: 174 ins; 0 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4900.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4900/head:pull/4900

PR: https://git.openjdk.java.net/jdk/pull/4900

From alanb at openjdk.java.net  Mon Jul 26 07:54:06 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Mon, 26 Jul 2021 07:54:06 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside
In-Reply-To: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
Message-ID: <EenpuGghvFrTL7WCedsVxo-6BCzSxMQJOr9fKotMELw=.54217ed0-e351-4797-9df7-567296768585@github.com>

On Sun, 25 Jul 2021 21:56:10 GMT, Lance Andersen <lancea at openjdk.org> wrote:

> Hi,
> 
> As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS
> 
> Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.
> 
> 
> Mach5 tiers 1 through 3 have been run without any errors encountered .
> 
> Best,
> Lance

This is behavior change (to reject zip/JAR files) so a CSR will be required. I've also added a label to the JBS issue to remind us to add a release note.

I think it would be good for @jaikiran to review too.

src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java line 1573:

> 1571:             }
> 1572:             IndexNode inode = new IndexNode(cen, pos, nlen);
> 1573:             if(hasDotOrDotDot(inode.name)) {

Minor nit, missing space in "if(".

src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java line 1602:

> 1600:         // Inode.name always includes "/" in path[0]
> 1601:         if (path.length == 1) {
> 1602:             return false;

It may be useful to add "assert path[0] == '/';" at the start of this method.

test/jdk/jdk/nio/zipfs/HasDotDotTest.java line 1:

> 1: import org.testng.annotations.DataProvider;

Missing copyright header.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4900

From github.com+10835776+stsypanov at openjdk.java.net  Mon Jul 26 08:27:20 2021
From: github.com+10835776+stsypanov at openjdk.java.net (=?UTF-8?B?0KHQtdGA0LPQtdC5?= =?UTF-8?B?IA==?= =?UTF-8?B?0KbRi9C/0LDQvdC+0LI=?=)
Date: Mon, 26 Jul 2021 08:27:20 GMT
Subject: RFR: 8263561: Re-examine uses of LinkedList [v5]
In-Reply-To: <eCQ8oFDjAJgFR3SG0K7GlaM1lxivFOkQGkIi-i7I7Xw=.bd379708-2bb4-4d52-81e3-175fb7a9c46b@github.com>
References: <eCQ8oFDjAJgFR3SG0K7GlaM1lxivFOkQGkIi-i7I7Xw=.bd379708-2bb4-4d52-81e3-175fb7a9c46b@github.com>
Message-ID: <oj-2SM4qef-9bKhmErlifgLmp3bNeyNi7P-cqkQutxA=.e8891bb3-d00c-4a27-bbe6-a179a475343d@github.com>

> After I've renamed remove branch GitHub for some reason has closed original https://github.com/openjdk/jdk/pull/2744, so I've decided to recreate it.

?????? ??????? has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 10 commits:

 - Merge branch 'master' into 8263561
 - Merge branch 'master' into 8263561
 - Merge branch 'master' into 8263561
 - Merge branch 'master' into 8263561
 - Merge branch 'master' into 8263561
   
   # Conflicts:
   #	src/java.base/unix/classes/sun/net/dns/ResolverConfigurationImpl.java
 - Merge branch 'master' into purge-linked-list
 - 8263561: Use sized constructor where reasonable
 - 8263561: Use interface List instead of particular type where possible
 - 8263561: Rename requestList -> requests
 - 8263561: Re-examine uses of LinkedList

-------------

Changes: https://git.openjdk.java.net/jdk/pull/4304/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4304&range=04
  Stats: 48 lines in 9 files changed: 0 ins; 2 del; 46 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4304.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4304/head:pull/4304

PR: https://git.openjdk.java.net/jdk/pull/4304

From jpai at openjdk.java.net  Mon Jul 26 09:55:07 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Mon, 26 Jul 2021 09:55:07 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside
In-Reply-To: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
Message-ID: <KsKUat1TucLU0aKYTiQ3LnH0B40G70ZWEs4ycbeDleQ=.8f12288c-1089-4226-bbda-af212119644e@github.com>

On Sun, 25 Jul 2021 21:56:10 GMT, Lance Andersen <lancea at openjdk.org> wrote:

> Hi,
> 
> As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS
> 
> Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.
> 
> 
> Mach5 tiers 1 through 3 have been run without any errors encountered .
> 
> Best,
> Lance

This change looks fine to me. I was unsure how the writing/creating entries with `.` or `..` with `ZipFileSystem` would behave in context of this change, so I gave this a try locally with the changes from this PR:


try (FileSystem zipfs = FileSystems.newFileSystem(ZIPFILE)) {
            final Path[] paths = new Path[]{
                    zipfs.getPath("./Hello.txt"),
                    zipfs.getPath("../../../Hello.txt"),
                    zipfs.getPath("../Hello.txt")};
            for (int i = 0; i < paths.length; i++) {
                try (OutputStream os = Files.newOutputStream(paths[i])) {
                    os.write(("Hello " + i).getBytes(StandardCharsets.UTF_8));
                }
            }
        }

This code runs fine and it ends up creating (just one) CEN entry for `Hello.txt`:

JTwork/scratch/zipfsDotDotTest.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        7  07-26-2021 15:07   Hello.txt

In other words, the `ZipFileSystem` doesn't end up creating a zip file which is then rejected by `ZipFileSystem` when creating a new filesystem using `Files.newFileSystem`. That's a good thing.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4900

From lancea at openjdk.java.net  Mon Jul 26 10:16:47 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Mon, 26 Jul 2021 10:16:47 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v2]
In-Reply-To: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
Message-ID: <MEu03waUkJEgptOFz8noYr3hKH7eaTSRSpXWUhOB-2I=.97f556fb-5543-45da-beb8-b671ae209f45@github.com>

> Hi,
> 
> As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS
> 
> Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.
> 
> 
> Mach5 tiers 1 through 3 have been run without any errors encountered .
> 
> Best,
> Lance

Lance Andersen has updated the pull request incrementally with one additional commit since the last revision:

  Add missing Copyright header and address minor comments

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4900/files
  - new: https://git.openjdk.java.net/jdk/pull/4900/files/68af64c4..2265ffe1

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4900&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4900&range=00-01

  Stats: 25 lines in 2 files changed: 24 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4900.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4900/head:pull/4900

PR: https://git.openjdk.java.net/jdk/pull/4900

From lancea at openjdk.java.net  Mon Jul 26 10:16:53 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Mon, 26 Jul 2021 10:16:53 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v2]
In-Reply-To: <EenpuGghvFrTL7WCedsVxo-6BCzSxMQJOr9fKotMELw=.54217ed0-e351-4797-9df7-567296768585@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
 <EenpuGghvFrTL7WCedsVxo-6BCzSxMQJOr9fKotMELw=.54217ed0-e351-4797-9df7-567296768585@github.com>
Message-ID: <yOfKwQZp7mqnKcPKBJEnrWZmx-tAB1BqzdlKLZ8q034=.d50e6fff-59fb-47c6-be10-7d0cce9f6aef@github.com>

On Mon, 26 Jul 2021 07:30:12 GMT, Alan Bateman <alanb at openjdk.org> wrote:

>> Lance Andersen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Add missing Copyright header and address minor comments
>
> src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java line 1573:
> 
>> 1571:             }
>> 1572:             IndexNode inode = new IndexNode(cen, pos, nlen);
>> 1573:             if(hasDotOrDotDot(inode.name)) {
> 
> Minor nit, missing space in "if(".

fixed

> src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java line 1602:
> 
>> 1600:         // Inode.name always includes "/" in path[0]
>> 1601:         if (path.length == 1) {
>> 1602:             return false;
> 
> It may be useful to add "assert path[0] == '/';" at the start of this method.

I added it per your suggestion, though IndexNode(byte[] cen, int pos, int nlen) which is only used by initCen() will already guarantee the leading "/" is there

> test/jdk/jdk/nio/zipfs/HasDotDotTest.java line 1:
> 
>> 1: import org.testng.annotations.DataProvider;
> 
> Missing copyright header.

Geez, how did I miss that.  Added

-------------

PR: https://git.openjdk.java.net/jdk/pull/4900

From lancea at openjdk.java.net  Mon Jul 26 10:19:06 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Mon, 26 Jul 2021 10:19:06 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside
In-Reply-To: <KsKUat1TucLU0aKYTiQ3LnH0B40G70ZWEs4ycbeDleQ=.8f12288c-1089-4226-bbda-af212119644e@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
 <KsKUat1TucLU0aKYTiQ3LnH0B40G70ZWEs4ycbeDleQ=.8f12288c-1089-4226-bbda-af212119644e@github.com>
Message-ID: <hBMOJU-d8rc1scvBZOBLSe5oQJ56iBRcM8NMBLE01KM=.6f3bb1b8-74de-437e-8f82-5d9e622a22c9@github.com>

On Mon, 26 Jul 2021 09:52:09 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> This change looks fine to me. I was unsure how the writing/creating entries with `.` or `..` with `ZipFileSystem` would behave in context of this change, so I gave this a try locally with the changes from this PR:
> 
> ```
> try (FileSystem zipfs = FileSystems.newFileSystem(ZIPFILE)) {
>             final Path[] paths = new Path[]{
>                     zipfs.getPath("./Hello.txt"),
>                     zipfs.getPath("../../../Hello.txt"),
>                     zipfs.getPath("../Hello.txt")};
>             for (int i = 0; i < paths.length; i++) {
>                 try (OutputStream os = Files.newOutputStream(paths[i])) {
>                     os.write(("Hello " + i).getBytes(StandardCharsets.UTF_8));
>                 }
>             }
>         }
> ```
> 
> This code runs fine and it ends up creating (just one) CEN entry for `Hello.txt`:
> 
> ```
> JTwork/scratch/zipfsDotDotTest.zip
>   Length      Date    Time    Name
> ---------  ---------- -----   ----
>         7  07-26-2021 15:07   Hello.txt
> ```
> 
> In other words, the `ZipFileSystem` doesn't end up creating a zip file which is then rejected by `ZipFileSystem` when creating a new filesystem using `Files.newFileSystem`. That's a good thing.


With the exception of creating the Inodes table,  Zip FS always calls ZipPath::getResolvedPath for access to Zip entries.  So there is no issues with the creation and access of entries in the case above

-------------

PR: https://git.openjdk.java.net/jdk/pull/4900

From alanb at openjdk.java.net  Mon Jul 26 10:26:07 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Mon, 26 Jul 2021 10:26:07 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v2]
In-Reply-To: <MEu03waUkJEgptOFz8noYr3hKH7eaTSRSpXWUhOB-2I=.97f556fb-5543-45da-beb8-b671ae209f45@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
 <MEu03waUkJEgptOFz8noYr3hKH7eaTSRSpXWUhOB-2I=.97f556fb-5543-45da-beb8-b671ae209f45@github.com>
Message-ID: <ZQw1s2E8HzzpD-QuIWyEF_SRKkzFqJOVWabN9ofyN4Q=.3cf1d3e7-915b-4c60-a128-6241dba3a95c@github.com>

On Mon, 26 Jul 2021 10:16:47 GMT, Lance Andersen <lancea at openjdk.org> wrote:

>> Hi,
>> 
>> As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS
>> 
>> Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.
>> 
>> 
>> Mach5 tiers 1 through 3 have been run without any errors encountered .
>> 
>> Best,
>> Lance
>
> Lance Andersen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add missing Copyright header and address minor comments

>                     zipfs.getPath("./Hello.txt"),
>                     zipfs.getPath("../../../Hello.txt"),
>                     zipfs.getPath("../Hello.txt")};
>
> 
> In other words, the `ZipFileSystem` doesn't end up creating a zip file which is then rejected by `ZipFileSystem` when creating a new filesystem using `Files.newFileSystem`. That's a good thing.

Right, it's always existing behavior and matches the behavior of the platform file system.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4900

From paul.sandoz at oracle.com  Mon Jul 26 15:54:13 2021
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 26 Jul 2021 15:54:13 +0000
Subject: Make java.nio.ByteBuffer a sealed class
In-Reply-To: <F15BD1D3-DB39-4DEA-AE1A-093AB71BD72E@gmail.com>
References: <11344009-DD3A-411F-93E5-0121DC95180D@oracle.com>
 <F15BD1D3-DB39-4DEA-AE1A-093AB71BD72E@gmail.com>
Message-ID: <81996939-A032-4C61-8A35-AD007F8DCB4D@oracle.com>

I understand the motivation, it's well intentioned, but I think there are better longer term approaches to reduce the technical debt by exploring use of the memory access and foreign API.

?

It may be that with method patterns (something not yet proposed in a JEP) we could embed the knowledge and match, in totality, using ByteBuffer.isDirect(). Something to explore if/when such a pattern feature is available.

Paul.


> On Jul 23, 2021, at 2:48 PM, Sebastian Stenzel <sebastian.stenzel at gmail.com> wrote:
> 
> Sure, we can just assert that any ByteBuffer that isn't direct must be a HeapByteBuffer. But this assertion is based on our knowlege. The point is, that we could use a mechanism which would instead cause a compile time error if this ever changes. In my opinion this is the stronger tool to ensure correctness than relying on every contributor to know that only these existing implementations of ByteBuffers are allowed.
> 
> This change has no direct functional effect, it merely helps to reduce branches that are technically dead code and just exist to satisfy ByteBuffer's contract. In other words: There is no functional advantage. This is about reducing technical debt.
> 
>> Am 23.07.2021 um 17:46 schrieb Paul Sandoz <paul.sandoz at oracle.com>:
>> 
>> ?I don?t see much advantage to this. Most of the (byte) buffer hierarchy is package private. Further, the hierarchy is set up for convenience: DirectByteBuffer extends MappedByteBuffer [*], even though not all DirectByteBuffer instances are mapped.
>> 
>> Internally we can assume if a ByteBuffer instance is also an instance sun.nio.ch.DirectBuffer (which in turn implies that ByteBuffer.isDirect() == true) then it?s a direct buffer. Otherwise, its a heap buffer and if hasArray() == true then the underlying array is accessible.
>> 
>> In the referenced code there is a precondition that the byte buffer is not read-only. Therefore, we can assume for a heap buffer the src array is accessible (use an assert if you wish).
>> 
>> My hope is this kind of code can improve when we can use MemorySegment.
>> 
>> Paul.
>> 
>> [*]
>> public abstract class MappedByteBuffer
>>   extends ByteBuffer
>> {
>> 
>>   // This is a little bit backwards: By rights MappedByteBuffer should be a
>>   // subclass of DirectByteBuffer, but to keep the spec clear and simple, and
>>   // for optimization purposes, it's easier to do it the other way around.
>>   // This works because DirectByteBuffer is a package-private class.
>> 
>>> On Jul 23, 2021, at 3:59 AM, Sebastian Stenzel <sebastian.stenzel at gmail.com> wrote:
>>> 
>>> Any thoughts on this?
>>> 
>>>>> On 8. Jul 2021, at 16:54, Sebastian Stenzel <sebastian.stenzel at gmail.com> wrote:
>>>> 
>>>> A while back, when working on JDK-8264110, we had a discussion about the nature of ByteBuffers, where I asked whether we could eventually make this a sealed class. [1]
>>>> 
>>>> The idea is, that basically there are only two kinds of ByteBuffers:
>>>> - HeapByteBuffers
>>>> - MappedByteBuffer ? DirectByteBuffer
>>>> 
>>>> While the former is backed by an array, the latter isn't, which leads to branching such as `if (buffer.hasArray())`, which could be simplified, if we knew for sure to deal with either of the two kinds.
>>>> 
>>>> Now, that sealed classes are stable with JDK 17, I'd like to ask for your approval to make ByteBuffer a sealed class, so we can rely on an exhaustive list of ByteBuffer "flavours" at compile time.
>>>> 
>>>> This is a requirement for follow-up tasks, I'd like to work on, such as JDK-8264341.
>>>> 
>>>> 
>>>> [1] https://urldefense.com/v3/__https://github.com/openjdk/jdk/pull/3217*discussion_r602528233__;Iw!!ACWV5N9M2RV99hQ!d_JnBuyAJIsQrELaJHP-M0nHv3V2m3DSmNuEbXwf9IUGjQhCS_2YioIA2Vs5gTysvA$ 
>>> 
>> 


From sebastian.stenzel at gmail.com  Mon Jul 26 16:07:42 2021
From: sebastian.stenzel at gmail.com (Sebastian Stenzel)
Date: Mon, 26 Jul 2021 18:07:42 +0200
Subject: Make java.nio.ByteBuffer a sealed class
In-Reply-To: <81996939-A032-4C61-8A35-AD007F8DCB4D@oracle.com>
References: <81996939-A032-4C61-8A35-AD007F8DCB4D@oracle.com>
Message-ID: <C9E96B9C-AC8F-4591-AC22-241C262C4F13@gmail.com>

I see, then I will add some asserts for now and remove the dead code and leave ByteBuffers untouched. :-)

> Am 26.07.2021 um 17:54 schrieb Paul Sandoz <paul.sandoz at oracle.com>:
> 
> ?I understand the motivation, it's well intentioned, but I think there are better longer term approaches to reduce the technical debt by exploring use of the memory access and foreign API.
> 
> ?
> 
> It may be that with method patterns (something not yet proposed in a JEP) we could embed the knowledge and match, in totality, using ByteBuffer.isDirect(). Something to explore if/when such a pattern feature is available.
> 
> Paul.
> 
> 
>> On Jul 23, 2021, at 2:48 PM, Sebastian Stenzel <sebastian.stenzel at gmail.com> wrote:
>> 
>> Sure, we can just assert that any ByteBuffer that isn't direct must be a HeapByteBuffer. But this assertion is based on our knowlege. The point is, that we could use a mechanism which would instead cause a compile time error if this ever changes. In my opinion this is the stronger tool to ensure correctness than relying on every contributor to know that only these existing implementations of ByteBuffers are allowed.
>> 
>> This change has no direct functional effect, it merely helps to reduce branches that are technically dead code and just exist to satisfy ByteBuffer's contract. In other words: There is no functional advantage. This is about reducing technical debt.
>> 
>>>> Am 23.07.2021 um 17:46 schrieb Paul Sandoz <paul.sandoz at oracle.com>:
>>> 
>>> ?I don?t see much advantage to this. Most of the (byte) buffer hierarchy is package private. Further, the hierarchy is set up for convenience: DirectByteBuffer extends MappedByteBuffer [*], even though not all DirectByteBuffer instances are mapped.
>>> 
>>> Internally we can assume if a ByteBuffer instance is also an instance sun.nio.ch.DirectBuffer (which in turn implies that ByteBuffer.isDirect() == true) then it?s a direct buffer. Otherwise, its a heap buffer and if hasArray() == true then the underlying array is accessible.
>>> 
>>> In the referenced code there is a precondition that the byte buffer is not read-only. Therefore, we can assume for a heap buffer the src array is accessible (use an assert if you wish).
>>> 
>>> My hope is this kind of code can improve when we can use MemorySegment.
>>> 
>>> Paul.
>>> 
>>> [*]
>>> public abstract class MappedByteBuffer
>>>  extends ByteBuffer
>>> {
>>> 
>>>  // This is a little bit backwards: By rights MappedByteBuffer should be a
>>>  // subclass of DirectByteBuffer, but to keep the spec clear and simple, and
>>>  // for optimization purposes, it's easier to do it the other way around.
>>>  // This works because DirectByteBuffer is a package-private class.
>>> 
>>>> On Jul 23, 2021, at 3:59 AM, Sebastian Stenzel <sebastian.stenzel at gmail.com> wrote:
>>>> 
>>>> Any thoughts on this?
>>>> 
>>>>>> On 8. Jul 2021, at 16:54, Sebastian Stenzel <sebastian.stenzel at gmail.com> wrote:
>>>>> 
>>>>> A while back, when working on JDK-8264110, we had a discussion about the nature of ByteBuffers, where I asked whether we could eventually make this a sealed class. [1]
>>>>> 
>>>>> The idea is, that basically there are only two kinds of ByteBuffers:
>>>>> - HeapByteBuffers
>>>>> - MappedByteBuffer ? DirectByteBuffer
>>>>> 
>>>>> While the former is backed by an array, the latter isn't, which leads to branching such as `if (buffer.hasArray())`, which could be simplified, if we knew for sure to deal with either of the two kinds.
>>>>> 
>>>>> Now, that sealed classes are stable with JDK 17, I'd like to ask for your approval to make ByteBuffer a sealed class, so we can rely on an exhaustive list of ByteBuffer "flavours" at compile time.
>>>>> 
>>>>> This is a requirement for follow-up tasks, I'd like to work on, such as JDK-8264341.
>>>>> 
>>>>> 
>>>>> [1] https://urldefense.com/v3/__https://github.com/openjdk/jdk/pull/3217*discussion_r602528233__;Iw!!ACWV5N9M2RV99hQ!d_JnBuyAJIsQrELaJHP-M0nHv3V2m3DSmNuEbXwf9IUGjQhCS_2YioIA2Vs5gTysvA$ 
>>>> 
>>> 
> 

From github.com+1701815+mkarg at openjdk.java.net  Mon Jul 26 18:02:29 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Mon, 26 Jul 2021 18:02:29 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v3]
In-Reply-To: <aS5i72i7KLnTndIFIA_94OUjEgyuwXiHDHu1H898QtE=.0fcdd01c-2219-4318-86d7-8826941b6f41@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <aS5i72i7KLnTndIFIA_94OUjEgyuwXiHDHu1H898QtE=.0fcdd01c-2219-4318-86d7-8826941b6f41@github.com>
Message-ID: <MexwixZ1HWTbPlrkcrR4LaoNgjNiRKAICIb5trE5ewk=.98ec3da1-b6de-4cb8-a329-d93f6567ca3e@github.com>

On Fri, 2 Jul 2021 06:20:29 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
>> 
>> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
>> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
>> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
>> 
>> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
>> 
>> I encourage everybody to discuss this draft:
>> * Are there valid arguments for *not* doing this change?
>> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
>> * How to go on from here: What is missing to get this ready for an actual review?
>
> Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR.

Sorry for the delay. I now have pushed a first draft of the test. IIUC then five implementation have to be tested just for the actualy API, i. e. whether they throws NPE when `out` is `null`, and whether they transfer small, single turn, and multiple turn loads correctly. The tests are taken from `InputStream/TransferTo.java` as you proposed, and I am not using mocking but instead use `Channels.new*` as the existing tests you proposed as a blueprint. I hope I understood your requests and proposals correctly, if not please tell me. The tests pass well and proof that the new implementations work as expected and API-compliant.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Mon Jul 26 18:02:27 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Mon, 26 Jul 2021 18:02:27 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v4]
In-Reply-To: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
Message-ID: <7PBopxz12bSrundwCfzqD7QBYqhVU39CYehUGIYGOwg=.632fc69f-2dfd-41df-aace-ad8965b2139b@github.com>

> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
> 
> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
> 
> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
> 
> I encourage everybody to discuss this draft:
> * Are there valid arguments for *not* doing this change?
> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
> * How to go on from here: What is missing to get this ready for an actual review?

Markus KARG has updated the pull request incrementally with four additional commits since the last revision:

 - Draft: Test for ChannelInputStream::transferTo
 - Draft: Using Channels API to invoke sun.nio.ch classes
 - Draft: MUST check params before ANY other operation
 - Draft: ChannelInputStream::transferTo Test

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4263/files
  - new: https://git.openjdk.java.net/jdk/pull/4263/files/47ee00a2..b431fcd6

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=03
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=02-03

  Stats: 229 lines in 2 files changed: 229 ins; 0 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4263.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4263/head:pull/4263

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Mon Jul 26 20:24:51 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Mon, 26 Jul 2021 20:24:51 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v5]
In-Reply-To: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
Message-ID: <_ImTEjqgdig9vcnINDYEXqG2y0WYzY-fYJ8uOUlvnis=.6f8a51cc-12d9-4037-9078-0a3ebd71d4c3@github.com>

> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
> 
> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
> 
> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
> 
> I encourage everybody to discuss this draft:
> * Are there valid arguments for *not* doing this change?
> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
> * How to go on from here: What is missing to get this ready for an actual review?

Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision:

  Draft: Test for ChannelInputStream::transferTo

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4263/files
  - new: https://git.openjdk.java.net/jdk/pull/4263/files/b431fcd6..bfc699a0

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=04
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=03-04

  Stats: 227 lines in 1 file changed: 0 ins; 0 del; 227 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4263.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4263/head:pull/4263

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Mon Jul 26 20:45:14 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Mon, 26 Jul 2021 20:45:14 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
Message-ID: <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>

> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
> 
> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
> 
> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
> 
> I encourage everybody to discuss this draft:
> * Are there valid arguments for *not* doing this change?
> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
> * How to go on from here: What is missing to get this ready for an actual review?

Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision:

  Draft: Test for ChannelInputStream::transferTo

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4263/files
  - new: https://git.openjdk.java.net/jdk/pull/4263/files/bfc699a0..1f9dba3d

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=05
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=04-05

  Stats: 173 lines in 1 file changed: 6 ins; 0 del; 167 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4263.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4263/head:pull/4263

PR: https://git.openjdk.java.net/jdk/pull/4263

From lancea at openjdk.java.net  Mon Jul 26 21:16:31 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Mon, 26 Jul 2021 21:16:31 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v11]
In-Reply-To: <27fDIfBum2vSv7DmFs_PH8Z27NAYMVeMUmAa6OOJgfI=.b125d8c7-83a0-48d5-8342-d0da696fdbff@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <eglX2CJeKvHWYuASSDER6ClGtBUbYk3YVD9cpensiDY=.2450e7d6-9c0e-4de6-a907-d878f21a0e6e@github.com>
 <2ttMZAIqSLz7XfsDi0CQjsaApF19Vk68hNNgh_WDIw0=.a4b275b9-3ce9-4101-9dfd-0db920386eb1@github.com>
 <27fDIfBum2vSv7DmFs_PH8Z27NAYMVeMUmAa6OOJgfI=.b125d8c7-83a0-48d5-8342-d0da696fdbff@github.com>
Message-ID: <0hitV3isecJqQy6DhbOYBrBufoIj5I0xNINiPBzkbLA=.2fb39881-8a70-46b2-ad43-d0b090941667@github.com>

On Fri, 23 Jul 2021 16:22:01 GMT, Lance Andersen <lancea at openjdk.org> wrote:

> > Thank you for the review Alan.
> > @LanceAndersen, I've run the tier1 tests locally with the latest PR and they have passed without any regressions. Given that we changed the implementation to wrap ByteArrayOutputStream instead of extending it, would you want to rerun some of your other tests that you had previously run, before I integrate this?
> 
> Yes, I will run additional tests and report back after they complete

I did not notice any new issues after running tier1 - tier3

-------------

PR: https://git.openjdk.java.net/jdk/pull/4607

From bpb at openjdk.java.net  Tue Jul 27 00:02:39 2021
From: bpb at openjdk.java.net (Brian Burkhalter)
Date: Tue, 27 Jul 2021 00:02:39 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
Message-ID: <IZKluRfiLnIovyrAw3TdXDSzB6e59nHmG3ee7PEtjSE=.a778bb70-9938-4599-a3d9-9ad8fa235b00@github.com>

On Mon, 26 Jul 2021 20:45:14 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
>> 
>> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
>> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
>> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
>> 
>> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
>> 
>> I encourage everybody to discuss this draft:
>> * Are there valid arguments for *not* doing this change?
>> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
>> * How to go on from here: What is missing to get this ready for an actual review?
>
> Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR.

src/java.base/share/classes/sun/nio/ch/ChannelInputStream.java line 179:

> 177:             for (long n = srcSize - srcPos; bytesWritten < n;)
> 178:                 bytesWritten += src.transferTo(srcPos + bytesWritten, Long.MAX_VALUE, dest);
> 179:             return bytesWritten;

If `src` is extended at an inconvenient point in time, for example between the return from a call to `src.transferTo()` that makes `bytesWritten < n` false and the evaluation of that terminating condition, then it appears that not all the content of `src` will be transferred to `dest`. I am not however sure that this violates any contract but it could be a behavioral change from the existing implementation.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From bpb at openjdk.java.net  Tue Jul 27 01:00:38 2021
From: bpb at openjdk.java.net (Brian Burkhalter)
Date: Tue, 27 Jul 2021 01:00:38 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
Message-ID: <eqqcs-9h_-p55lUyHJmmQgqbLCZ48MyBtDPi9eZH_bM=.17d7f6d4-6a37-4350-b774-e7040c85cab9@github.com>

On Mon, 26 Jul 2021 20:45:14 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
>> 
>> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
>> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
>> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
>> 
>> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
>> 
>> I encourage everybody to discuss this draft:
>> * Are there valid arguments for *not* doing this change?
>> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
>> * How to go on from here: What is missing to get this ready for an actual review?
>
> Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR.

src/java.base/share/classes/sun/nio/ch/ChannelOutputStream.java line 85:

> 83:     private byte[] bs;       // Invoker's previous array
> 84:     private byte[] b1;
> 85: 

It might be better to put the field declarations at the beginning of the class before the static methods.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From jpai at openjdk.java.net  Tue Jul 27 01:57:52 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Tue, 27 Jul 2021 01:57:52 GMT
Subject: RFR: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream [v11]
In-Reply-To: <eglX2CJeKvHWYuASSDER6ClGtBUbYk3YVD9cpensiDY=.2450e7d6-9c0e-4de6-a907-d878f21a0e6e@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
 <eglX2CJeKvHWYuASSDER6ClGtBUbYk3YVD9cpensiDY=.2450e7d6-9c0e-4de6-a907-d878f21a0e6e@github.com>
Message-ID: <B1yo3mGV6Q_Pb46wGa3o8JB_-iWlVio7asluZHA-TOo=.fcd71ffb-338c-41bf-b197-b7f2f2f79859@github.com>

On Fri, 23 Jul 2021 14:58:01 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
>> 
>> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
>> 
>> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146
>
> Jaikiran Pai has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 15 additional commits since the last revision:
> 
>  - Merge latest from master branch
>  - Implement review suggestions:
>     - remove unnecessary "synchronized"
>     - no need to open the temp file twice
>  - remove no longer necessary javadoc comment on test
>  - review suggestion - wrap ByteArrayOutputStream instead of extending it
>  - reorganize the tests now that the temp file creation threshold isn't exposed as a user configurable value
>  - minor update to comment on FileRolloverOutputStream class
>  - remove no longer used constant
>  - use jtreg's construct of manual test
>  - Implement Alan's review suggestion - don't expose the threshold as a configuration property, instead set it internally to a specific value
>  - propagate back the original checked IOException to the callers
>  - ... and 5 more: https://git.openjdk.java.net/jdk/compare/f875aac8...991de6b9

Thank you for running the tests, Lance.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4607

From jpai at openjdk.java.net  Tue Jul 27 02:00:39 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Tue, 27 Jul 2021 02:00:39 GMT
Subject: Integrated: 8190753: (zipfs): Accessing a large entry (> 2^31 bytes)
 leads to a negative initial size for ByteArrayOutputStream
In-Reply-To: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
References: <0GJVLeCPoDI4FePU-Z-NMwJHaS2XyH9S28LgqI1LNB8=.e32743dc-a6dd-475d-88ab-16ed3f46beab@github.com>
Message-ID: <waIETIaMapLWj63rHOjx2Vcf7JyKFaW4LwVICki2E9w=.9d4557aa-a4f3-47f4-ad21-4a3ea2a5243d@github.com>

On Mon, 28 Jun 2021 03:41:20 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> Can I please get a review for this proposed fix for the issue reported in https://bugs.openjdk.java.net/browse/JDK-8190753?
> 
> The commit here checks for the size of the zip entry before trying to create a `ByteArrayOutputStream` for that entry's content. A new jtreg test has been included in this commit to reproduce the issue and verify the fix.
> 
> P.S: It's still a bit arguable whether it's a good idea to create the `ByteArrayOutputStream` with an initial size potentially as large as the `MAX_ARRAY_SIZE` or whether it's better to just use some smaller default value. However, I think that can be addressed separately while dealing with https://bugs.openjdk.java.net/browse/JDK-8011146

This pull request has now been integrated.

Changeset: c3d8e922
Author:    Jaikiran Pai <jpai at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/c3d8e9228d0558a2ce3e093c105c61ea7af2e1d1
Stats:     356 lines in 3 files changed: 350 ins; 0 del; 6 mod

8190753: (zipfs): Accessing a large entry (> 2^31 bytes) leads to a negative initial size for ByteArrayOutputStream

Reviewed-by: lancea

-------------

PR: https://git.openjdk.java.net/jdk/pull/4607

From bpb at openjdk.java.net  Tue Jul 27 02:22:32 2021
From: bpb at openjdk.java.net (Brian Burkhalter)
Date: Tue, 27 Jul 2021 02:22:32 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
Message-ID: <g2LuISWVJcLFeSuUXfmfeqe7wl7kvcKsVgH2WXI90p4=.14176736-5173-41f3-b859-64cb9b986623@github.com>

On Mon, 26 Jul 2021 20:45:14 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
>> 
>> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
>> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
>> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
>> 
>> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
>> 
>> I encourage everybody to discuss this draft:
>> * Are there valid arguments for *not* doing this change?
>> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
>> * How to go on from here: What is missing to get this ready for an actual review?
>
> Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR.

test/jdk/sun/nio/ch/ChannelInputStream/TransferTo.java line 67:

> 65:         test(readableByteChannelInput(), defaultOutput());
> 66:     }
> 67: 

This test looks like it's doing what it's supposed to do. I'm not asking to change it, but I think using TestNG might have given a simpler result with less work. For example, there could be one `DataProvider` supplying inputs which feed a `@Test` which expects an NPE, and another `DataProvider` supplying input-output pairs which feeds a `@Test` which validates the functionality.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From mik3hall at gmail.com  Thu Jul 29 00:02:09 2021
From: mik3hall at gmail.com (Michael Hall)
Date: Wed, 28 Jul 2021 19:02:09 -0500
Subject: New release DefaultFileSystemProviders
Message-ID: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>

Have changes been made to the new jdk17 and jdk18 releases for the jdk DefaultFileSystemProvider test?

I was asked to verify a bug against the latest releases for DefaultFileSystemProvider related code. 
The code is not itself modular but includes some optional modular parts. 
It gets one error if run without any modular and a different one if run with these modular parts.
That error with modular is?

Error: LinkageError occurred while loading main class us.hall.hp.common.LoaderLaunchStub
	java.lang.ExceptionInInitializerError: null

At first I thought this was a native link error but after looking it up it seems more like a serialization type error where some current class doesn?t match a prior one?
So I am wondering if recent releases have made changes that make my older provider derived code incompatible?
These would of probably been indicated by changes made to the jdk DefaulFileSystemProvider test case to keep that working?
I don?t keep current with the latest development release sources or I could probably try comparing myself.
If helpful verbose class loading that seems to lead up to the error on OS/X shows?

[0.076s][info][class,load] jdk.internal.loader.NativeLibraries$NativeLibraryContext$$Lambda$5/0x000000080003c208 source: jdk.internal.loader.NativeLibraries
[0.076s][info][class,load] java.util.ArrayDeque$DeqIterator source: jrt:/java.base
[0.076s][info][class,load] jdk.internal.loader.NativeLibrary source: jrt:/java.base
[0.076s][info][class,load] jdk.internal.loader.NativeLibraries$NativeLibraryImpl source: jrt:/java.base
[0.077s][info][class,load] jdk.internal.loader.NativeLibraries$$Lambda$6/0x000000080003caa0 source: jdk.internal.loader.NativeLibraries
[0.078s][info][class,load] java.util.concurrent.ConcurrentHashMap$ValuesView source: jrt:/java.base
[0.078s][info][class,load] java.util.Enumeration source: jrt:/java.base
[0.078s][info][class,load] java.util.concurrent.ConcurrentHashMap$Traverser source: jrt:/java.base
[0.078s][info][class,load] java.util.concurrent.ConcurrentHashMap$BaseIterator source: jrt:/java.base
[0.078s][info][class,load] java.util.concurrent.ConcurrentHashMap$ValueIterator source: jrt:/java.base
[0.078s][info][class,load] java.nio.file.attribute.BasicFileAttributes source: jrt:/java.base
[0.078s][info][class,load] java.nio.file.attribute.PosixFileAttributes source: jrt:/java.base
[0.078s][info][class,load] sun.nio.fs.UnixFileAttributes source: jrt:/java.base
[0.078s][info][class,load] sun.nio.fs.UnixFileStoreAttributes source: jrt:/java.base
[0.078s][info][class,load] sun.nio.fs.UnixMountEntry source: jrt:/java.base
[0.078s][info][class,load] jdk.internal.loader.BuiltinClassLoader$5 source: jrt:/java.base
[0.079s][info][class,load] java.lang.module.ModuleReader source: jrt:/java.base
[0.079s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemModuleReader source: jrt:/java.base
[0.079s][info][class,load] jdk.internal.module.ModulePatcher$PatchedModuleReader source: jrt:/java.base
[0.079s][info][class,load] jdk.internal.module.SystemModuleFinders$SystemImage source: jrt:/java.base
[0.079s][info][class,load] jdk.internal.jimage.ImageReaderFactory source: jrt:/java.base
[0.079s][info][class,load] java.nio.file.Paths source: jrt:/java.base
[0.079s][info][class,load] java.lang.ExceptionInInitializerError source: jrt:/java.base

From this it still almost seems to indicate a problem in loading a native library but I commented out the code that does that in the main class and the error remained.





From brian.burkhalter at oracle.com  Thu Jul 29 00:28:55 2021
From: brian.burkhalter at oracle.com (Brian Burkhalter)
Date: Thu, 29 Jul 2021 00:28:55 +0000
Subject: New release DefaultFileSystemProviders
In-Reply-To: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
Message-ID: <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>

Do you mean the test SetDefaultProvider.java? Is so there was one change earlier this month [1] but it does not appear related to the problems you are seeing.

[1] https://github.com/openjdk/jdk17/commit/4fc3180f75e1cea4ebd613f8253be205d95f830c#diff-1b5076b5b17cb0adeb60349df3d2a50387b14bb2af0406ce4c5d0a577281e97c

On Jul 28, 2021, at 5:02 PM, Michael Hall <mik3hall at gmail.com<mailto:mik3hall at gmail.com>> wrote:

Have changes been made to the new jdk17 and jdk18 releases for the jdk DefaultFileSystemProvider test?

I was asked to verify a bug against the latest releases for DefaultFileSystemProvider related code.
The code is not itself modular but includes some optional modular parts.
It gets one error if run without any modular and a different one if run with these modular parts.
That error with modular is?

Error: LinkageError occurred while loading main class us.hall.hp.common.LoaderLaunchStub
java.lang.ExceptionInInitializerError: null

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210729/dcc49808/attachment-0001.htm>

From mik3hall at gmail.com  Thu Jul 29 00:36:13 2021
From: mik3hall at gmail.com (Michael Hall)
Date: Wed, 28 Jul 2021 19:36:13 -0500
Subject: New release DefaultFileSystemProviders
In-Reply-To: <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
Message-ID: <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>



> On Jul 28, 2021, at 7:28 PM, Brian Burkhalter <brian.burkhalter at oracle.com> wrote:
> 
> Do you mean the test SetDefaultProvider.java? Is so there was one change earlier this month [1] but it does not appear related to the problems you are seeing.
> 
> [1] https://github.com/openjdk/jdk17/commit/4fc3180f75e1cea4ebd613f8253be205d95f830c#diff-1b5076b5b17cb0adeb60349df3d2a50387b14bb2af0406ce4c5d0a577281e97c <https://github.com/openjdk/jdk17/commit/4fc3180f75e1cea4ebd613f8253be205d95f830c#diff-1b5076b5b17cb0adeb60349df3d2a50387b14bb2af0406ce4c5d0a577281e97c>
Related to the right bug but as you say not seeming to be related to mine.

I did verify that bug is gone for your test provider.

This is running against that with my modular turned on.

/usr/libexec/java_home --exec java -cp . -Djava.security.manager -Djava.security.policy=all.policy -Djava.nio.file.spi.DefaultFileSystemProvider=TestProvider --module-path mods --add-modules us.hall.trz.osx,org.openjdk.nashorn,java.compiler,java.desktop,java.logging,java.management,java.prefs,java.se,java.rmi,java.scripting,java.sql,java.xml,jdk.attach,jdk.jshell,jdk.crypto.ec,jdk.jdeps,jdk.jcmd  --add-exports org.openjdk.nashorn/org.openjdk.nashorn.tools=ALL-UNNAMED org.test.Test 
WARNING: A command line option has enabled the Security Manager
WARNING: The Security Manager is deprecated and will be removed in a future release
TestProvider$TestFileSystem at 5b6f7412
TestProvider

Runs. Something different about mine somewhere.
With your link maybe I can compare my source against current on the browser.

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210728/ffeb4833/attachment.htm>

From mik3hall at gmail.com  Thu Jul 29 00:46:13 2021
From: mik3hall at gmail.com (Michael Hall)
Date: Wed, 28 Jul 2021 19:46:13 -0500
Subject: New release DefaultFileSystemProviders
In-Reply-To: <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
 <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
Message-ID: <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>



> On Jul 28, 2021, at 7:36 PM, Michael Hall <mik3hall at gmail.com> wrote:
> 
> 
> 
>> On Jul 28, 2021, at 7:28 PM, Brian Burkhalter <brian.burkhalter at oracle.com <mailto:brian.burkhalter at oracle.com>> wrote:
>> 
>> Do you mean the test SetDefaultProvider.java? Is so there was one change earlier this month [1] but it does not appear related to the problems you are seeing.
>> 
>> [1] https://github.com/openjdk/jdk17/commit/4fc3180f75e1cea4ebd613f8253be205d95f830c#diff-1b5076b5b17cb0adeb60349df3d2a50387b14bb2af0406ce4c5d0a577281e97c <https://github.com/openjdk/jdk17/commit/4fc3180f75e1cea4ebd613f8253be205d95f830c#diff-1b5076b5b17cb0adeb60349df3d2a50387b14bb2af0406ce4c5d0a577281e97c>
> Related to the right bug but as you say not seeming to be related to mine.
> 
> I did verify that bug is gone for your test provider.
> 
> This is running against that with my modular turned on.
> 
> /usr/libexec/java_home --exec java -cp . -Djava.security.manager -Djava.security.policy=all.policy -Djava.nio.file.spi.DefaultFileSystemProvider=TestProvider --module-path mods --add-modules us.hall.trz.osx,org.openjdk.nashorn,java.compiler,java.desktop,java.logging,java.management,java.prefs,java.se <http://java.se/>,java.rmi,java.scripting,java.sql,java.xml,jdk.attach,jdk.jshell,jdk.crypto.ec,jdk.jdeps,jdk.jcmd  --add-exports org.openjdk.nashorn/org.openjdk.nashorn.tools=ALL-UNNAMED org.test.Test 
> WARNING: A command line option has enabled the Security Manager
> WARNING: The Security Manager is deprecated and will be removed in a future release
> TestProvider$TestFileSystem at 5b6f7412
> TestProvider
> 
> Runs. Something different about mine somewhere.
> With your link maybe I can compare my source against current on the browser.
> 
> Thanks.

If of interest, change that test to mine.

/usr/libexec/java_home --exec java -cp .:macnio2 -Djava.security.manager -Djava.security.policy=all.policy -Djava.nio.file.spi.DefaultFileSystemProvider=us.hall.trz.osx.MacFileSystemProvider --module-path mods --add-modules us.hall.trz.osx,org.openjdk.nashorn,java.compiler,java.desktop,java.logging,java.management,java.prefs,java.se,java.rmi,java.scripting,java.sql,java.xml,jdk.attach,jdk.jshell,jdk.crypto.ec,jdk.jdeps,jdk.jcmd  --add-exports org.openjdk.nashorn/org.openjdk.nashorn.tools=ALL-UNNAMED org.test.Test 
Error occurred during initialization of VM
java.lang.Error: java.lang.NullPointerException: Cannot invoke "java.nio.file.FileSystem.getPath(String, String[])" because the return value of "java.nio.file.FileSystems.getDefault()" is null
	at java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(java.base at 18-ea/FileSystems.java:134)
	at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(java.base at 18-ea/FileSystems.java:103)
	at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(java.base at 18-ea/FileSystems.java:101)
	at java.security.AccessController.doPrivileged(java.base at 18-ea/AccessController.java:318)
	at java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(java.base at 18-ea/FileSystems.java:101)
	at java.nio.file.FileSystems$DefaultFileSystemHolder.<clinit>(java.base at 18-ea/FileSystems.java:94)
	at java.nio.file.FileSystems.getDefault(java.base at 18-ea/FileSystems.java:183)
	at java.nio.file.Path.of(java.base at 18-ea/Path.java:147)
	at java.nio.file.Paths.get(java.base at 18-ea/Paths.java:69)
	at jdk.internal.jimage.ImageReaderFactory.<clinit>(java.base at 18-ea/ImageReaderFactory.java:51)
	at jdk.internal.module.SystemModuleFinders$SystemImage.<clinit>(java.base at 18-ea/SystemModuleFinders.java:385)
	at jdk.internal.module.SystemModuleFinders$SystemModuleReader.containsImageLocation(java.base at 18-ea/SystemModuleFinders.java:446)
	at jdk.internal.module.SystemModuleFinders$SystemModuleReader.find(java.base at 18-ea/SystemModuleFinders.java:457)
	at jdk.internal.loader.BuiltinClassLoader.findResource(java.base at 18-ea/BuiltinClassLoader.java:497)
	at jdk.internal.loader.BuiltinClassLoader.findResource(java.base at 18-ea/BuiltinClassLoader.java:277)
	at jdk.internal.loader.BootLoader.findResource(java.base at 18-ea/BootLoader.java:164)
	at java.lang.Class.getResource(java.base at 18-ea/Class.java:2925)
	at java.lang.System.implSetSecurityManager(java.base at 18-ea/System.java:418)
	at java.lang.System.initPhase3(java.base at 18-ea/System.java:2203)
Caused by: java.lang.NullPointerException
	at java.io.File.toPath(java.base at 18-ea/File.java:2396)
	at java.util.zip.ZipFile$Source.get(java.base at 18-ea/ZipFile.java:1246)
	at java.util.zip.ZipFile$CleanableResource.<init>(java.base at 18-ea/ZipFile.java:708)
	at java.util.zip.ZipFile.<init>(java.base at 18-ea/ZipFile.java:242)
	at java.util.zip.ZipFile.<init>(java.base at 18-ea/ZipFile.java:172)
	at java.util.jar.JarFile.<init>(java.base at 18-ea/JarFile.java:347)
	at jdk.internal.module.ModuleReferences$JarModuleReader.newJarFile(java.base at 18-ea/ModuleReferences.java:230)
	at jdk.internal.module.ModuleReferences$JarModuleReader.<init>(java.base at 18-ea/ModuleReferences.java:237)
	at jdk.internal.module.ModuleReferences.lambda$newJarModule$0(java.base at 18-ea/ModuleReferences.java:94)
	at jdk.internal.module.ModuleReferenceImpl.open(java.base at 18-ea/ModuleReferenceImpl.java:93)
	at jdk.internal.loader.BuiltinClassLoader$5.apply(java.base at 18-ea/BuiltinClassLoader.java:1023)
	at jdk.internal.loader.BuiltinClassLoader$5.apply(java.base at 18-ea/BuiltinClassLoader.java:1020)
	at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(java.base at 18-ea/ConcurrentHashMap.java:1708)
	at jdk.internal.loader.BuiltinClassLoader.moduleReaderFor(java.base at 18-ea/BuiltinClassLoader.java:1031)
	at jdk.internal.loader.BuiltinClassLoader.defineClass(java.base at 18-ea/BuiltinClassLoader.java:793)
	at jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(java.base at 18-ea/BuiltinClassLoader.java:741)
	at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base at 18-ea/BuiltinClassLoader.java:665)
	at jdk.internal.loader.BuiltinClassLoader.loadClass(java.base at 18-ea/BuiltinClassLoader.java:639)
	at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(java.base at 18-ea/ClassLoaders.java:188)
	at java.lang.ClassLoader.loadClass(java.base at 18-ea/ClassLoader.java:520)
	at java.lang.Class.forName0(java.base at 18-ea/Native Method)
	at java.lang.Class.forName(java.base at 18-ea/Class.java:467)
	at java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(java.base at 18-ea/FileSystems.java:124)
	at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(java.base at 18-ea/FileSystems.java:103)
	at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(java.base at 18-ea/FileSystems.java:101)
	at java.security.AccessController.doPrivileged(java.base at 18-ea/AccessController.java:318)
	at java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(java.base at 18-ea/FileSystems.java:101)
	at java.nio.file.FileSystems$DefaultFileSystemHolder.<clinit>(java.base at 18-ea/FileSystems.java:94)
	at java.nio.file.FileSystems.getDefault(java.base at 18-ea/FileSystems.java:183)
	at java.nio.file.Path.of(java.base at 18-ea/Path.java:147)
	at java.nio.file.Paths.get(java.base at 18-ea/Paths.java:69)
	at jdk.internal.jimage.ImageReaderFactory.<clinit>(java.base at 18-ea/ImageReaderFactory.java:51)
	at jdk.internal.module.SystemModuleFinders$SystemImage.<clinit>(java.base at 18-ea/SystemModuleFinders.java:385)
	at jdk.internal.module.SystemModuleFinders$SystemModuleReader.containsImageLocation(java.base at 18-ea/SystemModuleFinders.java:446)
	at jdk.internal.module.SystemModuleFinders$SystemModuleReader.find(java.base at 18-ea/SystemModuleFinders.java:457)
	at jdk.internal.loader.BuiltinClassLoader.findResource(java.base at 18-ea/BuiltinClassLoader.java:497)
	at jdk.internal.loader.BuiltinClassLoader.findResource(java.base at 18-ea/BuiltinClassLoader.java:277)
	at jdk.internal.loader.BootLoader.findResource(java.base at 18-ea/BootLoader.java:164)
	at java.lang.Class.getResource(java.base at 18-ea/Class.java:2925)
	at java.lang.System.implSetSecurityManager(java.base at 18-ea/System.java:418)
	at java.lang.System.initPhase3(java.base at 18-ea/System.java:2203)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210728/d558c8f8/attachment-0001.htm>

From Alan.Bateman at oracle.com  Thu Jul 29 07:52:58 2021
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 29 Jul 2021 08:52:58 +0100
Subject: New release DefaultFileSystemProviders
In-Reply-To: <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
 <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
 <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>
Message-ID: <2da73a16-df41-26c0-2d2a-ab382951a75a@oracle.com>

On 29/07/2021 01:46, Michael Hall wrote:
>
>
> If of interest, change that test to mine.
>
> /usr/libexec/java_home --exec java -cp .:macnio2 
> -Djava.security.manager -Djava.security.policy=all.policy 
> -Djava.nio.file.spi.DefaultFileSystemProvider=us.hall.trz.osx.MacFileSystemProvider 
> --module-path mods 
> --add-modules?us.hall.trz.osx,org.openjdk.nashorn,java.compiler,java.desktop,java.logging,java.management,java.prefs,java.se 
> <http://java.se>,java.rmi,java.scripting,java.sql,java.xml,jdk.attach,jdk.jshell,jdk.crypto.ec,jdk.jdeps,jdk.jcmd??--add-exports?org.openjdk.nashorn/org.openjdk.nashorn.tools=ALL-UNNAMED 
> org.test.Test
> Error occurred during initialization of VM
> java.lang.Error: java.lang.NullPointerException: Cannot invoke 
> "java.nio.file.FileSystem.getPath(String, String[])" because?the 
> return value of "java.nio.file.FileSystems.getDefault()" is null

Thanks for the stack trace, I think I understand it. It's another 
bootstrapping issue that arises when starting with a security manager, 
replacing the default file system provider, and running with an 
application module path. So three planets need to align to create the 
conditions. So while the changes in JDK-8266345 are correct, it's not 
the complete fix and there is work beyond the PolicyFile change to be 
able to startup with this configuration. I'll create another bug to 
track this and we'll try again.

-Alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210729/75896a72/attachment.htm>

From github.com+1701815+mkarg at openjdk.java.net  Thu Jul 29 08:11:33 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Thu, 29 Jul 2021 08:11:33 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <eqqcs-9h_-p55lUyHJmmQgqbLCZ48MyBtDPi9eZH_bM=.17d7f6d4-6a37-4350-b774-e7040c85cab9@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
 <eqqcs-9h_-p55lUyHJmmQgqbLCZ48MyBtDPi9eZH_bM=.17d7f6d4-6a37-4350-b774-e7040c85cab9@github.com>
Message-ID: <gRdJJ4CgTIrzXmSpyJOboD0XKhq4VZX1GQqlq8FYrws=.eb4be86d-23f7-4eab-b368-3f3efc6bff44@github.com>

On Tue, 27 Jul 2021 00:57:02 GMT, Brian Burkhalter <bpb at openjdk.org> wrote:

>> Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR.
>
> src/java.base/share/classes/sun/nio/ch/ChannelOutputStream.java line 85:
> 
>> 83:     private byte[] bs;       // Invoker's previous array
>> 84:     private byte[] b1;
>> 85: 
> 
> It might be better to put the field declarations at the beginning of the class before the static methods.

This is a question of style and personal likes. Code maintenance is much easier if variables are defined *near* to its first use, not *far away* (as in the Pascal or C language). If the reviewers want me to change it, I will do it.

> test/jdk/sun/nio/ch/ChannelInputStream/TransferTo.java line 67:
> 
>> 65:         test(readableByteChannelInput(), defaultOutput());
>> 66:     }
>> 67: 
> 
> This test looks like it's doing what it's supposed to do. I'm not asking to change it, but I think using TestNG might have given a simpler result with less work. For example, there could be one `DataProvider` supplying inputs which feed a `@Test` which expects an NPE, and another `DataProvider` supplying input-output pairs which feeds a `@Test` which validates the functionality.

No doubt, using modern tools would have spared me work, and indeed I would have chosen JUnit or TestNG if there would be a clear commitment to that tool *upfront*. For now, I see now use in reworking the code afterwards.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Thu Jul 29 08:15:32 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Thu, 29 Jul 2021 08:15:32 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <IZKluRfiLnIovyrAw3TdXDSzB6e59nHmG3ee7PEtjSE=.a778bb70-9938-4599-a3d9-9ad8fa235b00@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
 <IZKluRfiLnIovyrAw3TdXDSzB6e59nHmG3ee7PEtjSE=.a778bb70-9938-4599-a3d9-9ad8fa235b00@github.com>
Message-ID: <D-BolUEcvPGEA-JzbvXEUQTtaxUbl7duhesuVERIZc4=.4edfe8a0-8284-45cf-a2ad-993d795b14b6@github.com>

On Mon, 26 Jul 2021 23:59:05 GMT, Brian Burkhalter <bpb at openjdk.org> wrote:

>> Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR.
>
> src/java.base/share/classes/sun/nio/ch/ChannelInputStream.java line 179:
> 
>> 177:             for (long n = srcSize - srcPos; bytesWritten < n;)
>> 178:                 bytesWritten += src.transferTo(srcPos + bytesWritten, Long.MAX_VALUE, dest);
>> 179:             return bytesWritten;
> 
> If `src` is extended at an inconvenient point in time, for example between the return from a call to `src.transferTo()` that makes `bytesWritten < n` false and the evaluation of that terminating condition, then it appears that not all the content of `src` will be transferred to `dest`. I am not however sure that this violates any contract but it could be a behavioral change from the existing implementation.

The JavaDocs in `InputStream::transferTo` *cleary* tell the caller that there is **no** guarantee of *any* specific behavior in that particular case: 
>The behavior for the case where the input and/or output stream is asynchronously closed, or the thread interrupted during the transfer, is highly input and output stream specific, and therefore not specified.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From mik3hall at gmail.com  Thu Jul 29 09:36:24 2021
From: mik3hall at gmail.com (Michael Hall)
Date: Thu, 29 Jul 2021 04:36:24 -0500
Subject: New release DefaultFileSystemProviders
In-Reply-To: <2da73a16-df41-26c0-2d2a-ab382951a75a@oracle.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
 <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
 <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>
 <2da73a16-df41-26c0-2d2a-ab382951a75a@oracle.com>
Message-ID: <BB70AF11-9819-474F-9099-C7A21F3FEA81@gmail.com>



> On Jul 29, 2021, at 2:52 AM, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> 
> On 29/07/2021 01:46, Michael Hall wrote:
>> 
>> 
>> If of interest, change that test to mine.
>> 
>> /usr/libexec/java_home --exec java -cp .:macnio2 -Djava.security.manager -Djava.security.policy=all.policy -Djava.nio.file.spi.DefaultFileSystemProvider=us.hall.trz.osx.MacFileSystemProvider --module-path mods --add-modules us.hall.trz.osx,org.openjdk.nashorn,java.compiler,java.desktop,java.logging,java.management,java.prefs,java.se <http://java.se/>,java.rmi,java.scripting,java.sql,java.xml,jdk.attach,jdk.jshell,jdk.crypto.ec,jdk.jdeps,jdk.jcmd  --add-exports org.openjdk.nashorn/org.openjdk.nashorn.tools=ALL-UNNAMED org.test.Test 
>> Error occurred during initialization of VM
>> java.lang.Error: java.lang.NullPointerException: Cannot invoke "java.nio.file.FileSystem.getPath(String, String[])" because the return value of "java.nio.file.FileSystems.getDefault()" is null
> 
> Thanks for the stack trace, I think I understand it. It's another bootstrapping issue that arises when starting with a security manager, replacing the default file system provider, and running with an application module path. So three planets need to align to create the conditions. So while the changes in JDK-8266345 are correct, it's not the complete fix and there is work beyond the PolicyFile change to be able to startup with this configuration. I'll create another bug to track this and we'll try again.
> 
> -Alan

I hadn?t thought to change to the prior test before to get a stack crawl. I was getting pretty sure Astrology was involved somehow. If I can get a trace for the non-modular launch I?ll post that too. I?m pretty sure it is different.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210729/c9f80b5a/attachment-0001.htm>

From bpb at openjdk.java.net  Thu Jul 29 16:45:34 2021
From: bpb at openjdk.java.net (Brian Burkhalter)
Date: Thu, 29 Jul 2021 16:45:34 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <D-BolUEcvPGEA-JzbvXEUQTtaxUbl7duhesuVERIZc4=.4edfe8a0-8284-45cf-a2ad-993d795b14b6@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
 <IZKluRfiLnIovyrAw3TdXDSzB6e59nHmG3ee7PEtjSE=.a778bb70-9938-4599-a3d9-9ad8fa235b00@github.com>
 <D-BolUEcvPGEA-JzbvXEUQTtaxUbl7duhesuVERIZc4=.4edfe8a0-8284-45cf-a2ad-993d795b14b6@github.com>
Message-ID: <UQzAp1eaWHObRgVpa8bwHLdbdyauIE9NdpDd-MwsvUM=.cd00637c-80bf-43da-8ab3-97825a7d92d0@github.com>

On Thu, 29 Jul 2021 08:12:23 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>> src/java.base/share/classes/sun/nio/ch/ChannelInputStream.java line 179:
>> 
>>> 177:             for (long n = srcSize - srcPos; bytesWritten < n;)
>>> 178:                 bytesWritten += src.transferTo(srcPos + bytesWritten, Long.MAX_VALUE, dest);
>>> 179:             return bytesWritten;
>> 
>> If `src` is extended at an inconvenient point in time, for example between the return from a call to `src.transferTo()` that makes `bytesWritten < n` false and the evaluation of that terminating condition, then it appears that not all the content of `src` will be transferred to `dest`. I am not however sure that this violates any contract but it could be a behavioral change from the existing implementation.
>
> The JavaDocs in `InputStream::transferTo` *cleary* tell the caller that there is **no** guarantee of *any* specific behavior in that particular case: 
>>The behavior for the case where the input and/or output stream is asynchronously closed, or the thread interrupted during the transfer, is highly input and output stream specific, and therefore not specified.

Good point.

>> src/java.base/share/classes/sun/nio/ch/ChannelOutputStream.java line 85:
>> 
>>> 83:     private byte[] bs;       // Invoker's previous array
>>> 84:     private byte[] b1;
>>> 85: 
>> 
>> It might be better to put the field declarations at the beginning of the class before the static methods.
>
> This is a question of style and personal likes. Code maintenance is much easier if variables are defined *near* to its first use, not *far away* (as in the Pascal or C language). If the reviewers want me to change it, I will do it.

It's actually a matter of convention but I think it can remain as it is.

>> test/jdk/sun/nio/ch/ChannelInputStream/TransferTo.java line 67:
>> 
>>> 65:         test(readableByteChannelInput(), defaultOutput());
>>> 66:     }
>>> 67: 
>> 
>> This test looks like it's doing what it's supposed to do. I'm not asking to change it, but I think using TestNG might have given a simpler result with less work. For example, there could be one `DataProvider` supplying inputs which feed a `@Test` which expects an NPE, and another `DataProvider` supplying input-output pairs which feeds a `@Test` which validates the functionality.
>
> No doubt, using modern tools would have spared me work, and indeed I would have chosen JUnit or TestNG if there would be a clear commitment to that tool *upfront*. For now, I see now use in reworking the code afterwards.

No need to rework it now.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From lancea at openjdk.java.net  Thu Jul 29 18:21:07 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Thu, 29 Jul 2021 18:21:07 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v3]
In-Reply-To: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
Message-ID: <yDLTHJdWNZPTUfj2K0TAdMSGj21csVGcXorLKl2YVRk=.c41cf54d-28be-4414-82ab-36daa4cba473@github.com>

> Hi,
> 
> As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS
> 
> Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.
> 
> 
> Mach5 tiers 1 through 3 have been run without any errors encountered .
> 
> Best,
> Lance

Lance Andersen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:

 - Add Impl Note to Zip FS module-info
 - Merge
 - Add missing Copyright header and address minor comments
 - Address missing linefeed  after package name
 - Address overzelous intellij import update
 - Patch to address JDK-8251329

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4900/files
  - new: https://git.openjdk.java.net/jdk/pull/4900/files/2265ffe1..22731ac3

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4900&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4900&range=01-02

  Stats: 3049 lines in 76 files changed: 1357 ins; 359 del; 1333 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4900.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4900/head:pull/4900

PR: https://git.openjdk.java.net/jdk/pull/4900

From github.com+1701815+mkarg at openjdk.java.net  Thu Jul 29 20:00:34 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Thu, 29 Jul 2021 20:00:34 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <UQzAp1eaWHObRgVpa8bwHLdbdyauIE9NdpDd-MwsvUM=.cd00637c-80bf-43da-8ab3-97825a7d92d0@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
 <IZKluRfiLnIovyrAw3TdXDSzB6e59nHmG3ee7PEtjSE=.a778bb70-9938-4599-a3d9-9ad8fa235b00@github.com>
 <D-BolUEcvPGEA-JzbvXEUQTtaxUbl7duhesuVERIZc4=.4edfe8a0-8284-45cf-a2ad-993d795b14b6@github.com>
 <UQzAp1eaWHObRgVpa8bwHLdbdyauIE9NdpDd-MwsvUM=.cd00637c-80bf-43da-8ab3-97825a7d92d0@github.com>
Message-ID: <AC0DI4y81YwidEwnIAAqKjbRCPrFWdBJVz1k2LmOvu8=.497fd5dc-91b2-4294-8049-a5766323539a@github.com>

On Thu, 29 Jul 2021 16:42:35 GMT, Brian Burkhalter <bpb at openjdk.org> wrote:

>> The JavaDocs in `InputStream::transferTo` *cleary* tell the caller that there is **no** guarantee of *any* specific behavior in that particular case: 
>>>The behavior for the case where the input and/or output stream is asynchronously closed, or the thread interrupted during the transfer, is highly input and output stream specific, and therefore not specified.
>
> Good point.

:-)

>> This is a question of style and personal likes. Code maintenance is much easier if variables are defined *near* to its first use, not *far away* (as in the Pascal or C language). If the reviewers want me to change it, I will do it.
>
> It's actually a matter of convention but I think it can remain as it is.

Ok for me, otherwise just clearly tell me and I do change it.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From naoto at openjdk.java.net  Thu Jul 29 22:56:35 2021
From: naoto at openjdk.java.net (Naoto Sato)
Date: Thu, 29 Jul 2021 22:56:35 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v3]
In-Reply-To: <yDLTHJdWNZPTUfj2K0TAdMSGj21csVGcXorLKl2YVRk=.c41cf54d-28be-4414-82ab-36daa4cba473@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
 <yDLTHJdWNZPTUfj2K0TAdMSGj21csVGcXorLKl2YVRk=.c41cf54d-28be-4414-82ab-36daa4cba473@github.com>
Message-ID: <7zHatSac3UvRIlIuqeW4IvcvLIHAkc0P75O_gznJdlA=.e892404c-8063-4174-bb89-a5b295ac7507@github.com>

On Thu, 29 Jul 2021 18:21:07 GMT, Lance Andersen <lancea at openjdk.org> wrote:

>> Hi,
>> 
>> As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS
>> 
>> Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.
>> 
>> 
>> Mach5 tiers 1 through 3 have been run without any errors encountered .
>> 
>> Best,
>> Lance
>
> Lance Andersen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - Add Impl Note to Zip FS module-info
>  - Merge
>  - Add missing Copyright header and address minor comments
>  - Address missing linefeed  after package name
>  - Address overzelous intellij import update
>  - Patch to address JDK-8251329

src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java line 1622:

> 1620:             index++;
> 1621:         }
> 1622:         return dotOrDotDotFound;

Could simply return `false` here, and eliminate the local variable `dotOrDotDotFound`.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4900

From mik3hall at gmail.com  Thu Jul 29 23:00:24 2021
From: mik3hall at gmail.com (Michael Hall)
Date: Thu, 29 Jul 2021 18:00:24 -0500
Subject: New release DefaultFileSystemProviders
In-Reply-To: <BB70AF11-9819-474F-9099-C7A21F3FEA81@gmail.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
 <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
 <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>
 <2da73a16-df41-26c0-2d2a-ab382951a75a@oracle.com>
 <BB70AF11-9819-474F-9099-C7A21F3FEA81@gmail.com>
Message-ID: <73171134-7733-45C3-A258-A0F88152C352@gmail.com>

> 
> I hadn?t thought to change to the prior test before to get a stack crawl. I was getting pretty sure Astrology was involved somehow. If I can get a trace for the non-modular launch I?ll post that too. I?m pretty sure it is different.

Correction non-modular exploded works

java 18 exploded works
/usr/libexec/java_home -v 18 --exec java -cp .:macnio2 -Djava.security.manager -Djava.security.policy=all.policy -Djava.nio.file.spi.DefaultFileSystemProvider=us.hall.trz.osx.MacFileSystemProvider org.test.Test 
WARNING: A command line option has enabled the Security Manager
WARNING: The Security Manager is deprecated and will be removed in a future release
us.hall.trz.osx.MacFileSystem at 60e53b93
us.hall.trz.osx.MacFileSystemProvider

java 17 exploded works
/usr/libexec/java_home -v 17 --exec java -cp .:macnio2 -Djava.security.manager -Djava.security.policy=all.policy -Djava.nio.file.spi.DefaultFileSystemProvider=us.hall.trz.osx.MacFileSystemProvider org.test.Test 
WARNING: A command line option has enabled the Security Manager
WARNING: The Security Manager is deprecated and will be removed in a future release
us.hall.trz.osx.MacFileSystem at 511d50c0
us.hall.trz.osx.MacFileSystemProvider

Non-exploded, or jar?d, is probably still https://bugs.openjdk.java.net/browse/JDK-8263940

From bpb at openjdk.java.net  Thu Jul 29 23:17:46 2021
From: bpb at openjdk.java.net (Brian Burkhalter)
Date: Thu, 29 Jul 2021 23:17:46 GMT
Subject: RFR: 8271308: (fc) FileChannel.transferTo() transfers no more than
 Integer.MAX_VALUE bytes in one call
Message-ID: <AF7i5NoRGaMS7UZkW_YqwLF76WniEELSyq5VLBgY_IQ=.77d66870-82ef-4b12-9caa-10888bfe6e16@github.com>

Please consider this request to enhance `java.nio.channels.FileChannel.transferTo()` to be able to transfer more than `Integer.MAX_VALUE` bytes in one call.

-------------

Commit messages:
 - 8271308: (fc) FileChannel.transferTo() transfers no more than Integer.MAX_VALUE bytes in one call

Changes: https://git.openjdk.java.net/jdk/pull/4940/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4940&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8271308
  Stats: 197 lines in 4 files changed: 178 ins; 10 del; 9 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4940.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4940/head:pull/4940

PR: https://git.openjdk.java.net/jdk/pull/4940

From bpb at openjdk.java.net  Thu Jul 29 23:17:46 2021
From: bpb at openjdk.java.net (Brian Burkhalter)
Date: Thu, 29 Jul 2021 23:17:46 GMT
Subject: RFR: 8271308: (fc) FileChannel.transferTo() transfers no more
 than Integer.MAX_VALUE bytes in one call
In-Reply-To: <AF7i5NoRGaMS7UZkW_YqwLF76WniEELSyq5VLBgY_IQ=.77d66870-82ef-4b12-9caa-10888bfe6e16@github.com>
References: <AF7i5NoRGaMS7UZkW_YqwLF76WniEELSyq5VLBgY_IQ=.77d66870-82ef-4b12-9caa-10888bfe6e16@github.com>
Message-ID: <u878MDM6DqsEam2l8em4iDOoYXvfEhg0E8nIX9Dn04Y=.0d338297-0b13-4c62-849c-94820edd1cc9@github.com>

On Thu, 29 Jul 2021 23:08:58 GMT, Brian Burkhalter <bpb at openjdk.org> wrote:

> Please consider this request to enhance `java.nio.channels.FileChannel.transferTo()` to be able to transfer more than `Integer.MAX_VALUE` bytes in one call.

The approach taken avoids adding loops inside the various private `transferToX()` methods. Without this change the new test fails. No existing tests fail because of the change.

The native method `maxTransferSize0()` might be better replaced by a constant

    private static final long MAX_TRANSFER_SIZE = (long)Integer.MAX_VALUE - 1;

This would guarantee that Windows? `TransmitFile()` would not reject a transfer as being too large (`count == Integer.MAX_VALUE`) thereby forcing a slower path and would little affect the other platforms.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4940

From bpb at openjdk.java.net  Thu Jul 29 23:42:41 2021
From: bpb at openjdk.java.net (Brian Burkhalter)
Date: Thu, 29 Jul 2021 23:42:41 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <AC0DI4y81YwidEwnIAAqKjbRCPrFWdBJVz1k2LmOvu8=.497fd5dc-91b2-4294-8049-a5766323539a@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
 <IZKluRfiLnIovyrAw3TdXDSzB6e59nHmG3ee7PEtjSE=.a778bb70-9938-4599-a3d9-9ad8fa235b00@github.com>
 <D-BolUEcvPGEA-JzbvXEUQTtaxUbl7duhesuVERIZc4=.4edfe8a0-8284-45cf-a2ad-993d795b14b6@github.com>
 <UQzAp1eaWHObRgVpa8bwHLdbdyauIE9NdpDd-MwsvUM=.cd00637c-80bf-43da-8ab3-97825a7d92d0@github.com>
 <AC0DI4y81YwidEwnIAAqKjbRCPrFWdBJVz1k2LmOvu8=.497fd5dc-91b2-4294-8049-a5766323539a@github.com>
Message-ID: <1llS7FL5O9A3depy1oQWVTEBSuRQ1iKbJrhQUcEgREY=.020798ab-c8ce-40a1-804e-1112f51f6a44@github.com>

On Thu, 29 Jul 2021 19:56:40 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>> It's actually a matter of convention but I think it can remain as it is.
>
> Ok for me, otherwise just clearly tell me and I do change it.

I think you can leave it unless someone else thinks otherwise.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From lancea at openjdk.java.net  Fri Jul 30 00:55:57 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Fri, 30 Jul 2021 00:55:57 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v4]
In-Reply-To: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
Message-ID: <_HLwah9MHEE00YhzFYmKEg38WvW5atpQetuzzv6qtbE=.6614b555-0b70-463c-aed1-2a6cc892089a@github.com>

> Hi,
> 
> As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS
> 
> Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.
> 
> 
> Mach5 tiers 1 through 3 have been run without any errors encountered .
> 
> Best,
> Lance

Lance Andersen has updated the pull request incrementally with one additional commit since the last revision:

  Remove uneeded variable

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4900/files
  - new: https://git.openjdk.java.net/jdk/pull/4900/files/22731ac3..67c726af

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4900&range=03
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4900&range=02-03

  Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4900.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4900/head:pull/4900

PR: https://git.openjdk.java.net/jdk/pull/4900

From jpai at openjdk.java.net  Fri Jul 30 01:46:33 2021
From: jpai at openjdk.java.net (Jaikiran Pai)
Date: Fri, 30 Jul 2021 01:46:33 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v3]
In-Reply-To: <yDLTHJdWNZPTUfj2K0TAdMSGj21csVGcXorLKl2YVRk=.c41cf54d-28be-4414-82ab-36daa4cba473@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
 <yDLTHJdWNZPTUfj2K0TAdMSGj21csVGcXorLKl2YVRk=.c41cf54d-28be-4414-82ab-36daa4cba473@github.com>
Message-ID: <ylloIpcL8HiX8Q45Dv-4yi72VT6nyd8UMtdmWr4EuoY=.10f5d387-9743-410c-8df2-dc48a5711e5d@github.com>

On Thu, 29 Jul 2021 18:21:07 GMT, Lance Andersen <lancea at openjdk.org> wrote:

>> Hi,
>> 
>> As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS
>> 
>> Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.
>> 
>> 
>> Mach5 tiers 1 through 3 have been run without any errors encountered .
>> 
>> Best,
>> Lance
>
> Lance Andersen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - Add Impl Note to Zip FS module-info
>  - Merge
>  - Add missing Copyright header and address minor comments
>  - Address missing linefeed  after package name
>  - Address overzelous intellij import update
>  - Patch to address JDK-8251329

src/jdk.zipfs/share/classes/module-info.java line 49:

> 47:  *
> 48:  * @implNote The Zip File System will throw a ZipException when opening an
> 49:  * existing Zip file that contains Zip entries with "." or ".." in its name elements.

Hello Lance, reading this sentence adds a bit of confusion since it uses the word "contains". Had I not known the implemenation details, this sentence would have made me think zip file with name elements of the form `foo.bar` or `hello..`  would also be rejected since these name elements "contain" `.` or `..`

Do you think we should change the word to something like "The Zip File System will throw a ZipException when opening an existing Zip file that has "." or ".." named entries"?

-------------

PR: https://git.openjdk.java.net/jdk/pull/4900

From alanb at openjdk.java.net  Fri Jul 30 11:27:30 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Fri, 30 Jul 2021 11:27:30 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v3]
In-Reply-To: <ylloIpcL8HiX8Q45Dv-4yi72VT6nyd8UMtdmWr4EuoY=.10f5d387-9743-410c-8df2-dc48a5711e5d@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
 <yDLTHJdWNZPTUfj2K0TAdMSGj21csVGcXorLKl2YVRk=.c41cf54d-28be-4414-82ab-36daa4cba473@github.com>
 <ylloIpcL8HiX8Q45Dv-4yi72VT6nyd8UMtdmWr4EuoY=.10f5d387-9743-410c-8df2-dc48a5711e5d@github.com>
Message-ID: <uGv-_9dkbCqhIeoCQhRJ7lEs3l22ZoVckOrHqcvsKac=.50c73a0d-37d6-41fa-8348-5f5c1a22ef12@github.com>

On Fri, 30 Jul 2021 01:43:56 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Lance Andersen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
>> 
>>  - Add Impl Note to Zip FS module-info
>>  - Merge
>>  - Add missing Copyright header and address minor comments
>>  - Address missing linefeed  after package name
>>  - Address overzelous intellij import update
>>  - Patch to address JDK-8251329
>
> src/jdk.zipfs/share/classes/module-info.java line 49:
> 
>> 47:  *
>> 48:  * @implNote The Zip File System will throw a ZipException when opening an
>> 49:  * existing Zip file that contains Zip entries with "." or ".." in its name elements.
> 
> Hello Lance, reading this sentence adds a bit of confusion since it uses the word "contains". Had I not known the implemenation details, this sentence would have made me think zip file with name elements of the form `foo.bar` or `hello..`  would also be rejected since these name elements "contain" `.` or `..`
> 
> Do you think we should change the word to something like "The Zip File System will throw a ZipException when opening an existing Zip file that has "." or ".." named entries"?

I think the `@implNote` tag can be dropped here. For the text then maybe it could be tweaked to something like "The ZIP file system provider does not support opening an existing ZIP that contains entries with ...".

-------------

PR: https://git.openjdk.java.net/jdk/pull/4900

From alanb at openjdk.java.net  Fri Jul 30 12:12:30 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Fri, 30 Jul 2021 12:12:30 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <AC0DI4y81YwidEwnIAAqKjbRCPrFWdBJVz1k2LmOvu8=.497fd5dc-91b2-4294-8049-a5766323539a@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
 <IZKluRfiLnIovyrAw3TdXDSzB6e59nHmG3ee7PEtjSE=.a778bb70-9938-4599-a3d9-9ad8fa235b00@github.com>
 <D-BolUEcvPGEA-JzbvXEUQTtaxUbl7duhesuVERIZc4=.4edfe8a0-8284-45cf-a2ad-993d795b14b6@github.com>
 <UQzAp1eaWHObRgVpa8bwHLdbdyauIE9NdpDd-MwsvUM=.cd00637c-80bf-43da-8ab3-97825a7d92d0@github.com>
 <AC0DI4y81YwidEwnIAAqKjbRCPrFWdBJVz1k2LmOvu8=.497fd5dc-91b2-4294-8049-a5766323539a@github.com>
Message-ID: <RL6f0JWEcP22xlawlCxU3zhNrUhJ6iy_rgEr2ZXA688=.b4abee11-6c2c-4d60-8790-e6b05c45f10f@github.com>

On Thu, 29 Jul 2021 19:57:30 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>> Good point.
>
> :-)

Is this loop correct for the case that the channel gets truncated? In that case transferTo will return 0 as no bytes will be transferred and I'm concerned this code will go into an infinite loop.

Also can you can check that IllegalBlockingModeException is thrown for the case ch is a SelectableChannel configured non-blocking and the destination is a FileChannel?

Just on naming, the existing channel implementations use "dst" for the destination and "wbc" (not "oc") for writable byte channels. Just mentioning it so that the new code can be kept consistent where possible.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From Alan.Bateman at oracle.com  Fri Jul 30 14:36:53 2021
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 30 Jul 2021 15:36:53 +0100
Subject: New release DefaultFileSystemProviders
In-Reply-To: <73171134-7733-45C3-A258-A0F88152C352@gmail.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
 <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
 <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>
 <2da73a16-df41-26c0-2d2a-ab382951a75a@oracle.com>
 <BB70AF11-9819-474F-9099-C7A21F3FEA81@gmail.com>
 <73171134-7733-45C3-A258-A0F88152C352@gmail.com>
Message-ID: <d737d57e-73b0-6aa7-1d99-3d6d707731a1@oracle.com>

On 30/07/2021 00:00, Michael Hall wrote:
> Correction non-modular exploded works
>
> java 18 exploded works
> /usr/libexec/java_home -v 18 --exec java -cp .:macnio2 -Djava.security.manager -Djava.security.policy=all.policy -Djava.nio.file.spi.DefaultFileSystemProvider=us.hall.trz.osx.MacFileSystemProvider org.test.Test
> WARNING: A command line option has enabled the Security Manager
> WARNING: The Security Manager is deprecated and will be removed in a future release
> us.hall.trz.osx.MacFileSystem at 60e53b93
> us.hall.trz.osx.MacFileSystemProvider
>
> java 17 exploded works
> /usr/libexec/java_home -v 17 --exec java -cp .:macnio2 -Djava.security.manager -Djava.security.policy=all.policy -Djava.nio.file.spi.DefaultFileSystemProvider=us.hall.trz.osx.MacFileSystemProvider org.test.Test
> WARNING: A command line option has enabled the Security Manager
> WARNING: The Security Manager is deprecated and will be removed in a future release
> us.hall.trz.osx.MacFileSystem at 511d50c0
> us.hall.trz.osx.MacFileSystemProvider
>
> Non-exploded, or jar?d, is probably still https://bugs.openjdk.java.net/browse/JDK-8263940
Yes, the exploded case should be okay. However, if 
java.nio.file.spi.DefaultFileSystemProvider specifies a class in a JAR 
file that is on the class path or module path then you'll run into this 
issue.? Setting the security manager on the command line will trigger 
this occur at VM startup.
-Alan

From mik3hall at gmail.com  Fri Jul 30 15:32:05 2021
From: mik3hall at gmail.com (Michael Hall)
Date: Fri, 30 Jul 2021 10:32:05 -0500
Subject: New release DefaultFileSystemProviders
In-Reply-To: <d737d57e-73b0-6aa7-1d99-3d6d707731a1@oracle.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
 <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
 <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>
 <2da73a16-df41-26c0-2d2a-ab382951a75a@oracle.com>
 <BB70AF11-9819-474F-9099-C7A21F3FEA81@gmail.com>
 <73171134-7733-45C3-A258-A0F88152C352@gmail.com>
 <d737d57e-73b0-6aa7-1d99-3d6d707731a1@oracle.com>
Message-ID: <E36D0BD4-94D2-424C-8DE5-3152041A0E69@gmail.com>



> On Jul 30, 2021, at 9:36 AM, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> 
> On 30/07/2021 00:00, Michael Hall wrote:
>> Correction non-modular exploded works
>> 
>> java 18 exploded works
>> /usr/libexec/java_home -v 18 --exec java -cp .:macnio2 -Djava.security.manager -Djava.security.policy=all.policy -Djava.nio.file.spi.DefaultFileSystemProvider=us.hall.trz.osx.MacFileSystemProvider org.test.Test
>> WARNING: A command line option has enabled the Security Manager
>> WARNING: The Security Manager is deprecated and will be removed in a future release
>> us.hall.trz.osx.MacFileSystem at 60e53b93
>> us.hall.trz.osx.MacFileSystemProvider
>> 
>> java 17 exploded works
>> /usr/libexec/java_home -v 17 --exec java -cp .:macnio2 -Djava.security.manager -Djava.security.policy=all.policy -Djava.nio.file.spi.DefaultFileSystemProvider=us.hall.trz.osx.MacFileSystemProvider org.test.Test
>> WARNING: A command line option has enabled the Security Manager
>> WARNING: The Security Manager is deprecated and will be removed in a future release
>> us.hall.trz.osx.MacFileSystem at 511d50c0
>> us.hall.trz.osx.MacFileSystemProvider
>> 
>> Non-exploded, or jar?d, is probably still https://bugs.openjdk.java.net/browse/JDK-8263940
> Yes, the exploded case should be okay. However, if java.nio.file.spi.DefaultFileSystemProvider specifies a class in a JAR file that is on the class path or module path then you'll run into this issue.  Setting the security manager on the command line will trigger this occur at VM startup.
> -Alan

Thanks for explaining the conditions. I will hold off on additional testing for now.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210730/ccead2f7/attachment-0001.htm>

From lancea at openjdk.java.net  Fri Jul 30 15:43:52 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Fri, 30 Jul 2021 15:43:52 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v5]
In-Reply-To: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
Message-ID: <O0Y1U6kfFjKDahEZNqTPyZt1NLLYSC9dgo5dd9AzIJU=.c0af9501-bb91-4cca-849b-cf6989110f28@github.com>

> Hi,
> 
> As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS
> 
> Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.
> 
> 
> Mach5 tiers 1 through 3 have been run without any errors encountered .
> 
> Best,
> Lance

Lance Andersen has updated the pull request incrementally with one additional commit since the last revision:

  Update note added to Zip FS module-info

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4900/files
  - new: https://git.openjdk.java.net/jdk/pull/4900/files/67c726af..0bf065ea

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4900&range=04
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4900&range=03-04

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4900.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4900/head:pull/4900

PR: https://git.openjdk.java.net/jdk/pull/4900

From lancea at openjdk.java.net  Fri Jul 30 15:47:31 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Fri, 30 Jul 2021 15:47:31 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v3]
In-Reply-To: <uGv-_9dkbCqhIeoCQhRJ7lEs3l22ZoVckOrHqcvsKac=.50c73a0d-37d6-41fa-8348-5f5c1a22ef12@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
 <yDLTHJdWNZPTUfj2K0TAdMSGj21csVGcXorLKl2YVRk=.c41cf54d-28be-4414-82ab-36daa4cba473@github.com>
 <ylloIpcL8HiX8Q45Dv-4yi72VT6nyd8UMtdmWr4EuoY=.10f5d387-9743-410c-8df2-dc48a5711e5d@github.com>
 <uGv-_9dkbCqhIeoCQhRJ7lEs3l22ZoVckOrHqcvsKac=.50c73a0d-37d6-41fa-8348-5f5c1a22ef12@github.com>
Message-ID: <jVFxzEa0TAqVCyqF_As36AGCUSfAodpTQg0NuOIRQ8c=.e2e100e9-7b32-484a-9550-2053247017d1@github.com>

On Fri, 30 Jul 2021 11:24:37 GMT, Alan Bateman <alanb at openjdk.org> wrote:

>> src/jdk.zipfs/share/classes/module-info.java line 49:
>> 
>>> 47:  *
>>> 48:  * @implNote The Zip File System will throw a ZipException when opening an
>>> 49:  * existing Zip file that contains Zip entries with "." or ".." in its name elements.
>> 
>> Hello Lance, reading this sentence adds a bit of confusion since it uses the word "contains". Had I not known the implemenation details, this sentence would have made me think zip file with name elements of the form `foo.bar` or `hello..`  would also be rejected since these name elements "contain" `.` or `..`
>> 
>> Do you think we should change the word to something like "The Zip File System will throw a ZipException when opening an existing Zip file that has "." or ".." named entries"?
>
> I think the `@implNote` tag can be dropped here. For the text then maybe it could be tweaked to something like "The ZIP file system provider does not support opening an existing ZIP that contains entries with ...".

Updated the note removing `@impleNote` thought about pre-pending **Note:** to make the wording stick out but held off for now. Updated the Release note accordingly

Jaikiran,  I think the use of name elements is fine as we use similar phrasing in `Path::getName()`

-------------

PR: https://git.openjdk.java.net/jdk/pull/4900

From naoto at openjdk.java.net  Fri Jul 30 17:09:29 2021
From: naoto at openjdk.java.net (Naoto Sato)
Date: Fri, 30 Jul 2021 17:09:29 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v5]
In-Reply-To: <O0Y1U6kfFjKDahEZNqTPyZt1NLLYSC9dgo5dd9AzIJU=.c0af9501-bb91-4cca-849b-cf6989110f28@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
 <O0Y1U6kfFjKDahEZNqTPyZt1NLLYSC9dgo5dd9AzIJU=.c0af9501-bb91-4cca-849b-cf6989110f28@github.com>
Message-ID: <V9bOfRpGfDTHX_vRB0eVxsOemTo-GXAtkrEdbQkT8dA=.540f8940-214c-43de-808c-01bfbd9b81dd@github.com>

On Fri, 30 Jul 2021 15:43:52 GMT, Lance Andersen <lancea at openjdk.org> wrote:

>> Hi,
>> 
>> As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS
>> 
>> Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.
>> 
>> 
>> Mach5 tiers 1 through 3 have been run without any errors encountered .
>> 
>> Best,
>> Lance
>
> Lance Andersen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update note added to Zip FS module-info

Looks good to me.

test/jdk/jdk/nio/zipfs/HasDotDotTest.java line 116:

> 114:     @Test(dataProvider = "checkForDotOrDotDotPaths")
> 115:     public void hasDotOrDotDotTest(String path) throws IOException {
> 116:         if(DEBUG) {

Nit: space after `if`

-------------

Marked as reviewed by naoto (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/4900

From lancea at openjdk.java.net  Fri Jul 30 17:22:59 2021
From: lancea at openjdk.java.net (Lance Andersen)
Date: Fri, 30 Jul 2021 17:22:59 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v6]
In-Reply-To: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
Message-ID: <14bj0f-zLqVMsHsMvKV96kj8VyhOHya_6eOHj2DUbnM=.8f96c9d0-d64b-47bc-9f97-0b5b3a037e16@github.com>

> Hi,
> 
> As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS
> 
> Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.
> 
> 
> Mach5 tiers 1 through 3 have been run without any errors encountered .
> 
> Best,
> Lance

Lance Andersen has updated the pull request incrementally with one additional commit since the last revision:

  Add missing space in if statement

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4900/files
  - new: https://git.openjdk.java.net/jdk/pull/4900/files/0bf065ea..292e698f

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4900&range=05
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4900&range=04-05

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4900.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4900/head:pull/4900

PR: https://git.openjdk.java.net/jdk/pull/4900

From mik3hall at gmail.com  Sat Jul 31 00:21:19 2021
From: mik3hall at gmail.com (Michael Hall)
Date: Fri, 30 Jul 2021 19:21:19 -0500
Subject: New release DefaultFileSystemProviders
In-Reply-To: <E36D0BD4-94D2-424C-8DE5-3152041A0E69@gmail.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
 <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
 <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>
 <2da73a16-df41-26c0-2d2a-ab382951a75a@oracle.com>
 <BB70AF11-9819-474F-9099-C7A21F3FEA81@gmail.com>
 <73171134-7733-45C3-A258-A0F88152C352@gmail.com>
 <d737d57e-73b0-6aa7-1d99-3d6d707731a1@oracle.com>
 <E36D0BD4-94D2-424C-8DE5-3152041A0E69@gmail.com>
Message-ID: <22EC882F-8C1F-44C7-ACF2-4DA67DCF33D7@gmail.com>



>>> 
>> Yes, the exploded case should be okay. However, if java.nio.file.spi.DefaultFileSystemProvider specifies a class in a JAR file that is on the class path or module path then you'll run into this issue.  Setting the security manager on the command line will trigger this occur at VM startup.
>> -Alan
> 
> Thanks for explaining the conditions. I will hold off on additional testing for now.
> 
If you could maybe post the bug report number so I can track when I might test again. 

Thanks again.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210730/158ac9d7/attachment-0001.htm>

From brian.burkhalter at oracle.com  Sat Jul 31 00:40:31 2021
From: brian.burkhalter at oracle.com (Brian Burkhalter)
Date: Sat, 31 Jul 2021 00:40:31 +0000
Subject: New release DefaultFileSystemProviders
In-Reply-To: <22EC882F-8C1F-44C7-ACF2-4DA67DCF33D7@gmail.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
 <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
 <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>
 <2da73a16-df41-26c0-2d2a-ab382951a75a@oracle.com>
 <BB70AF11-9819-474F-9099-C7A21F3FEA81@gmail.com>
 <73171134-7733-45C3-A258-A0F88152C352@gmail.com>
 <d737d57e-73b0-6aa7-1d99-3d6d707731a1@oracle.com>
 <E36D0BD4-94D2-424C-8DE5-3152041A0E69@gmail.com>
 <22EC882F-8C1F-44C7-ACF2-4DA67DCF33D7@gmail.com>
Message-ID: <2C103735-02B8-4C0D-AA11-F0A22BD669AF@oracle.com>


On Jul 30, 2021, at 5:21 PM, Michael Hall <mik3hall at gmail.com<mailto:mik3hall at gmail.com>> wrote:

Yes, the exploded case should be okay. However, if java.nio.file.spi.DefaultFileSystemProvider specifies a class in a JAR file that is on the class path or module path then you'll run into this issue.  Setting the security manager on the command line will trigger this occur at VM startup.
-Alan

Thanks for explaining the conditions. I will hold off on additional testing for now.

If you could maybe post the bug report number so I can track when I might test again.

Thanks again.

This is it, I think: https://bugs.openjdk.java.net/browse/JDK-8263940

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210731/041c038e/attachment.htm>

From mik3hall at gmail.com  Sat Jul 31 00:42:22 2021
From: mik3hall at gmail.com (Michael Hall)
Date: Fri, 30 Jul 2021 19:42:22 -0500
Subject: New release DefaultFileSystemProviders
In-Reply-To: <2C103735-02B8-4C0D-AA11-F0A22BD669AF@oracle.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
 <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
 <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>
 <2da73a16-df41-26c0-2d2a-ab382951a75a@oracle.com>
 <BB70AF11-9819-474F-9099-C7A21F3FEA81@gmail.com>
 <73171134-7733-45C3-A258-A0F88152C352@gmail.com>
 <d737d57e-73b0-6aa7-1d99-3d6d707731a1@oracle.com>
 <E36D0BD4-94D2-424C-8DE5-3152041A0E69@gmail.com>
 <22EC882F-8C1F-44C7-ACF2-4DA67DCF33D7@gmail.com>
 <2C103735-02B8-4C0D-AA11-F0A22BD669AF@oracle.com>
Message-ID: <6468C0A8-DC38-4F1D-BFFC-66BC4C768AD2@gmail.com>



> On Jul 30, 2021, at 7:40 PM, Brian Burkhalter <brian.burkhalter at oracle.com> wrote:
> 
> 
>> On Jul 30, 2021, at 5:21 PM, Michael Hall <mik3hall at gmail.com <mailto:mik3hall at gmail.com>> wrote:
>> 
>>>> Yes, the exploded case should be okay. However, if java.nio.file.spi.DefaultFileSystemProvider specifies a class in a JAR file that is on the class path or module path then you'll run into this issue.  Setting the security manager on the command line will trigger this occur at VM startup.
>>>> -Alan
>>> 
>>> Thanks for explaining the conditions. I will hold off on additional testing for now.
>>> 
>> If you could maybe post the bug report number so I can track when I might test again. 
>> 
>> Thanks again.
> 
> This is it, I think: https://bugs.openjdk.java.net/browse/JDK-8263940 <https://bugs.openjdk.java.net/browse/JDK-8263940>
> 
> Brian

Looks correct. Thanks Brian.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210730/4c07a81e/attachment.htm>

From mik3hall at gmail.com  Sat Jul 31 00:47:13 2021
From: mik3hall at gmail.com (Michael Hall)
Date: Fri, 30 Jul 2021 19:47:13 -0500
Subject: New release DefaultFileSystemProviders
In-Reply-To: <6468C0A8-DC38-4F1D-BFFC-66BC4C768AD2@gmail.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
 <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
 <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>
 <2da73a16-df41-26c0-2d2a-ab382951a75a@oracle.com>
 <BB70AF11-9819-474F-9099-C7A21F3FEA81@gmail.com>
 <73171134-7733-45C3-A258-A0F88152C352@gmail.com>
 <d737d57e-73b0-6aa7-1d99-3d6d707731a1@oracle.com>
 <E36D0BD4-94D2-424C-8DE5-3152041A0E69@gmail.com>
 <22EC882F-8C1F-44C7-ACF2-4DA67DCF33D7@gmail.com>
 <2C103735-02B8-4C0D-AA11-F0A22BD669AF@oracle.com>
 <6468C0A8-DC38-4F1D-BFFC-66BC4C768AD2@gmail.com>
Message-ID: <63C00619-F00E-44B8-BB52-BE29E9D674A5@gmail.com>



> On Jul 30, 2021, at 7:42 PM, Michael Hall <mik3hall at gmail.com> wrote:
> 
> 
> 
>> On Jul 30, 2021, at 7:40 PM, Brian Burkhalter <brian.burkhalter at oracle.com <mailto:brian.burkhalter at oracle.com>> wrote:
>> 
>> 
>>> On Jul 30, 2021, at 5:21 PM, Michael Hall <mik3hall at gmail.com <mailto:mik3hall at gmail.com>> wrote:
>>> 
>>>>> Yes, the exploded case should be okay. However, if java.nio.file.spi.DefaultFileSystemProvider specifies a class in a JAR file that is on the class path or module path then you'll run into this issue.  Setting the security manager on the command line will trigger this occur at VM startup.
>>>>> -Alan
>>>> 
>>>> Thanks for explaining the conditions. I will hold off on additional testing for now.
>>>> 
>>> If you could maybe post the bug report number so I can track when I might test again. 
>>> 
>>> Thanks again.
>> 
>> This is it, I think: https://bugs.openjdk.java.net/browse/JDK-8263940 <https://bugs.openjdk.java.net/browse/JDK-8263940>
>> 
>> Brian
> 
> Looks correct. Thanks Brian.
> 
Actually the stack trace looks somewhat different from what I previously posted but I assume duplicates the same issue.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210730/145b2c7b/attachment-0001.htm>

From mik3hall at gmail.com  Sat Jul 31 00:53:03 2021
From: mik3hall at gmail.com (Michael Hall)
Date: Fri, 30 Jul 2021 19:53:03 -0500
Subject: New release DefaultFileSystemProviders
In-Reply-To: <63C00619-F00E-44B8-BB52-BE29E9D674A5@gmail.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
 <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
 <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>
 <2da73a16-df41-26c0-2d2a-ab382951a75a@oracle.com>
 <BB70AF11-9819-474F-9099-C7A21F3FEA81@gmail.com>
 <73171134-7733-45C3-A258-A0F88152C352@gmail.com>
 <d737d57e-73b0-6aa7-1d99-3d6d707731a1@oracle.com>
 <E36D0BD4-94D2-424C-8DE5-3152041A0E69@gmail.com>
 <22EC882F-8C1F-44C7-ACF2-4DA67DCF33D7@gmail.com>
 <2C103735-02B8-4C0D-AA11-F0A22BD669AF@oracle.com>
 <6468C0A8-DC38-4F1D-BFFC-66BC4C768AD2@gmail.com>
 <63C00619-F00E-44B8-BB52-BE29E9D674A5@gmail.com>
Message-ID: <64FFD862-C5F9-47E0-A306-E1CCF42E9E16@gmail.com>



> On Jul 30, 2021, at 7:47 PM, Michael Hall <mik3hall at gmail.com> wrote:
> 
> 
> 
>> On Jul 30, 2021, at 7:42 PM, Michael Hall <mik3hall at gmail.com <mailto:mik3hall at gmail.com>> wrote:
>> 
>> 
>> 
>>> On Jul 30, 2021, at 7:40 PM, Brian Burkhalter <brian.burkhalter at oracle.com <mailto:brian.burkhalter at oracle.com>> wrote:
>>> 
>>> 
>>>> On Jul 30, 2021, at 5:21 PM, Michael Hall <mik3hall at gmail.com <mailto:mik3hall at gmail.com>> wrote:
>>>> 
>>>>>> Yes, the exploded case should be okay. However, if java.nio.file.spi.DefaultFileSystemProvider specifies a class in a JAR file that is on the class path or module path then you'll run into this issue.  Setting the security manager on the command line will trigger this occur at VM startup.
>>>>>> -Alan
>>>>> 
>>>>> Thanks for explaining the conditions. I will hold off on additional testing for now.
>>>>> 
>>>> If you could maybe post the bug report number so I can track when I might test again. 
>>>> 
>>>> Thanks again.
>>> 
>>> This is it, I think: https://bugs.openjdk.java.net/browse/JDK-8263940 <https://bugs.openjdk.java.net/browse/JDK-8263940>
>>> 
>>> Brian
>> 
>> Looks correct. Thanks Brian.
>> 
> Actually the stack trace looks somewhat different from what I previously posted but I assume duplicates the same issue.

Yeah, should of noticed, that was the first bug report with a jar class path that could be worked around by including the provider in an exploded directory. That works at jdk12. There is no current workaround.
Could still use a number so I can include the provider with a current version of the application when it is fixed.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210730/f74cc626/attachment.htm>

From bpb at openjdk.java.net  Sat Jul 31 00:59:19 2021
From: bpb at openjdk.java.net (Brian Burkhalter)
Date: Sat, 31 Jul 2021 00:59:19 GMT
Subject: RFR: 8057113: (fs) Path should have a method to obtain the
 filename extension [v9]
In-Reply-To: <oMckDRkrX8lqRMCmDaMydUG8Vth7o-EmZWW8YsOZC8U=.67db01a3-4849-46bc-b257-b46a300f720a@github.com>
References: <oMckDRkrX8lqRMCmDaMydUG8Vth7o-EmZWW8YsOZC8U=.67db01a3-4849-46bc-b257-b46a300f720a@github.com>
Message-ID: <8vwe-oQ5ilpMKuep4zSHycQisXMIFImfyE4hRVFvCS8=.c0fbf5b2-ccf3-46f4-bce1-6bb5fb3403e3@github.com>

> Please review this proposed change to add a method `java.nio.file.Path.getExtension()`. This was initially discussed in the thread http://mail.openjdk.java.net/pipermail/nio-dev/2018-February/004716.html. This method would return the filename extension of the file name of the `Path`. The extension is defined to be the portion of the file name after the last dot `(?.?)`.
> 
> The definitions of file extension for about fifteen platforms and languages were surveyed to try to find a reasonable compromise for the definition of extension. The most common definition was the last segment of the name including and after the last dot. The second definition omitted the last dot from the extension. Java-related platforms all exclude the last dot. (One divergent definition in the internal Java NIO method `AbstractFileTypeDetector.getExtension(String)` defines the extension as the part after the *first* dot.)
> 
> All examined cases define the extension to be an empty string if it cannot be determined. None of these cases used `null` to represent an indeterminate extension.
> 
> Little in the way of specifying behavior for special cases (consisting mainly of file names with one or more leading dots) was found. Most definitions concern themselves only with the last dot and what comes after it and ignore leading dots altogether. A few definitions ignore a leading dot at the zeroth character. The current proposal ignores a dot at character zero.
> 
> The behavior of the proposed method for some example cases is as:
> 
> 
> .              -> 
> ..             -> 
> .a.b           -> b
> ......         -> 
> .....a         -> a
> ....a.b        -> b
> ..foo          -> foo
> test.rb        -> rb
> a/b/d/test.rb  -> rb
> .a/b/d/test.rb -> rb
> foo.           -> 
> test           -> 
> .profile       -> 
> .profile.sh    -> sh
> ..foo          -> foo
> .....foo       -> foo
> .vimrc         -> 
> test.          -> 
> test..         -> 
> test...        -> 
> foo.tar.gz     -> gz
> foo.bar.       -> 
> image.jpg      -> jpg
> music.mp3      -> mp3
> video.mp4      -> mp4
> document.txt   -> txt
> foo.tar.gz     -> gz
> foo.bar.       ->

Brian Burkhalter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 10 additional commits since the last revision:

 - 8057113: Optimistically update @since tag to 18
 - Merge
 - 8057113: Change first sentence; change param name
 - 8057113: Change Path.getExtension() to accept a default return value in case the extension is indeterminate
 - 8057113: Tweak spec again, and @implSpec code
 - 8057113: Add @since tag
 - 8057113: Tweak first sentence of spec
 - 8057113: Handle getFileName() == null; revise spec
 - 8057113: Changes pursuant to PR conversation
 - 8057113: (fs) Path should have a method to obtain the filename extension

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2319/files
  - new: https://git.openjdk.java.net/jdk/pull/2319/files/e380d09e..fa00aec7

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2319&range=08
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2319&range=07-08

  Stats: 178951 lines in 5300 files changed: 113351 ins; 39684 del; 25916 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2319.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2319/head:pull/2319

PR: https://git.openjdk.java.net/jdk/pull/2319

From mik3hall at gmail.com  Sat Jul 31 01:39:11 2021
From: mik3hall at gmail.com (Michael Hall)
Date: Fri, 30 Jul 2021 20:39:11 -0500
Subject: New release DefaultFileSystemProviders
In-Reply-To: <64FFD862-C5F9-47E0-A306-E1CCF42E9E16@gmail.com>
References: <1F9AE755-5E4D-43E9-920E-36AA398BC0F6@gmail.com>
 <68B9ADCD-B7F7-46CC-94C9-F8BF46A89B35@oracle.com>
 <CDC1106C-CC88-448D-9F10-36B596E949B8@gmail.com>
 <DAA8C4BB-8F24-4AB3-9C6B-8B9632BAE250@gmail.com>
 <2da73a16-df41-26c0-2d2a-ab382951a75a@oracle.com>
 <BB70AF11-9819-474F-9099-C7A21F3FEA81@gmail.com>
 <73171134-7733-45C3-A258-A0F88152C352@gmail.com>
 <d737d57e-73b0-6aa7-1d99-3d6d707731a1@oracle.com>
 <E36D0BD4-94D2-424C-8DE5-3152041A0E69@gmail.com>
 <22EC882F-8C1F-44C7-ACF2-4DA67DCF33D7@gmail.com>
 <2C103735-02B8-4C0D-AA11-F0A22BD669AF@oracle.com>
 <6468C0A8-DC38-4F1D-BFFC-66BC4C768AD2@gmail.com>
 <63C00619-F00E-44B8-BB52-BE29E9D674A5@gmail.com>
 <64FFD862-C5F9-47E0-A306-E1CCF42E9E16@gmail.com>
Message-ID: <20AB8EFB-0017-4827-B328-A8529824E192@gmail.com>



> On Jul 30, 2021, at 7:53 PM, Michael Hall <mik3hall at gmail.com> wrote:
> 
> 
> 
>> On Jul 30, 2021, at 7:47 PM, Michael Hall <mik3hall at gmail.com <mailto:mik3hall at gmail.com>> wrote:
>> 
>> 
> 
> Yeah, should of noticed, that was the first bug report with a jar class path that could be worked around by including the provider in an exploded directory. That works at jdk12. There is no current workaround.
> Could still use a number so I can include the provider with a current version of the application when it is fixed.
> 

Looking at it again maybe the same bug report works? Would the workaround no longer being available if any zip at all is present be worth a comment? 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210730/f5713b3b/attachment-0001.htm>

From alanb at openjdk.java.net  Sat Jul 31 09:40:31 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Sat, 31 Jul 2021 09:40:31 GMT
Subject: RFR: 8251329: (zipfs) Files.walkFileTree walks infinitely if zip
 has dir named "." inside [v6]
In-Reply-To: <14bj0f-zLqVMsHsMvKV96kj8VyhOHya_6eOHj2DUbnM=.8f96c9d0-d64b-47bc-9f97-0b5b3a037e16@github.com>
References: <BnwVE1xrk_yNwA-SSQWdU-s3Q6aWxeoNn2udHvQi9QE=.40ebc385-34fd-4be1-98ea-1514c46c89fb@github.com>
 <14bj0f-zLqVMsHsMvKV96kj8VyhOHya_6eOHj2DUbnM=.8f96c9d0-d64b-47bc-9f97-0b5b3a037e16@github.com>
Message-ID: <eef9RQM0P7hYZSw-_W4wj8F0_rDbM_pDk7arxbrF4hM=.576373e1-e180-4f18-8e6e-f946e280ea7c@github.com>

On Fri, 30 Jul 2021 17:22:59 GMT, Lance Andersen <lancea at openjdk.org> wrote:

>> Hi,
>> 
>> As discussed in the https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079621.html thread, this is the revised patch to address the use of '.' and '..' within Zip FS
>> 
>> Zip FS needs to use "." and ".." as links to the current and parent directories and cannot reliably support entries that have "." and ".." as name elements.  This patch updates Zip Fs  to reject ZIP files that have entries in the CEN that can't be used for files in a file system.
>> 
>> 
>> Mach5 tiers 1 through 3 have been run without any errors encountered .
>> 
>> Best,
>> Lance
>
> Lance Andersen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add missing space in if statement

Marked as reviewed by alanb (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk/pull/4900

From github.com+1701815+mkarg at openjdk.java.net  Sat Jul 31 13:17:08 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Sat, 31 Jul 2021 13:17:08 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v7]
In-Reply-To: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
Message-ID: <sNd5lVF-Z9OoK1ltwyCzLC-5xXSwPTUda-oEpKo5rJQ=.9319c08d-c044-422f-8500-3be4ba2773db@github.com>

> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
> 
> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
> 
> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
> 
> I encourage everybody to discuss this draft:
> * Are there valid arguments for *not* doing this change?
> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
> * How to go on from here: What is missing to get this ready for an actual review?

Markus KARG has updated the pull request incrementally with one additional commit since the last revision:

  Draft: Renamed oc to wbc and dest to dst

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4263/files
  - new: https://git.openjdk.java.net/jdk/pull/4263/files/1f9dba3d..f91d5422

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=06
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=05-06

  Stats: 17 lines in 1 file changed: 0 ins; 0 del; 17 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4263.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4263/head:pull/4263

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Sat Jul 31 13:17:09 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Sat, 31 Jul 2021 13:17:09 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <RL6f0JWEcP22xlawlCxU3zhNrUhJ6iy_rgEr2ZXA688=.b4abee11-6c2c-4d60-8790-e6b05c45f10f@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
 <IZKluRfiLnIovyrAw3TdXDSzB6e59nHmG3ee7PEtjSE=.a778bb70-9938-4599-a3d9-9ad8fa235b00@github.com>
 <D-BolUEcvPGEA-JzbvXEUQTtaxUbl7duhesuVERIZc4=.4edfe8a0-8284-45cf-a2ad-993d795b14b6@github.com>
 <UQzAp1eaWHObRgVpa8bwHLdbdyauIE9NdpDd-MwsvUM=.cd00637c-80bf-43da-8ab3-97825a7d92d0@github.com>
 <AC0DI4y81YwidEwnIAAqKjbRCPrFWdBJVz1k2LmOvu8=.497fd5dc-91b2-4294-8049-a5766323539a@github.com>
 <RL6f0JWEcP22xlawlCxU3zhNrUhJ6iy_rgEr2ZXA688=.b4abee11-6c2c-4d60-8790-e6b05c45f10f@github.com>
Message-ID: <v6VO-i9pTbZ-nFktsPex13dbxWkznsSCUtUbWiSltm8=.807072c3-d39a-47b0-ae2c-bf21a41c264b@github.com>

On Fri, 30 Jul 2021 12:09:30 GMT, Alan Bateman <alanb at openjdk.org> wrote:

> Just on naming, the existing channel implementations use "dst" for the destination and "wbc" (not "oc") for writable byte channels. Just mentioning it so that the new code can be kept consistent where possible.

I have renamed `dest` to `dst` and `oc` to `wbc` in https://github.com/openjdk/jdk/pull/4263/commits/f91d5422c41e25410a81aad54a45b2d0e171305a.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Sat Jul 31 14:14:03 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Sat, 31 Jul 2021 14:14:03 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v8]
In-Reply-To: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
Message-ID: <MdGtXn1mTIdFLuD9JnJgGq8p24Q6duL2_u5rSR0DZyk=.c64f1f51-0a40-4851-ad2f-06ce16585d8b@github.com>

> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
> 
> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
> 
> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
> 
> I encourage everybody to discuss this draft:
> * Are there valid arguments for *not* doing this change?
> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
> * How to go on from here: What is missing to get this ready for an actual review?

Markus KARG has updated the pull request incrementally with two additional commits since the last revision:

 - Draft: Renamed r to bytesRead
 - Draft: Prevent infinite loop after truncation

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4263/files
  - new: https://git.openjdk.java.net/jdk/pull/4263/files/f91d5422..f3e37fe1

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=07
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=06-07

  Stats: 16 lines in 1 file changed: 2 ins; 2 del; 12 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4263.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4263/head:pull/4263

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Sat Jul 31 14:38:04 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Sat, 31 Jul 2021 14:38:04 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v9]
In-Reply-To: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
Message-ID: <acfH4DF_94kY0uCz_smLNKPfx44fEMYQg7aiZc-kvww=.c49913b2-01f2-458d-89f3-92e3c7c36b0e@github.com>

> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
> 
> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
> 
> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
> 
> I encourage everybody to discuss this draft:
> * Are there valid arguments for *not* doing this change?
> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
> * How to go on from here: What is missing to get this ready for an actual review?

Markus KARG has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains two new commits since the last revision:

 - Draft: Renamed r to bytesRead
 - Draft: Prevent infinite loop after truncation

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4263/files
  - new: https://git.openjdk.java.net/jdk/pull/4263/files/f3e37fe1..e0724718

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=08
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=07-08

  Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4263.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4263/head:pull/4263

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Sat Jul 31 14:41:39 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Sat, 31 Jul 2021 14:41:39 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <v6VO-i9pTbZ-nFktsPex13dbxWkznsSCUtUbWiSltm8=.807072c3-d39a-47b0-ae2c-bf21a41c264b@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
 <IZKluRfiLnIovyrAw3TdXDSzB6e59nHmG3ee7PEtjSE=.a778bb70-9938-4599-a3d9-9ad8fa235b00@github.com>
 <D-BolUEcvPGEA-JzbvXEUQTtaxUbl7duhesuVERIZc4=.4edfe8a0-8284-45cf-a2ad-993d795b14b6@github.com>
 <UQzAp1eaWHObRgVpa8bwHLdbdyauIE9NdpDd-MwsvUM=.cd00637c-80bf-43da-8ab3-97825a7d92d0@github.com>
 <AC0DI4y81YwidEwnIAAqKjbRCPrFWdBJVz1k2LmOvu8=.497fd5dc-91b2-4294-8049-a5766323539a@github.com>
 <RL6f0JWEcP22xlawlCxU3zhNrUhJ6iy_rgEr2ZXA688=.b4abee11-6c2c-4d60-8790-e6b05c45f10f@github.com>
 <v6VO-i9pTbZ-nFktsPex13dbxWkznsSCUtUbWiSltm8=.807072c3-d39a-47b0-ae2c-bf21a41c264b@github.com>
Message-ID: <kj9rYshksvPAyIdUjhlR5As1A1TYdfBfLMMDb8P7Yuo=.e32a782b-3cfb-4297-8b48-cac1d4d77de5@github.com>

On Sat, 31 Jul 2021 13:12:54 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>> Is this loop correct for the case that the channel gets truncated? In that case transferTo will return 0 as no bytes will be transferred and I'm concerned this code will go into an infinite loop.
>> 
>> Also can you can check that IllegalBlockingModeException is thrown for the case ch is a SelectableChannel configured non-blocking and the destination is a FileChannel?
>> 
>> Just on naming, the existing channel implementations use "dst" for the destination and "wbc" (not "oc") for writable byte channels. Just mentioning it so that the new code can be kept consistent where possible.
>
>> Just on naming, the existing channel implementations use "dst" for the destination and "wbc" (not "oc") for writable byte channels. Just mentioning it so that the new code can be kept consistent where possible.
> 
> I have renamed `dest` to `dst` and `oc` to `wbc` in https://github.com/openjdk/jdk/pull/4263/commits/f91d5422c41e25410a81aad54a45b2d0e171305a.

> Is this loop correct for the case that the channel gets truncated? In that case transferTo will return 0 as no bytes will be transferred and I'm concerned this code will go into an infinite loop.

Good catch, indeed missed this particular situation!

The modified code found in https://github.com/openjdk/jdk/pull/4263/commits/4b501b205c6f1c48bbc82d7a154aed3fc41b1a20 should be safe from infinite loops, as it checks the actual file length in each iteration (even if this costs us one more `synchronized` per iteration).

This will also improve the situation with concurrent *extension* of the file as outlined in:

>If src is extended at an inconvenient point in time, for example between the return from a call to src.transferTo() that makes bytesWritten < n false and the evaluation of that terminating condition, then it appears that not all the content of src will be transferred to dest. I am not however sure that this violates any contract but it could be a behavioral change from the existing implementation.

As `size()` is called in each iteration, it will pick up the appendend bytes. Still the situation exists that some bytes are *replaced* in the source file -- but that is actually out-of-scope of this PR...! ;-)

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Sat Jul 31 16:02:07 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Sat, 31 Jul 2021 16:02:07 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <kj9rYshksvPAyIdUjhlR5As1A1TYdfBfLMMDb8P7Yuo=.e32a782b-3cfb-4297-8b48-cac1d4d77de5@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
 <IZKluRfiLnIovyrAw3TdXDSzB6e59nHmG3ee7PEtjSE=.a778bb70-9938-4599-a3d9-9ad8fa235b00@github.com>
 <D-BolUEcvPGEA-JzbvXEUQTtaxUbl7duhesuVERIZc4=.4edfe8a0-8284-45cf-a2ad-993d795b14b6@github.com>
 <UQzAp1eaWHObRgVpa8bwHLdbdyauIE9NdpDd-MwsvUM=.cd00637c-80bf-43da-8ab3-97825a7d92d0@github.com>
 <AC0DI4y81YwidEwnIAAqKjbRCPrFWdBJVz1k2LmOvu8=.497fd5dc-91b2-4294-8049-a5766323539a@github.com>
 <RL6f0JWEcP22xlawlCxU3zhNrUhJ6iy_rgEr2ZXA688=.b4abee11-6c2c-4d60-8790-e6b05c45f10f@github.com>
 <v6VO-i9pTbZ-nFktsPex13dbxWkznsSCUtUbWiSltm8=.807072c3-d39a-47b0-ae2c-bf21a41c264b@github.com>
 <kj9rYshksvPAyIdUjhlR5As1A1TYdfBfLMMDb8P7Yuo=.e32a782b-3cfb-4297-8b48-cac1d4d77de5@github.com>
Message-ID: <XNg_rSaiybQpqnEKbsnUDsjpjD_L8ZS8NcR3LM56jO0=.1d60c4f0-ac46-4fc1-8784-4d51bb12c6e6@github.com>

On Sat, 31 Jul 2021 14:39:02 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>>> Just on naming, the existing channel implementations use "dst" for the destination and "wbc" (not "oc") for writable byte channels. Just mentioning it so that the new code can be kept consistent where possible.
>> 
>> I have renamed `dest` to `dst` and `oc` to `wbc` in https://github.com/openjdk/jdk/pull/4263/commits/f91d5422c41e25410a81aad54a45b2d0e171305a.
>
>> Is this loop correct for the case that the channel gets truncated? In that case transferTo will return 0 as no bytes will be transferred and I'm concerned this code will go into an infinite loop.
> 
> Good catch, indeed missed this particular situation!
> 
> The modified code found in https://github.com/openjdk/jdk/pull/4263/commits/4b501b205c6f1c48bbc82d7a154aed3fc41b1a20 should be safe from infinite loops, as it checks the actual file length in each iteration (even if this costs us one more `synchronized` per iteration).
> 
> This will also improve the situation with concurrent *extension* of the file as outlined in:
> 
>>If src is extended at an inconvenient point in time, for example between the return from a call to src.transferTo() that makes bytesWritten < n false and the evaluation of that terminating condition, then it appears that not all the content of src will be transferred to dest. I am not however sure that this violates any contract but it could be a behavioral change from the existing implementation.
> 
> As `size()` is called in each iteration, it will pick up the appendend bytes. Still the situation exists that some bytes are *replaced* in the source file -- but that is actually out-of-scope of this PR...! ;-)

> Also can you can check that IllegalBlockingModeException is thrown for the case ch is a SelectableChannel configured non-blocking and the destination is a FileChannel?

Done in https://github.com/openjdk/jdk/pull/4263/commits/8e2889fd6138d8aa8e308a1cd2aea3546a7c43a7, but honestly I'd kindly like to ask for a brief explanation why this has to be done. What actual bad effect would happen if I do not throw?

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Sat Jul 31 16:02:05 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Sat, 31 Jul 2021 16:02:05 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v10]
In-Reply-To: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
Message-ID: <PhwB86a3itLg2DEMOyTnmiOukpDcloT5_enLmYH1MJY=.d1b0ea1a-9b6e-43ef-b348-05891b63bd4d@github.com>

> This PR-*draft* is **work in progress** and an invitation to discuss a possible solution for issue [JDK-8265891](https://bugs.openjdk.java.net/browse/JDK-8265891). It is *not yet* intended for a final review.
> 
> As proposed in JDK-8265891, this PR provides an implementation for `Channels.newInputStream().transferTo()` which provide superior performance compared to the current implementation. The changes are:
> * Prevents transfers through the JVM heap as much as possibly by offloading to deeper levels via NIO, hence allowing the operating system to optimize the transfer.
> * Using more JRE heap in the fallback case when no NIO is possible (still only KiBs, hence mostl ynegligible even on SBCs) to better perform on modern hardware / fast I/O devides.
> 
> Using JMH I have benchmarked both, the original implementation and this implementation, and (depending on the used hardware and use case) performance change was approx. doubled performance. So this PoC proofs that it makes sense to finalize this work and turn it into an actual OpenJDK contribution. 
> 
> I encourage everybody to discuss this draft:
> * Are there valid arguments for *not* doing this change?
> * Is there a *better* way to improve performance of `Channels.newInputStream().transferTo()`?
> * How to go on from here: What is missing to get this ready for an actual review?

Markus KARG has updated the pull request incrementally with one additional commit since the last revision:

  Draft: Throwing IllegalBlockingModeException if src is SelectableChannel and dst is FileChannel

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4263/files
  - new: https://git.openjdk.java.net/jdk/pull/4263/files/e0724718..8e2889fd

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=09
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4263&range=08-09

  Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4263.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4263/head:pull/4263

PR: https://git.openjdk.java.net/jdk/pull/4263

From alanb at openjdk.java.net  Sat Jul 31 17:36:30 2021
From: alanb at openjdk.java.net (Alan Bateman)
Date: Sat, 31 Jul 2021 17:36:30 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <XNg_rSaiybQpqnEKbsnUDsjpjD_L8ZS8NcR3LM56jO0=.1d60c4f0-ac46-4fc1-8784-4d51bb12c6e6@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
 <IZKluRfiLnIovyrAw3TdXDSzB6e59nHmG3ee7PEtjSE=.a778bb70-9938-4599-a3d9-9ad8fa235b00@github.com>
 <D-BolUEcvPGEA-JzbvXEUQTtaxUbl7duhesuVERIZc4=.4edfe8a0-8284-45cf-a2ad-993d795b14b6@github.com>
 <UQzAp1eaWHObRgVpa8bwHLdbdyauIE9NdpDd-MwsvUM=.cd00637c-80bf-43da-8ab3-97825a7d92d0@github.com>
 <AC0DI4y81YwidEwnIAAqKjbRCPrFWdBJVz1k2LmOvu8=.497fd5dc-91b2-4294-8049-a5766323539a@github.com>
 <RL6f0JWEcP22xlawlCxU3zhNrUhJ6iy_rgEr2ZXA688=.b4abee11-6c2c-4d60-8790-e6b05c45f10f@github.com>
 <v6VO-i9pTbZ-nFktsPex13dbxWkznsSCUtUbWiSltm8=.807072c3-d39a-47b0-ae2c-bf21a41c264b@github.com>
 <kj9rYshksvPAyIdUjhlR5As1A1TYdfBfLMMDb8P7Yuo=.e32a782b-3cfb-4297-8b48-cac1d4d77de5@github.com>
 <XNg_rSaiybQpqnEKbsnUDsjpjD_L8ZS8NcR3LM56jO0=.1d60c4f0-ac46-4fc1-8784-4d51bb12c6e6@github.com>
Message-ID: <FwCL5TrB0b64s9UwjBUdE-cgWX8ImaLL-AagyFR62vY=.ef3b9775-7ca9-4e70-8cce-4adfc78a254a@github.com>

On Sat, 31 Jul 2021 15:58:07 GMT, Markus KARG <github.com+1701815+mkarg at openjdk.org> wrote:

>>> Is this loop correct for the case that the channel gets truncated? In that case transferTo will return 0 as no bytes will be transferred and I'm concerned this code will go into an infinite loop.
>> 
>> Good catch, indeed missed this particular situation!
>> 
>> The modified code found in https://github.com/openjdk/jdk/pull/4263/commits/4b501b205c6f1c48bbc82d7a154aed3fc41b1a20 should be safe from infinite loops, as it checks the actual file length in each iteration (even if this costs us one more `synchronized` per iteration).
>> 
>> This will also improve the situation with concurrent *extension* of the file as outlined in:
>> 
>>>If src is extended at an inconvenient point in time, for example between the return from a call to src.transferTo() that makes bytesWritten < n false and the evaluation of that terminating condition, then it appears that not all the content of src will be transferred to dest. I am not however sure that this violates any contract but it could be a behavioral change from the existing implementation.
>> 
>> As `size()` is called in each iteration, it will pick up the appendend bytes. Still the situation exists that some bytes are *replaced* in the source file -- but that is actually out-of-scope of this PR...! ;-)
>
>> Also can you can check that IllegalBlockingModeException is thrown for the case ch is a SelectableChannel configured non-blocking and the destination is a FileChannel?
> 
> Done in https://github.com/openjdk/jdk/pull/4263/commits/8e2889fd6138d8aa8e308a1cd2aea3546a7c43a7, but honestly I'd kindly like to ask for a brief explanation why this has to be done. What actual bad effect would happen if I do not throw?

> The modified code found in [4b501b2](https://github.com/openjdk/jdk/commit/4b501b205c6f1c48bbc82d7a154aed3fc41b1a20) should be safe from infinite loops, as it checks the actual file length in each iteration (even if this costs us one more `synchronized` per iteration).

I need to look at it closely but I suspect this introduces a potential overflow. Also if output stream is backed by a SocketChannel configured non-blocking then FC::transferTo may return 0 so I assume there is a potential infinite loop there too. I suspect the eventually patch will need have to make use of the blockingLock to prevent the underlying channels from being changed to non-blocking during the transfer.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263

From github.com+1701815+mkarg at openjdk.java.net  Sat Jul 31 21:23:32 2021
From: github.com+1701815+mkarg at openjdk.java.net (Markus KARG)
Date: Sat, 31 Jul 2021 21:23:32 GMT
Subject: RFR: 8265891: (ch) InputStream returned by
 Channels.newInputStream should override transferTo [v6]
In-Reply-To: <FwCL5TrB0b64s9UwjBUdE-cgWX8ImaLL-AagyFR62vY=.ef3b9775-7ca9-4e70-8cce-4adfc78a254a@github.com>
References: <XlU5gCBUEa4qTTz8yhMeElh95FuiPcFYQ_nj_WYvbLE=.6e9fc6a4-f3b1-4faf-91f0-d8b06f875780@github.com>
 <5PpEoJT1JZoo75hz4HAQFkipRfXCr_7MyTE8g4yhhgc=.5eeefcfe-205b-4b02-a59b-bead8b410139@github.com>
 <IZKluRfiLnIovyrAw3TdXDSzB6e59nHmG3ee7PEtjSE=.a778bb70-9938-4599-a3d9-9ad8fa235b00@github.com>
 <D-BolUEcvPGEA-JzbvXEUQTtaxUbl7duhesuVERIZc4=.4edfe8a0-8284-45cf-a2ad-993d795b14b6@github.com>
 <UQzAp1eaWHObRgVpa8bwHLdbdyauIE9NdpDd-MwsvUM=.cd00637c-80bf-43da-8ab3-97825a7d92d0@github.com>
 <AC0DI4y81YwidEwnIAAqKjbRCPrFWdBJVz1k2LmOvu8=.497fd5dc-91b2-4294-8049-a5766323539a@github.com>
 <RL6f0JWEcP22xlawlCxU3zhNrUhJ6iy_rgEr2ZXA688=.b4abee11-6c2c-4d60-8790-e6b05c45f10f@github.com>
 <v6VO-i9pTbZ-nFktsPex13dbxWkznsSCUtUbWiSltm8=.807072c3-d39a-47b0-ae2c-bf21a41c264b@github.com>
 <kj9rYshksvPAyIdUjhlR5As1A1TYdfBfLMMDb8P7Yuo=.e32a782b-3cfb-4297-8b48-cac1d4d77de5@github.com>
 <XNg_rSaiybQpqnEKbsnUDsjpjD_L8ZS8NcR3LM56jO0=.1d60c4f0-ac46-4fc1-8784-4d51bb12c6e6@github.com>
 <FwCL5TrB0b64s9UwjBUdE-cgWX8ImaLL-Aa
 gyFR62vY=.ef3b9775-7ca9-4e70-8cce-4adfc78a254a@github.com>
Message-ID: <oeFYG206jRvtO_n90mejVC57y6NZMMBpkYu8y_ovc9s=.4b30fc16-483e-418a-a088-88a14ba8f31c@github.com>

On Sat, 31 Jul 2021 17:33:50 GMT, Alan Bateman <alanb at openjdk.org> wrote:

> I need to look at it closely but I suspect this introduces a potential overflow. Also if output stream is backed by a SocketChannel configured non-blocking then FC::transferTo may return 0 so I assume there is a potential infinite loop there too. I suspect the eventually patch will need have to make use of the blockingLock to prevent the underlying channels from being changed to non-blocking during the transfer.

I need to confess that my NIO knowledge is not deep enough to follow you closely, so I trust on your advice how to go on from here.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4263