From jai.forums2013 at gmail.com Sat Feb 1 07:38:42 2020 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Sat, 1 Feb 2020 13:08:42 +0530 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start Message-ID: Can I please get a review and a sponsor for a patch for https://bugs.openjdk.java.net/browse/JDK-8211917? The webrev containing the patch is available at https://cr.openjdk.java.net/~jpai/webrev/8211917/1/webrev/ The commit in the patch updates the jdk.nio.zipfs.ZipFileSystem to write out the manifest (if any) as the first entry in the LOC. That then allows the java.util.jar.JarInputStream to find the manifest and return it through JarInputStream.getManifest(). In an initial attempt at this patch, I had tried to just reorder the CEN to add the manifest entry first instead of forcing the manifest entry first in the LOC. But that didn't work and the JarInputStream still couldn't find the manifest. So I used this approach to force the manifest as the first entry in the LOC (and the existing code effectively also ensures that it's also the first entry in the CEN). The patch also includes a test case to reproduce this issue and verify the change. In the test case I intentionally used the verbose version of the FileSystem.newFileSystem() API, instead of the newer simpler ones. This is to allow cleaner/easier backporting of this patch to Java version 8 if and when that happens. Functionally, which API variant of the FileSystem.newFileSystem() API is used shouldn't matter in the context of this test case. However, if someone feels that I should switch to the newer available API, then please do let me know and I'll update the patch. -Jaikiran From lance.andersen at oracle.com Mon Feb 3 23:42:02 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 3 Feb 2020 18:42:02 -0500 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: References: Message-ID: <942F0CD3-0C6F-46A5-A7BF-C2ACDDF9BE75@oracle.com> Hi Jaikiran, Thank you for tackling this feature request. I will finish going through the proposed patch and will sponsor once we have completed the overall review. > On Feb 1, 2020, at 2:38 AM, Jaikiran Pai wrote: > > Can I please get a review and a sponsor for a patch for > https://bugs.openjdk.java.net/browse/JDK-8211917? > > The webrev containing the patch is available at > https://cr.openjdk.java.net/~jpai/webrev/8211917/1/webrev/ > > The commit in the patch updates the jdk.nio.zipfs.ZipFileSystem to write > out the manifest (if any) as the first entry in the LOC. That then > allows the java.util.jar.JarInputStream to find the manifest and return > it through JarInputStream.getManifest(). > > In an initial attempt at this patch, I had tried to just reorder the CEN > to add the manifest entry first instead of forcing the manifest entry > first in the LOC. But that didn't work and the JarInputStream still > couldn't find the manifest. So I used this approach to force the > manifest as the first entry in the LOC (and the existing code > effectively also ensures that it's also the first entry in the CEN). Zip/JarInputStream rely on the LOC which is why just updating the CEN would not get you across the goal line. > > The patch also includes a test case to reproduce this issue and verify > the change. In the test case I intentionally used the verbose version of > the FileSystem.newFileSystem() API, instead of the newer simpler ones. > This is to allow cleaner/easier backporting of this patch to Java > version 8 if and when that happens. Functionally, which API variant of > the FileSystem.newFileSystem() API is used shouldn't matter in the > context of this test case. However, if someone feels that I should > switch to the newer available API, then please do let me know and I'll > update the patch. > > -Jaikiran > > Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From jai.forums2013 at gmail.com Tue Feb 4 03:03:13 2020 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Tue, 4 Feb 2020 08:33:13 +0530 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: <942F0CD3-0C6F-46A5-A7BF-C2ACDDF9BE75@oracle.com> References: <942F0CD3-0C6F-46A5-A7BF-C2ACDDF9BE75@oracle.com> Message-ID: Thank you Lance. -Jaikiran On 04/02/20 5:12 am, Lance Andersen wrote: > Hi Jaikiran, > > Thank you for tackling this feature request. > > > I will finish going through the proposed patch ?and will sponsor once > we have completed the overall review. > > > >> On Feb 1, 2020, at 2:38 AM, Jaikiran Pai > > wrote: >> >> Can I please get a review and a sponsor for a patch for >> https://bugs.openjdk.java.net/browse/JDK-8211917? >> >> The webrev containing the patch is available at >> https://cr.openjdk.java.net/~jpai/webrev/8211917/1/webrev/ >> >> The commit in the patch updates the jdk.nio.zipfs.ZipFileSystem to write >> out the manifest (if any) as the first entry in the LOC. That then >> allows the java.util.jar.JarInputStream to find the manifest and return >> it through JarInputStream.getManifest(). >> >> In an initial attempt at this patch, I had tried to just reorder the CEN >> to add the manifest entry first instead of forcing the manifest entry >> first in the LOC. But that didn't work and the JarInputStream still >> couldn't find the manifest. So I used this approach to force the >> manifest as the first entry in the LOC (and the existing code >> effectively also ensures that it's also the first entry in the CEN). > > Zip/JarInputStream rely on the LOC which is why just updating the CEN > would not get you across the goal line. >> >> The patch also includes a test case to reproduce this issue and verify >> the change. In the test case I intentionally used the verbose version of >> the FileSystem.newFileSystem() API, instead of the newer simpler ones. >> This is to allow cleaner/easier backporting of this patch to Java >> version 8 if and when that happens. Functionally, which API variant of >> the FileSystem.newFileSystem() API is used shouldn't matter in the >> context of this test case. However, if someone feels that I should >> switch to the newer available API, then please do let me know and I'll >> update the patch. >> >> -Jaikiran >> >> > > Best > Lance > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering? > 1 Network Drive? > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From lance.andersen at oracle.com Tue Feb 4 21:43:01 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 4 Feb 2020 16:43:01 -0500 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: References: Message-ID: Hi Jaikiran, Thank you again for tackling this feature request. Overall, I think this looks OK. In ZipFileSystem.java, I would reverse the if statement given a MANiFEST.MF present is going to be the exception vs the norm. Perhaps something similar to: ???????????? if(manifestInode == null || manifestProcessed) { inode = inodeIterator.next(); if(inode == manifestInode) continue; } else { inode = manifestInode; manifestProcessed = true; } ????????????????? A few comments/suggestions on the test: I would prefer that the tests use the newer FileSystems:: newFileSystem methods for the patch to the current openjdk repository Please use Map.of() vs Collections.xxxMap We should test with STORED and DEFLATED specified for compression. I would look at the CopyMoveTest and leverage the Entry class and verify method in this test. This will keep the tests looking somewhat similar Please add a test which copies the Manifest from an OS file to the JAR I would consider adding a test which creates a JAR with a Manifest and then uses Files::copy to create a another JAR from the original JAR I would consider a test which creates the JAR via the jar tool(using the ToolProvider API) and then updates the JAR via ZipFS You may want to consider executing the JAR if you are setting the main class attribute see the LargeEntriesTest as an example Best, Lance > On Feb 1, 2020, at 2:38 AM, Jaikiran Pai wrote: > > Can I please get a review and a sponsor for a patch for > https://bugs.openjdk.java.net/browse/JDK-8211917? > > The webrev containing the patch is available at > https://cr.openjdk.java.net/~jpai/webrev/8211917/1/webrev/ > > The commit in the patch updates the jdk.nio.zipfs.ZipFileSystem to write > out the manifest (if any) as the first entry in the LOC. That then > allows the java.util.jar.JarInputStream to find the manifest and return > it through JarInputStream.getManifest(). > > In an initial attempt at this patch, I had tried to just reorder the CEN > to add the manifest entry first instead of forcing the manifest entry > first in the LOC. But that didn't work and the JarInputStream still > couldn't find the manifest. So I used this approach to force the > manifest as the first entry in the LOC (and the existing code > effectively also ensures that it's also the first entry in the CEN). > > The patch also includes a test case to reproduce this issue and verify > the change. In the test case I intentionally used the verbose version of > the FileSystem.newFileSystem() API, instead of the newer simpler ones. > This is to allow cleaner/easier backporting of this patch to Java > version 8 if and when that happens. Functionally, which API variant of > the FileSystem.newFileSystem() API is used shouldn't matter in the > context of this test case. However, if someone feels that I should > switch to the newer available API, then please do let me know and I'll > update the patch. > > -Jaikiran > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From jai.forums2013 at gmail.com Wed Feb 5 15:55:22 2020 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Wed, 5 Feb 2020 21:25:22 +0530 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: References: Message-ID: Thank you very much for the review Lance. I'll come up with an updated patch containing the suggested changes. -Jaikiran On 05/02/20 3:13 am, Lance Andersen wrote: > Hi Jaikiran, > > Thank you again for tackling this feature request. > > Overall, I think this looks OK. > > In ZipFileSystem.java, I would reverse the if statement given a > MANiFEST.MF present is going to be the exception vs the norm. ?Perhaps > something similar to: > > ???????????? > if(manifestInode == null || manifestProcessed) { > inode = inodeIterator.next(); > if(inode == manifestInode) > continue; > } else { > inode = manifestInode; > manifestProcessed = true; > } > ????????????????? > > A few comments/suggestions on the test: > > * I would prefer ?that the tests use the newer FileSystems:: > newFileSystem methods for the patch to the current openjdk repository > * Please use Map.of() vs Collections.xxxMap > * We should test with STORED and DEFLATED specified for compression. ? > * I would look at the CopyMoveTest and leverage the Entry class and > verify method in this test. ?This will keep the tests looking > somewhat similar > * Please add a test which copies the Manifest from an OS file to the JAR > * I would consider adding a test which creates a JAR with a Manifest > and then uses Files::copy to create a another JAR from the > original JAR > * I would consider a test which creates the JAR via ?the jar > tool(using the ToolProvider API) and then updates the JAR via ZipFS > * You may want to consider executing the JAR if you are setting the > main class attribute see the LargeEntriesTest as an example > > > Best, > Lance > >> On Feb 1, 2020, at 2:38 AM, Jaikiran Pai > > wrote: >> >> Can I please get a review and a sponsor for a patch for >> https://bugs.openjdk.java.net/browse/JDK-8211917? >> >> The webrev containing the patch is available at >> https://cr.openjdk.java.net/~jpai/webrev/8211917/1/webrev/ >> >> The commit in the patch updates the jdk.nio.zipfs.ZipFileSystem to write >> out the manifest (if any) as the first entry in the LOC. That then >> allows the java.util.jar.JarInputStream to find the manifest and return >> it through JarInputStream.getManifest(). >> >> In an initial attempt at this patch, I had tried to just reorder the CEN >> to add the manifest entry first instead of forcing the manifest entry >> first in the LOC. But that didn't work and the JarInputStream still >> couldn't find the manifest. So I used this approach to force the >> manifest as the first entry in the LOC (and the existing code >> effectively also ensures that it's also the first entry in the CEN). >> >> The patch also includes a test case to reproduce this issue and verify >> the change. In the test case I intentionally used the verbose version of >> the FileSystem.newFileSystem() API, instead of the newer simpler ones. >> This is to allow cleaner/easier backporting of this patch to Java >> version 8 if and when that happens. Functionally, which API variant of >> the FileSystem.newFileSystem() API is used shouldn't matter in the >> context of this test case. However, if someone feels that I should >> switch to the newer available API, then please do let me know and I'll >> update the patch. >> >> -Jaikiran >> >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering? > 1 Network Drive? > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From lance.andersen at oracle.com Wed Feb 5 23:34:32 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 5 Feb 2020 18:34:32 -0500 Subject: RFR: JDK-8225507: (zipfs): No space left on device not thrown with ZipFileSystem In-Reply-To: <50746bf1-1d46-a0de-b9b5-2fc1aaaa5c7e@gmail.com> References: <50746bf1-1d46-a0de-b9b5-2fc1aaaa5c7e@gmail.com> Message-ID: Hi Jaikiran, I think overall the patch is correct but it would be good to get some extra eyes on this. > On Jan 31, 2020, at 1:32 AM, Jaikiran Pai wrote: > > Initial discussion about this was done here > https://mail.openjdk.java.net/pipermail/nio-dev/2020-January/007014.html > > I looked into this in a bit more detail. The first place where this > IOException is caught and swallowed involves the following code: > > try { > if (e.type == Entry.COPY) { > // entry copy: the only thing changed is the "name" > // and "nlen" in LOC header, so we update/rewrite the > // LOC in new file and simply copy the rest (data and > // ext) without enflating/deflating from the old zip > // file LOC entry. > if (buf == null) > buf = new byte[8192]; > written += copyLOCEntry(e, true, os, written, buf); > } else { // NEW, FILECH or CEN > e.locoff = written; > written += e.writeLOC(os); // write loc header > written += writeEntry(e, os); > } > elist.add(e); > } catch (IOException x) { > x.printStackTrace(); // skip any in-accurate entry > } > > In this, the copyLOCEntry(...), the writeLOC(...) and the > writeEntry(...) methods can potentially throw the IOException. Looking > at each of these methods, > > - The copyLOCEntry throws an IOException if it fails to write out > data to the OutputStream. It additionally throws a ZipException (which > is an IOException) if it fails to read a LOC. > > - The Entry#writeLOC method takes a OutputStream and writes out LOC > data to that stream and can potentially throw an IOException if the > writes to the stream fail. > > - The writeEntry method writes out the actual data to the > OutputStream and uses the "transferTo" API. Additionally this method > also deletes (any) temp file that might have been created for holding > the "Entry" content. Either of these cases can throw an IOException. > > So in all these above 3 methods, the potential IOException(s) are all > genuine issue that I think should propagate back. The only one that I'm > unsure is the case in writeEntry, where it does a Files.delete(e.file) > where e.file is any temporary file that was created previously to hold > on to the entry data. Should any IOException on this line propagate > back? Perhaps it should, or else it would potentially leave the temp > file (containing the actual data) around without the user knowing that > it got left behind. I think this is something we need to decide. I could argue this both ways. > > The other place where the IOException gets swallowed is this block: > > try { > if (buf == null) > buf = new byte[8192]; > written += copyLOCEntry(e, false, os, written, buf); > elist.add(e); > } catch (IOException x) { > x.printStackTrace(); // skip any wrong entry > } > > In this case the only potential call which can trigger the IOException > is the copyLOCEntry, which is actually the same method that we analyzed > above. > > So overall, I think propagating these IOException(s) should be fine > since they expose any real underlying issue. > > I've created a webrev which includes this change and is placed here[1]. > If the idea to propagate this exception back and the above analysis > seems right, can I get a review and a sponsor for the patch please? > > Given the nature of this issue, I couldn't think of any easy and > consistent test case to add for this change. However, I have ran the > test/jdk/jdk/nio/ jtreg tests and they have allowed passed: I don?t believe there is a way to automate a test for this. Best Lance > > jtreg -jdk:build/macosx-x86_64-server-release/images/jdk test/jdk/jdk/nio/ > > > Test results: passed: 21 > > > [1] https://cr.openjdk.java.net/~jpai/webrev/8225507/1/webrev/ > > -Jaikiran > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From paul.sandoz at oracle.com Thu Feb 6 19:32:07 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 6 Feb 2020 11:32:07 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> Message-ID: <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> I think an absolute bulk put method is useful (see below for caveat). There is some inconsistency with the existing relative bulk specifications and implementations when this and the source buffers overlap. The loopy implementation overwrites but heap buffers defer to System.arraycopy and direct buffers defer to UNSAFE.copyMemory, I believe both make temporary copies on overlapping regions. I am uncertain about when the loopy version is called. ? The new MemorySegment API supports creating segment views from buffers, from which a bulk copy can be performed using memory address within those segments (overlapping ranges create temporary copies). This gives me pause on such enhancements to Buffers. There is some tedious amount of work required to ensure all buffer implementations do the right thing. But, work required by a developer to transition from the buffer domain to segment and address domain is not so trivial either. If we can find a way to simplify the work in buffer I could be persuaded to add this method. Since buffers are not thread safe the absolute bulk implementation could save the positional state use the relative method and restore the positional state? Paul. > On Jan 29, 2020, at 3:49 PM, Brian Burkhalter wrote: > > Please consider this prototype fix [1] for [2]. The other potential method > > $Type$Buffer put(int index, $Type$Buffer src) {} > > mentioned in the issue description is not included here so as to avoid mixing relative bulk (for src) and absolute bulk (for this) operations, i.e., not having either buffer change state. The proposed method therefore simply is the equivalent of replacing the array parameter $type$[] of $Type$Buffer.put(int, $type$[], int, int) with a $Type$Buffer. > > More performant implementations specific to heap and direct buffers are intentionally omitted at this time but will be added later if this concept moves forward. > > One potentially problematic behavior in the proposed method is allowing overlapping source and target ranges when the source and target buffers are the same buffer. > > Thanks, > > Brian > > [1] http://cr.openjdk.java.net/~bpb/8219014/webrev.00/ > [2] https://bugs.openjdk.java.net/browse/JDK-8219014 From brian.burkhalter at oracle.com Thu Feb 6 19:53:57 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 6 Feb 2020 11:53:57 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> Message-ID: <685DAD49-6CCE-4CE7-AE6A-3A3E58B475BE@oracle.com> Hi Paul, > On Feb 6, 2020, at 11:32 AM, Paul Sandoz wrote: > > I think an absolute bulk put method is useful (see below for caveat). > > There is some inconsistency with the existing relative bulk specifications and implementations when this and the source buffers overlap. Yes, in the other case we get an IAE. > The loopy implementation overwrites but heap buffers defer to System.arraycopy and direct buffers defer to UNSAFE.copyMemory, I believe both make temporary copies on overlapping regions. I am uncertain about when the loopy version is called. Right. I have not looked into the specifics of the heap and direct implementations yet, preferring to wait until we decide whether to move forward with this. I don?t know that the loopy version is ever called, actually. It looks like all possible combinations of endianness, [un]swapped, {RW,RO} are generated from the templates. > ? > > The new MemorySegment API supports creating segment views from buffers, from which a bulk copy can be performed using memory address within those segments (overlapping ranges create temporary copies). > > This gives me pause on such enhancements to Buffers. There is some tedious amount of work required to ensure all buffer implementations do the right thing. But, work required by a developer to transition from the buffer domain to segment and address domain is not so trivial either. > > If we can find a way to simplify the work in buffer I could be persuaded to add this method. I think the required work would not be that much given the similarity to other existing code, at least once the specification were agreed. > Since buffers are not thread safe the absolute bulk implementation could save the positional state use the relative method and restore the positional state? Without any sort of locking I wonder if that would not be a bit dangerous. Thanks for reviewing. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Thu Feb 6 20:09:33 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 6 Feb 2020 12:09:33 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <685DAD49-6CCE-4CE7-AE6A-3A3E58B475BE@oracle.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <685DAD49-6CCE-4CE7-AE6A-3A3E58B475BE@oracle.com> Message-ID: > On Feb 6, 2020, at 11:53 AM, Brian Burkhalter wrote: > > Hi Paul, > >> On Feb 6, 2020, at 11:32 AM, Paul Sandoz > wrote: >> >> I think an absolute bulk put method is useful (see below for caveat). >> >> There is some inconsistency with the existing relative bulk specifications and implementations when this and the source buffers overlap. > > Yes, in the other case we get an IAE. > >> The loopy implementation overwrites but heap buffers defer to System.arraycopy and direct buffers defer to UNSAFE.copyMemory, I believe both make temporary copies on overlapping regions. I am uncertain about when the loopy version is called. > > Right. I have not looked into the specifics of the heap and direct implementations yet, preferring to wait until we decide whether to move forward with this. I don?t know that the loopy version is ever called, actually. It looks like all possible combinations of endianness, [un]swapped, {RW,RO} are generated from the templates. > >> ? >> >> The new MemorySegment API supports creating segment views from buffers, from which a bulk copy can be performed using memory address within those segments (overlapping ranges create temporary copies). >> >> This gives me pause on such enhancements to Buffers. There is some tedious amount of work required to ensure all buffer implementations do the right thing. But, work required by a developer to transition from the buffer domain to segment and address domain is not so trivial either. >> >> If we can find a way to simplify the work in buffer I could be persuaded to add this method. > > I think the required work would not be that much given the similarity to other existing code, at least once the specification were agreed. Ok. To me the spec is quite straightforward, it should behave as is the relative bulk method adjusting and reverting the positional state. > >> Since buffers are not thread safe the absolute bulk implementation could save the positional state use the relative method and restore the positional state? > > Without any sort of locking I wonder if that would not be a bit dangerous. > Why? Buffers are anyway stateful and not thread safe. Composing the implementation from public only methods should ensure the integrity of the buffers is maintained if operated on concurrently (since one can anyway code that up externally). Paul. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Feb 6 20:30:37 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 6 Feb 2020 12:30:37 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <685DAD49-6CCE-4CE7-AE6A-3A3E58B475BE@oracle.com> Message-ID: <3BCEE8F5-F1B7-4262-A481-89ABBF5776C0@oracle.com> > On Feb 6, 2020, at 12:09 PM, Paul Sandoz wrote: > > To me the spec is quite straightforward, it should behave as is the relative bulk method adjusting and reverting the positional state. That?s the behavior, except that in its present form the proposed method allows the source and target buffers to be the same. >>> Since buffers are not thread safe the absolute bulk implementation could save the positional state use the relative method and restore the positional state? >> >> Without any sort of locking I wonder if that would not be a bit dangerous. > > Why? Buffers are anyway stateful and not thread safe. Composing the implementation from public only methods should ensure the integrity of the buffers is maintained if operated on concurrently (since one can anyway code that up externally). That?s true. The implementation proposed here follows more from that used in ByteBuffer.put(int,byte[],int,int). Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From jai.forums2013 at gmail.com Fri Feb 7 02:59:46 2020 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Fri, 7 Feb 2020 08:29:46 +0530 Subject: RFR: JDK-8225507: (zipfs): No space left on device not thrown with ZipFileSystem In-Reply-To: References: <50746bf1-1d46-a0de-b9b5-2fc1aaaa5c7e@gmail.com> Message-ID: Thank you Lance for the review. Anyone else have any reviews and thoughts especially around the Files.delete(...) API call noted in the mail? -Jaikiran On 06/02/20 5:04 am, Lance Andersen wrote: > Hi Jaikiran, > > I think overall the patch is correct but it would be good to get some > extra eyes on this. > >> On Jan 31, 2020, at 1:32 AM, Jaikiran Pai > > wrote: >> >> Initial discussion about this was done here >> https://mail.openjdk.java.net/pipermail/nio-dev/2020-January/007014.html >> >> I looked into this in a bit more detail. The first place where this >> IOException is caught and swallowed involves the following code: >> >> try { >> ??? if (e.type == Entry.COPY) { >> ??????? // entry copy: the only thing changed is the "name" >> ??????? // and "nlen" in LOC header, so we update/rewrite the >> ??????? // LOC in new file and simply copy the rest (data and >> ??????? // ext) without enflating/deflating from the old zip >> ??????? // file LOC entry. >> ??????? if (buf == null) >> ??????????? buf = new byte[8192]; >> ??????? written += copyLOCEntry(e, true, os, written, buf); >> ??? } else {????????????????????????? // NEW, FILECH or CEN >> ??????? e.locoff = written; >> ??????? written += e.writeLOC(os);??? // write loc header >> ??????? written += writeEntry(e, os); >> ??? } >> ??? elist.add(e); >> } catch (IOException x) { >> ??? x.printStackTrace();??? // skip any in-accurate entry >> } >> >> In this, the copyLOCEntry(...), the writeLOC(...) and the >> writeEntry(...) methods can potentially throw the IOException. Looking >> at each of these methods, >> >> ??? - The copyLOCEntry throws an IOException if it fails to write out >> data to the OutputStream. It additionally throws a ZipException (which >> is an IOException) if it fails to read a LOC. >> >> ??? - The Entry#writeLOC method takes a OutputStream and writes out LOC >> data to that stream and can potentially throw an IOException if the >> writes to the stream fail. >> >> ??? - The writeEntry method writes out the actual data to the >> OutputStream and uses the "transferTo" API. Additionally this method >> also deletes (any) temp file that might have been created for holding >> the "Entry" content. Either of these cases can throw an IOException. >> >> So in all these above 3 methods, the potential IOException(s) are all >> genuine issue that I think should propagate back. The only one that I'm >> unsure is the case in writeEntry, where it does a Files.delete(e.file) >> where e.file is any temporary file that was created previously to hold >> on to the entry data. Should any IOException on this line propagate >> back? Perhaps it should, or else it would potentially leave the temp >> file (containing the actual data) around without the user knowing that >> it got left behind. > > I think this is something we need to decide. ?I could argue this both > ways. >> >> The other place where the IOException gets swallowed is this block: >> >> try { >> ??? if (buf == null) >> ??????? buf = new byte[8192]; >> ??? written += copyLOCEntry(e, false, os, written, buf); >> ??? elist.add(e); >> } catch (IOException x) { >> ??? x.printStackTrace();??? // skip any wrong entry >> } >> >> In this case the only potential call which can trigger the IOException >> is the copyLOCEntry, which is actually the same method that we analyzed >> above. >> >> So overall, I think propagating these IOException(s) should be fine >> since they expose any real underlying issue. >> >> I've created a webrev which includes this change and is placed here[1]. >> If the idea to propagate this exception back and the above analysis >> seems right, can I get a review and a sponsor for the patch please? >> >> Given the nature of this issue, I couldn't think of any easy and >> consistent test case to add for this change. However, I have ran the >> test/jdk/jdk/nio/ jtreg tests and they have allowed passed: > > I don?t believe there is a way to automate a test for this. ? > > Best > Lance >> >> jtreg -jdk:build/macosx-x86_64-server-release/images/jdk >> test/jdk/jdk/nio/ >> >> >> Test results: passed: 21 >> >> >> [1] https://cr.openjdk.java.net/~jpai/webrev/8225507/1/webrev/ >> >> -Jaikiran >> >> >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering? > 1 Network Drive? > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From paul.sandoz at oracle.com Fri Feb 7 16:07:50 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 7 Feb 2020 08:07:50 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <3BCEE8F5-F1B7-4262-A481-89ABBF5776C0@oracle.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <685DAD49-6CCE-4CE7-AE6A-3A3E58B475BE@oracle.com> <3BCEE8F5-F1B7-4262-A481-89ABBF5776C0@oracle.com> Message-ID: <1D8C50C7-42D3-4D65-A4DF-CF38C9D2983C@oracle.com> > On Feb 6, 2020, at 12:30 PM, Brian Burkhalter wrote: > > >> On Feb 6, 2020, at 12:09 PM, Paul Sandoz > wrote: >> >> To me the spec is quite straightforward, it should behave as is the relative bulk method adjusting and reverting the positional state. > > That?s the behavior, except that in its present form the proposed method allows the source and target buffers to be the same. > Yeah, i was over simplifying. Hmm? implementation-wise perhaps there is another approach. Make the implementation of the relative put defer to the absolute put. Thereby moving the implementations across with some minor adjustments rather than duplicating the code? The relative put would still throw if this == src. (Its possible to tickle overlapping with heap buffers backed by an explicit array and the relative put methods accepting that same array). I also now notice the heap/direct implementations can defer to the loopy version via a super call as a final option. But I still don?t know under what conditions that can occur. Paul. >>>> Since buffers are not thread safe the absolute bulk implementation could save the positional state use the relative method and restore the positional state? >>> >>> Without any sort of locking I wonder if that would not be a bit dangerous. >> >> Why? Buffers are anyway stateful and not thread safe. Composing the implementation from public only methods should ensure the integrity of the buffers is maintained if operated on concurrently (since one can anyway code that up externally). > > That?s true. The implementation proposed here follows more from that used in ByteBuffer.put(int,byte[],int,int). > > Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Feb 7 16:17:59 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 7 Feb 2020 08:17:59 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <1D8C50C7-42D3-4D65-A4DF-CF38C9D2983C@oracle.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <685DAD49-6CCE-4CE7-AE6A-3A3E58B475BE@oracle.com> <3BCEE8F5-F1B7-4262-A481-89ABBF5776C0@oracle.com> <1D8C50C7-42D3-4D65-A4DF-CF38C9D2983C@oracle.com> Message-ID: <89E27BB2-711A-4F11-A537-9D6285FDF244@oracle.com> > On Feb 7, 2020, at 8:07 AM, Paul Sandoz wrote: > > Hmm? implementation-wise perhaps there is another approach. Make the implementation of the relative put defer to the absolute put. Thereby moving the implementations across with some minor adjustments rather than duplicating the code? That?s an interesting idea if it would work. Suppressing code duplication is almost always a good thing. > The relative put would still throw if this == src. (Its possible to tickle overlapping with heap buffers backed by an explicit array and the relative put methods accepting that same array). > > I also now notice the heap/direct implementations can defer to the loopy version via a super call as a final option. But I still don?t know under what conditions that can occur. Right. I don?t know either but I do not think there are any, at least within the JDK. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Feb 7 16:47:18 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 7 Feb 2020 08:47:18 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <89E27BB2-711A-4F11-A537-9D6285FDF244@oracle.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <685DAD49-6CCE-4CE7-AE6A-3A3E58B475BE@oracle.com> <3BCEE8F5-F1B7-4262-A481-89ABBF5776C0@oracle.com> <1D8C50C7-42D3-4D65-A4DF-CF38C9D2983C@oracle.com> <89E27BB2-711A-4F11-A537-9D6285FDF244@oracle.com> Message-ID: <734D7BAA-2AE5-4F89-AAA2-BB9AC682E7D3@oracle.com> > On Feb 7, 2020, at 8:17 AM, Brian Burkhalter wrote: > >> On Feb 7, 2020, at 8:07 AM, Paul Sandoz > wrote: >> >> Hmm? implementation-wise perhaps there is another approach. Make the implementation of the relative put defer to the absolute put. Thereby moving the implementations across with some minor adjustments rather than duplicating the code? > > That?s an interesting idea if it would work. Suppressing code duplication is almost always a good thing. Changing the extant array relative bulk put to invoke the array absolute bulk put as public $Type$Buffer put($type$[] src, int offset, int length) { #if[rw] checkSegment(); - Objects.checkFromIndexSize(offset, length, src.length); int pos = position(); - if (length > limit() - pos) - throw new BufferOverflowException(); - System.arraycopy(src, offset, hb, ix(pos), length); + put(pos, src, offset, length); position(pos + length); return this; #else[rw] throw new ReadOnlyBufferException(); #end[rw] } does not cause any Buffer regression tests to fail. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Fri Feb 7 16:56:25 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 7 Feb 2020 16:56:25 +0000 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> Message-ID: <3e3cdedd-45cc-64d7-b67f-252d11f826bf@oracle.com> Hi Brian, [...] If the source buffer is this buffer and 1139 * the source and target intervals overlap, then the buffer elements in the 1140 * target range will be overwritten by those in the source range of the 1141 * same buffer which could lead to unexpected results. I wonder if the text should be a bit more generic here - as I suspect it's also true if the source buffer is a *view* of the destination buffer - that is - if the backend storage is the same for both buffers? best regards, -- daniel On 29/01/2020 23:49, Brian Burkhalter wrote: > Please consider this prototype fix [1] for [2]. The other potential method > > $Type$Buffer put(int index, $Type$Buffer src) {} > > mentioned in the issue description is not included here so as to avoid mixing relative bulk (for src) and absolute bulk (for this) operations, i.e., not having either buffer change state. The proposed method therefore simply is the equivalent of replacing the array parameter $type$[] of $Type$Buffer.put(int, $type$[], int, int) with a $Type$Buffer. > > More performant implementations specific to heap and direct buffers are intentionally omitted at this time but will be added later if this concept moves forward. > > One potentially problematic behavior in the proposed method is allowing overlapping source and target ranges when the source and target buffers are the same buffer. > > Thanks, > > Brian > > [1] http://cr.openjdk.java.net/~bpb/8219014/webrev.00/ > [2] https://bugs.openjdk.java.net/browse/JDK-8219014 > From brian.burkhalter at oracle.com Fri Feb 7 17:02:08 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 7 Feb 2020 09:02:08 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <3e3cdedd-45cc-64d7-b67f-252d11f826bf@oracle.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <3e3cdedd-45cc-64d7-b67f-252d11f826bf@oracle.com> Message-ID: Hi Daniel, > On Feb 7, 2020, at 8:56 AM, Daniel Fuchs wrote: > > [...] If the source buffer is this buffer and > 1139 * the source and target intervals overlap, then the buffer elements in the > 1140 * target range will be overwritten by those in the source range of the > 1141 * same buffer which could lead to unexpected results. > > I wonder if the text should be a bit more generic here - as I suspect > it's also true if the source buffer is a *view* of the destination > buffer - that is - if the backend storage is the same for both buffers? If it is a view then it should be either a read-only buffer of the same type as the target, or a buffer of a different type in which case it would cause a compilation error. So I don?t know about that this verbiage change would be called for, unless I am missing something. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Fri Feb 7 17:14:51 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 7 Feb 2020 17:14:51 +0000 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <3e3cdedd-45cc-64d7-b67f-252d11f826bf@oracle.com> Message-ID: <6ae3a576-0387-d73c-d44a-3f73e1406690@oracle.com> Hi Brian, On 07/02/2020 17:02, Brian Burkhalter wrote: > If it is a view then it should be either a read-only buffer of the same > type as the target, or a buffer of a different type in which case it > would cause a compilation error. So I don?t know about that this > verbiage change would be called for, unless I am missing something. > Well - I am not sure I understand what said verbiage means then. Assuming: byte[] bytes = ....; var b1 = ByteBuffer.wrap(bytes); var b2 = ByteBuffer.wrap(bytes); var b3 = b1.asReadOnly(); b2.put(1, b1, 0, len-1); b1.put(1, b3, 0, len-1); wouldn't any of these put you in the exact same troublesome situation as: b2.put(1, b2, 0, len-1); cheers, -- daniel From brian.burkhalter at oracle.com Fri Feb 7 17:14:27 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 7 Feb 2020 09:14:27 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <3e3cdedd-45cc-64d7-b67f-252d11f826bf@oracle.com> Message-ID: > On Feb 7, 2020, at 9:02 AM, Brian Burkhalter wrote: > >> On Feb 7, 2020, at 8:56 AM, Daniel Fuchs > wrote: >> >> [...] If the source buffer is this buffer and >> 1139 * the source and target intervals overlap, then the buffer elements in the >> 1140 * target range will be overwritten by those in the source range of the >> 1141 * same buffer which could lead to unexpected results. >> >> I wonder if the text should be a bit more generic here - as I suspect >> it's also true if the source buffer is a *view* of the destination >> buffer - that is - if the backend storage is the same for both buffers? > > If it is a view then it should be either a read-only buffer of the same type as the target, or a buffer of a different type in which case it would cause a compilation error. So I don?t know about that this verbiage change would be called for, unless I am missing something. Well actually maybe ?if the source buffer is this buffer or a read-only view of this buffer? would be better after all. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Feb 7 17:18:50 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 7 Feb 2020 09:18:50 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <6ae3a576-0387-d73c-d44a-3f73e1406690@oracle.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <3e3cdedd-45cc-64d7-b67f-252d11f826bf@oracle.com> <6ae3a576-0387-d73c-d44a-3f73e1406690@oracle.com> Message-ID: <8D4D00D5-33D4-41F6-B047-7C22FF686974@oracle.com> Hi Daniel, > On 07/02/2020 17:02, Brian Burkhalter wrote: >> If it is a view then it should be either a read-only buffer of the same type as the target, or a buffer of a different type in which case it would cause a compilation error. So I don?t know about that this verbiage change would be called for, unless I am missing something. > > Well - I am not sure I understand what said verbiage means then. > Assuming: > > byte[] bytes = ....; > > var b1 = ByteBuffer.wrap(bytes); > var b2 = ByteBuffer.wrap(bytes); > var b3 = b1.asReadOnly(); > > b2.put(1, b1, 0, len-1); > b1.put(1, b3, 0, len-1); > > wouldn't any of these put you in the exact same troublesome > situation as: > > b2.put(1, b2, 0, len-1); I see what you mean. Then something like ?if the source buffer is backed by the same storage as this buffer? might cover it. Thanks, Brian From Alan.Bateman at oracle.com Sun Feb 9 16:14:32 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 9 Feb 2020 16:14:32 +0000 Subject: 8238376: test/jdk/java/nio/channels/DatagramChannel/Loopback.java failing on multi-homed systems Message-ID: <12986ef6-0277-454b-a833-9108fd48795e@oracle.com> test/jdk/java/nio/channels/DatagramChannel/Loopback.java is a test we added recently to check the behavior of the IP_MULTICAST_LOOP socket option. This tests is failing on test systems that are connected to several networks because the test doesn't set the interface for outgoing multicast datagrams. Amy Lu has checked the test passes with this change. -Alan --- a/test/jdk/java/nio/channels/DatagramChannel/Loopback.java +++ b/test/jdk/java/nio/channels/DatagramChannel/Loopback.java @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 2019, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2019, 2020, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -45,7 +45,9 @@ ?import java.nio.channels.Selector; ?import java.util.List; ?import java.util.stream.Collectors; -import static java.net.StandardProtocolFamily.*; +import static java.net.StandardProtocolFamily.INET; +import static java.net.StandardProtocolFamily.INET6; +import static java.net.StandardSocketOptions.IP_MULTICAST_IF; ?import static java.net.StandardSocketOptions.IP_MULTICAST_LOOP; ?import jdk.test.lib.NetworkConfiguration; @@ -106,6 +108,9 @@ ???????????? System.out.format("join %s @ %s%n", group.getHostAddress(), ni.getName()); ???????????? dc.join(group, ni); +??????????? System.out.format("set outgoing multicast interface to %s%n", ni.getName()); +??????????? dc.setOption(IP_MULTICAST_IF, ni); + ???????????? // -- IP_MULTICAST_LOOP enabled -- ???????????? assertTrue(dc.getOption(IP_MULTICAST_LOOP), "IP_MULTICAST_LOOP not enabled"); From Alan.Bateman at oracle.com Sun Feb 9 17:02:23 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 9 Feb 2020 17:02:23 +0000 Subject: RFR: JDK-8225507: (zipfs): No space left on device not thrown with ZipFileSystem In-Reply-To: References: <50746bf1-1d46-a0de-b9b5-2fc1aaaa5c7e@gmail.com> Message-ID: <76d82ba5-8cea-9196-3989-2d8b9fd4e4f8@oracle.com> On 05/02/2020 23:34, Lance Andersen wrote: > Hi Jaikiran, > > I think overall the patch is correct but it would be good to get some > extra eyes on this. On the surface this looks okay but I think it would be useful if you (or Jai) could write a brief summary about the state of the file system and the underlying channel when the sync fails. I think the zip file system state will be closed (!isOpen) so a later attempt to close it will be a no-op. However, the state of the channel is not clear - is it open or closed? -Alan From christoph.langer at sap.com Sun Feb 9 22:17:05 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Sun, 9 Feb 2020 22:17:05 +0000 Subject: [11u] RFR: 8216472: (se) Stack overflow during selection operation leads to crash (win) In-Reply-To: References: Message-ID: Hi Alex, I think this looks good for 11u. Best regards Christoph > -----Original Message----- > From: nio-dev On Behalf Of Alex > Kashchenko > Sent: Freitag, 24. Januar 2020 12:00 > To: jdk-updates-dev at openjdk.java.net > Cc: nio-dev at openjdk.java.net > Subject: [11u] RFR: 8216472: (se) Stack overflow during selection operation > leads to crash (win) > > Hi, > > Please review the backport of JDK-8216472 to 11u: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8216472 > > Original review thread: > https://mail.openjdk.java.net/pipermail/nio-dev/2019-October/006684.html > > Original review thread (continuation): > https://mail.openjdk.java.net/pipermail/nio-dev/2019- > November/006777.html > > Original change: https://hg.openjdk.java.net/jdk/jdk/rev/6bc29ebe053e > > 11u webrev: > http://cr.openjdk.java.net/~akasko/jdk11u/8216472/webrev.00/ > > Please note, that original change, besides moving structs off the stack, > additionally removes the logic for the case, when select() call returns > SOCKET_ERROR: > > - starting here: > https://hg.openjdk.java.net/jdk/jdk/rev/6bc29ebe053e#l2.58 > - explanation (point 1 there): > https://mail.openjdk.java.net/pipermail/nio-dev/2019-October/006708.html > > To stay on the safe side with LTS versions, this 11u backport keeps > select() failure logic intact and allocates the space for 6 fd_set > structs (instead of 3 in original change). It is proposed to have the > change added to 11u in this "fix only" form. And, if necessary, to > remove select() failure logic in a follow-up change. > > Testing: 11u change in this form was included with Red Hat > 11.0.6-windows release and passed usual release testing. > > -- > -Alex From akashche at redhat.com Mon Feb 10 07:59:15 2020 From: akashche at redhat.com (Alex Kashchenko) Date: Mon, 10 Feb 2020 07:59:15 +0000 Subject: [11u] RFR: 8216472: (se) Stack overflow during selection operation leads to crash (win) In-Reply-To: References: Message-ID: <0ecd82f5-3a3b-a46e-c80d-fe89eec95f30@redhat.com> Hi Christoph, On 02/09/2020 10:17 PM, Langer, Christoph wrote: > Hi Alex, > > I think this looks good for 11u. Thanks for the review! I've added 11u fix request to the issue. > > Best regards > Christoph > >> -----Original Message----- >> From: nio-dev On Behalf Of Alex >> Kashchenko >> Sent: Freitag, 24. Januar 2020 12:00 >> To: jdk-updates-dev at openjdk.java.net >> Cc: nio-dev at openjdk.java.net >> Subject: [11u] RFR: 8216472: (se) Stack overflow during selection operation >> leads to crash (win) >> >> Hi, >> >> Please review the backport of JDK-8216472 to 11u: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8216472 >> >> Original review thread: >> https://mail.openjdk.java.net/pipermail/nio-dev/2019-October/006684.html >> >> Original review thread (continuation): >> https://mail.openjdk.java.net/pipermail/nio-dev/2019- >> November/006777.html >> >> Original change: https://hg.openjdk.java.net/jdk/jdk/rev/6bc29ebe053e >> >> 11u webrev: >> http://cr.openjdk.java.net/~akasko/jdk11u/8216472/webrev.00/ >> >> Please note, that original change, besides moving structs off the stack, >> additionally removes the logic for the case, when select() call returns >> SOCKET_ERROR: >> >> - starting here: >> https://hg.openjdk.java.net/jdk/jdk/rev/6bc29ebe053e#l2.58 >> - explanation (point 1 there): >> https://mail.openjdk.java.net/pipermail/nio-dev/2019-October/006708.html >> >> To stay on the safe side with LTS versions, this 11u backport keeps >> select() failure logic intact and allocates the space for 6 fd_set >> structs (instead of 3 in original change). It is proposed to have the >> change added to 11u in this "fix only" form. And, if necessary, to >> remove select() failure logic in a follow-up change. >> >> Testing: 11u change in this form was included with Red Hat >> 11.0.6-windows release and passed usual release testing. >> >> -- >> -Alex > -- -Alex From daniel.fuchs at oracle.com Mon Feb 10 10:50:20 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 10 Feb 2020 10:50:20 +0000 Subject: 8238376: test/jdk/java/nio/channels/DatagramChannel/Loopback.java failing on multi-homed systems In-Reply-To: <12986ef6-0277-454b-a833-9108fd48795e@oracle.com> References: <12986ef6-0277-454b-a833-9108fd48795e@oracle.com> Message-ID: <82e93ac7-e7bb-dc30-714c-30439e653664@oracle.com> Hi Alan, LGTM. Thanks for taking care of this one! best regards, -- daniel On 09/02/2020 16:14, Alan Bateman wrote: > test/jdk/java/nio/channels/DatagramChannel/Loopback.java is a test we > added recently to check the behavior of the IP_MULTICAST_LOOP socket > option. This tests is failing on test systems that are connected to > several networks because the test doesn't set the interface for outgoing > multicast datagrams. Amy Lu has checked the test passes with this change. > > -Alan > > --- a/test/jdk/java/nio/channels/DatagramChannel/Loopback.java > +++ b/test/jdk/java/nio/channels/DatagramChannel/Loopback.java > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 2019, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2019, 2020, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -45,7 +45,9 @@ > ?import java.nio.channels.Selector; > ?import java.util.List; > ?import java.util.stream.Collectors; > -import static java.net.StandardProtocolFamily.*; > +import static java.net.StandardProtocolFamily.INET; > +import static java.net.StandardProtocolFamily.INET6; > +import static java.net.StandardSocketOptions.IP_MULTICAST_IF; > ?import static java.net.StandardSocketOptions.IP_MULTICAST_LOOP; > > ?import jdk.test.lib.NetworkConfiguration; > @@ -106,6 +108,9 @@ > ???????????? System.out.format("join %s @ %s%n", > group.getHostAddress(), ni.getName()); > ???????????? dc.join(group, ni); > > +??????????? System.out.format("set outgoing multicast interface to > %s%n", ni.getName()); > +??????????? dc.setOption(IP_MULTICAST_IF, ni); > + > ???????????? // -- IP_MULTICAST_LOOP enabled -- > > ???????????? assertTrue(dc.getOption(IP_MULTICAST_LOOP), > "IP_MULTICAST_LOOP not enabled"); From brian.burkhalter at oracle.com Mon Feb 10 17:53:40 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 10 Feb 2020 09:53:40 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <1D8C50C7-42D3-4D65-A4DF-CF38C9D2983C@oracle.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <685DAD49-6CCE-4CE7-AE6A-3A3E58B475BE@oracle.com> <3BCEE8F5-F1B7-4262-A481-89ABBF5776C0@oracle.com> <1D8C50C7-42D3-4D65-A4DF-CF38C9D2983C@oracle.com> Message-ID: <260883EA-F597-4BA1-8822-6C732FE4E162@oracle.com> > On Feb 7, 2020, at 8:07 AM, Paul Sandoz wrote: > >> On Feb 6, 2020, at 12:30 PM, Brian Burkhalter > wrote: >> >> >>> On Feb 6, 2020, at 12:09 PM, Paul Sandoz > wrote: >>> >>> To me the spec is quite straightforward, it should behave as is the relative bulk method adjusting and reverting the positional state. >> >> That?s the behavior, except that in its present form the proposed method allows the source and target buffers to be the same. >> > > Yeah, i was over simplifying. On second thought it might be better to disallow the source buffer being the same as the target buffer as in the relative bulk version. That would retain consistency while avoiding potential problems. Also, I agree that removing some code duplication by re-implementing the relative bulk methods to call the absolute bulk methods would be a good idea, but this might better be done as a separate, follow-on issue to this one. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From fweimer at redhat.com Tue Feb 11 16:03:23 2020 From: fweimer at redhat.com (Florian Weimer) Date: Tue, 11 Feb 2020 17:03:23 +0100 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> (Paul Sandoz's message of "Thu, 6 Feb 2020 11:32:07 -0800") References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> Message-ID: <87wo8ti01g.fsf@oldenburg2.str.redhat.com> * Paul Sandoz: > There is some inconsistency with the existing relative bulk > specifications and implementations when this and the source buffers > overlap. The loopy implementation overwrites but heap buffers defer to > System.arraycopy and direct buffers defer to UNSAFE.copyMemory, I > believe both make temporary copies on overlapping regions. I am > uncertain about when the loopy version is called. But in general, it is impossible to detect overlaps because two buffers could refer to the same backing storage mapped at different addresses. It's not possible to detect that by comparing the object attachments or the addresses: they would all be different. Thanks, Florian From jai.forums2013 at gmail.com Tue Feb 11 16:33:49 2020 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Tue, 11 Feb 2020 22:03:49 +0530 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: References: Message-ID: Hello Lance, On 05/02/20 3:13 am, Lance Andersen wrote: > Hi Jaikiran, > > Thank you again for tackling this feature request. > > Overall, I think this looks OK. > > In ZipFileSystem.java, I would reverse the if statement given a > MANiFEST.MF present is going to be the exception vs the norm. ?Perhaps > something similar to: > > ???????????? > if(manifestInode == null || manifestProcessed) { > inode = inodeIterator.next(); > if(inode == manifestInode) > continue; > } else { > inode = manifestInode; > manifestProcessed = true; > } > ????????????????? > Done. > A few comments/suggestions on the test: > > * I would prefer ?that the tests use the newer FileSystems:: > newFileSystem methods for the patch to the current openjdk repository > Done - updated the testcase to use the newer available APIs. > * Please use Map.of() vs Collections.xxxMap > Done. > * We should test with STORED and DEFLATED specified for compression. ? > * I would look at the CopyMoveTest and leverage the Entry class and > verify method in this test. ?This will keep the tests looking > somewhat similar > Done - the testcase now tests the default (== DEFLATED), STORED and (explicit) DEFLATED compression methods. > * Please add a test which copies the Manifest from an OS file to the JAR > Done. New testManifestCopiedFromOSFile test method added. > * I would consider adding a test which creates a JAR with a Manifest > and then uses Files::copy to create a another JAR from the > original JAR > Done. New testDuplicateJar test method added. > * I would consider a test which creates the JAR via ?the jar > tool(using the ToolProvider API) and then updates the JAR via ZipFS > Done. New testJarToolGeneratedJarWithManifest added. > * You may want to consider executing the JAR if you are setting the > main class attribute see the LargeEntriesTest as an example > In my initial version, I included the Main-Class manifest attribute only to make sure the manifest file that is being verified is indeed the one we had added in the tests. I did not have any intention of testing the "Main-Class" semantics. In this newer updated version of the test case, I decided to just remove the "Main-Class" and instead use a dummy manifest attribute that I just check for equality. I decided to remove the "Main-Class" attribute since I didn't want this test to do too many things. Let me know if you prefer that the Main-Class to stay and be verified that it can be launched. All these changes are now part of the new webrev which is at https://cr.openjdk.java.net/~jpai/webrev/8211917/2/webrev/ -Jaikiran -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Tue Feb 11 17:04:53 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 11 Feb 2020 09:04:53 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <87wo8ti01g.fsf@oldenburg2.str.redhat.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <87wo8ti01g.fsf@oldenburg2.str.redhat.com> Message-ID: <824238F5-D399-4088-A1AA-A9AC8321577F@oracle.com> Are referring to some property of virtual memory or how buffers are implemented? Unsafe.copyMemory detect overlaps, given two addresses (double-register base + offset) and length. Buffers were enhanced to ensure, over all kinds of buffer, consistent access using double-register addressing. We could collapse all bulk copy operations to leverage Unsafe.copyMemory. Paul. > On Feb 11, 2020, at 8:03 AM, Florian Weimer wrote: > > * Paul Sandoz: > >> There is some inconsistency with the existing relative bulk >> specifications and implementations when this and the source buffers >> overlap. The loopy implementation overwrites but heap buffers defer to >> System.arraycopy and direct buffers defer to UNSAFE.copyMemory, I >> believe both make temporary copies on overlapping regions. I am >> uncertain about when the loopy version is called. > > But in general, it is impossible to detect overlaps because two buffers > could refer to the same backing storage mapped at different addresses. > It's not possible to detect that by comparing the object attachments or > the addresses: they would all be different. > > Thanks, > Florian > From fweimer at redhat.com Tue Feb 11 17:10:41 2020 From: fweimer at redhat.com (Florian Weimer) Date: Tue, 11 Feb 2020 18:10:41 +0100 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <824238F5-D399-4088-A1AA-A9AC8321577F@oracle.com> (Paul Sandoz's message of "Tue, 11 Feb 2020 09:04:53 -0800") References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <87wo8ti01g.fsf@oldenburg2.str.redhat.com> <824238F5-D399-4088-A1AA-A9AC8321577F@oracle.com> Message-ID: <87eev1hwxa.fsf@oldenburg2.str.redhat.com> * Paul Sandoz: > Are referring to some property of virtual memory or how buffers are implemented? > > Unsafe.copyMemory detect overlaps, given two addresses > (double-register base + offset) and length. > > Buffers were enhanced to ensure, over all kinds of buffer, consistent > access using double-register addressing. We could collapse all bulk > copy operations to leverage Unsafe.copyMemory. How does this work if you map the same file twice, or if JNI code creates direct buffers using NewDirectByteBuffer for memory that is aliased at multiple addresses? Thanks, Florian From paul.sandoz at oracle.com Tue Feb 11 17:15:23 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 11 Feb 2020 09:15:23 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <87eev1hwxa.fsf@oldenburg2.str.redhat.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <87wo8ti01g.fsf@oldenburg2.str.redhat.com> <824238F5-D399-4088-A1AA-A9AC8321577F@oracle.com> <87eev1hwxa.fsf@oldenburg2.str.redhat.com> Message-ID: Thanks for the clarification. I see what you are getting at now. Yes, under those aliased circumstances there is not much we can do. All we can talk about is overlapping regions in virtual memory. Paul. > On Feb 11, 2020, at 9:10 AM, Florian Weimer wrote: > > * Paul Sandoz: > >> Are referring to some property of virtual memory or how buffers are implemented? >> >> Unsafe.copyMemory detect overlaps, given two addresses >> (double-register base + offset) and length. >> >> Buffers were enhanced to ensure, over all kinds of buffer, consistent >> access using double-register addressing. We could collapse all bulk >> copy operations to leverage Unsafe.copyMemory. > > How does this work if you map the same file twice, or if JNI code > creates direct buffers using NewDirectByteBuffer for memory that is > aliased at multiple addresses? > > Thanks, > Florian > From paul.sandoz at oracle.com Tue Feb 11 17:17:16 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 11 Feb 2020 09:17:16 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <260883EA-F597-4BA1-8822-6C732FE4E162@oracle.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <685DAD49-6CCE-4CE7-AE6A-3A3E58B475BE@oracle.com> <3BCEE8F5-F1B7-4262-A481-89ABBF5776C0@oracle.com> <1D8C50C7-42D3-4D65-A4DF-CF38C9D2983C@oracle.com> <260883EA-F597-4BA1-8822-6C732FE4E162@oracle.com> Message-ID: <4A04A731-797A-4678-A2E5-EA1598F6D172@oracle.com> > On Feb 10, 2020, at 9:53 AM, Brian Burkhalter wrote: > > >> On Feb 7, 2020, at 8:07 AM, Paul Sandoz > wrote: >> >>> On Feb 6, 2020, at 12:30 PM, Brian Burkhalter > wrote: >>> >>> >>>> On Feb 6, 2020, at 12:09 PM, Paul Sandoz > wrote: >>>> >>>> To me the spec is quite straightforward, it should behave as is the relative bulk method adjusting and reverting the positional state. >>> >>> That?s the behavior, except that in its present form the proposed method allows the source and target buffers to be the same. >>> >> >> Yeah, i was over simplifying. > > On second thought it might be better to disallow the source buffer being the same as the target buffer as in the relative bulk version. That would retain consistency while avoiding potential problems. > That feels like taking the easy way out :-) I believe the constraint is there because buffers hold positional state. Two distinct buffer instances can refer to overlapping regions of virtual memory, In the case of an absolute bulk put there should be no observable update to positional state of either buffer. > Also, I agree that removing some code duplication by re-implementing the relative bulk methods to call the absolute bulk methods would be a good idea, but this might better be done as a separate, follow-on issue to this one. > Ok. Paul. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fweimer at redhat.com Tue Feb 11 17:34:58 2020 From: fweimer at redhat.com (Florian Weimer) Date: Tue, 11 Feb 2020 18:34:58 +0100 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: (Paul Sandoz's message of "Tue, 11 Feb 2020 09:15:23 -0800") References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <87wo8ti01g.fsf@oldenburg2.str.redhat.com> <824238F5-D399-4088-A1AA-A9AC8321577F@oracle.com> <87eev1hwxa.fsf@oldenburg2.str.redhat.com> Message-ID: <8736bhhvst.fsf@oldenburg2.str.redhat.com> * Paul Sandoz: > Thanks for the clarification. I see what you are getting at now. > > Yes, under those aliased circumstances there is not much we can do. > All we can talk about is overlapping regions in virtual memory. Is there a way to capture this in the specification? Thanks, Florian From simone.bordet at gmail.com Tue Feb 11 21:36:38 2020 From: simone.bordet at gmail.com (Simone Bordet) Date: Tue, 11 Feb 2020 22:36:38 +0100 Subject: Cancelled key may fail select() Message-ID: Hi, looking at SelectorImpl.processDeregisterQueue(), there is an iteration that, within the loop, may call SelChImpl.kill(), which may throw an IOException. If the exception is thrown, the iteration over the cancelled keys is aborted and the exception unwinds up to a select() calls, which basically means that the selector is now unusable as the only thing that applications can do when select() throws is to close the selector. Closing the selector causes all other (perfectly working) keys to be either closed or left lingering about. This has been reported for example here: https://github.com/battleblow/openjdk-jdk11u/issues/94 https://github.com/eclipse/jetty.project/issues/4418 Should not the call to kill() be wrapped in a try/catch, ignoring the exception, and the iteration be continued? I am an OpenJDK author and I can file a bug about this, if the issue is confirmed. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From Alan.Bateman at oracle.com Tue Feb 11 22:11:37 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Feb 2020 22:11:37 +0000 Subject: Cancelled key may fail select() In-Reply-To: References: Message-ID: On 11/02/2020 21:36, Simone Bordet wrote: > Hi, > > looking at SelectorImpl.processDeregisterQueue(), there is an > iteration that, within the loop, may call SelChImpl.kill(), which may > throw an IOException. > > If the exception is thrown, the iteration over the cancelled keys is > aborted and the exception unwinds up to a select() calls, which > basically means that the selector is now unusable as the only thing > that applications can do when select() throws is to close the > selector. > > Closing the selector causes all other (perfectly working) keys to be > either closed or left lingering about. > > This has been reported for example here: > https://github.com/battleblow/openjdk-jdk11u/issues/94 > https://github.com/eclipse/jetty.project/issues/4418 > > Should not the call to kill() be wrapped in a try/catch, ignoring the > exception, and the iteration be continued? > Do you have a reproducer? I assume this doesn't duplicate with JDK 13 or newer, the reason is that the SocketChannelImpl::kill implementation doesn't throw. There has been significant churn in this area and it might be that a careful subset of JDK-8223353 may need to be back-ported to the 11 updates. -Alan From simone.bordet at gmail.com Tue Feb 11 22:30:21 2020 From: simone.bordet at gmail.com (Simone Bordet) Date: Tue, 11 Feb 2020 23:30:21 +0100 Subject: Cancelled key may fail select() In-Reply-To: References: Message-ID: Hi, On Tue, Feb 11, 2020 at 11:11 PM Alan Bateman wrote: > Do you have a reproducer? No. It seems to happen more frequently on the FreeBSD port of OpenJDK. > I assume this doesn't duplicate with JDK 13 or > newer, the reason is that the SocketChannelImpl::kill implementation > doesn't throw. I see that SelChImpl.kill() still declares throws IOException. When you say "does not throw" you mean that the native implementation for all OSes does not throw even if the signature declares it? Would be it possible for other implementations (e.g. FreeBSD) to throw and still comply with the signature, but be handled badly by common OpenJDK classes? Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From jai.forums2013 at gmail.com Wed Feb 12 06:39:56 2020 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Wed, 12 Feb 2020 12:09:56 +0530 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: References: Message-ID: I realized that the verify() method in the testcase can include a couple of more tests while dealing with the JarInputStream. So I've updated that method and created a fresh webrev which is available at https://cr.openjdk.java.net/~jpai/webrev/8211917/3/webrev/ -Jaikiran On 11/02/20 10:03 pm, Jaikiran Pai wrote: > > Hello Lance, > > On 05/02/20 3:13 am, Lance Andersen wrote: >> Hi Jaikiran, >> >> Thank you again for tackling this feature request. >> >> Overall, I think this looks OK. >> >> In ZipFileSystem.java, I would reverse the if statement given a >> MANiFEST.MF present is going to be the exception vs the norm. >> ?Perhaps something similar to: >> >> ???????????? >> if(manifestInode == null || manifestProcessed) { >> inode = inodeIterator.next(); >> if(inode == manifestInode) >> continue; >> } else { >> inode = manifestInode; >> manifestProcessed = true; >> } >> ????????????????? >> > Done. > > >> A few comments/suggestions on the test: >> >> * I would prefer ?that the tests use the newer FileSystems:: >> newFileSystem methods for the patch to the current openjdk repository >> > Done - updated the testcase to use the newer available APIs. > > >> * Please use Map.of() vs Collections.xxxMap >> > Done. > > >> * We should test with STORED and DEFLATED specified for compression. ? >> * I would look at the CopyMoveTest and leverage the Entry class and >> verify method in this test. ?This will keep the tests looking >> somewhat similar >> > Done - the testcase now tests the default (== DEFLATED), STORED and > (explicit) DEFLATED compression methods. > > >> * Please add a test which copies the Manifest from an OS file to >> the JAR >> > Done. New testManifestCopiedFromOSFile test method added. > > >> * I would consider adding a test which creates a JAR with a >> Manifest and then uses Files::copy to create a another JAR from >> the original JAR >> > Done. New testDuplicateJar test method added. > > >> * I would consider a test which creates the JAR via ?the jar >> tool(using the ToolProvider API) and then updates the JAR via ZipFS >> > Done. New testJarToolGeneratedJarWithManifest added. > > >> * You may want to consider executing the JAR if you are setting the >> main class attribute see the LargeEntriesTest as an example >> > In my initial version, I included the Main-Class manifest attribute > only to make sure the manifest file that is being verified is indeed > the one we had added in the tests. I did not have any intention of > testing the "Main-Class" semantics. In this newer updated version of > the test case, I decided to just remove the "Main-Class" and instead > use a dummy manifest attribute that I just check for equality. I > decided to remove the "Main-Class" attribute since I didn't want this > test to do too many things. Let me know if you prefer that the > Main-Class to stay and be verified that it can be launched. > > All these changes are now part of the new webrev which is at > https://cr.openjdk.java.net/~jpai/webrev/8211917/2/webrev/ > > -Jaikiran > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Feb 12 08:37:53 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 12 Feb 2020 08:37:53 +0000 Subject: Cancelled key may fail select() In-Reply-To: References: Message-ID: <05f51d3d-b182-a32c-c105-bef9ced733b0@oracle.com> On 11/02/2020 22:30, Simone Bordet wrote: > Hi, > > On Tue, Feb 11, 2020 at 11:11 PM Alan Bateman wrote: >> Do you have a reproducer? > No. It seems to happen more frequently on the FreeBSD port of OpenJDK. The FreeBSD man page for close lists ECONNRESET as a possible error on that platform. I don't see it in the man pages on other platforms. The OpenJDK BSD Port Project seems to be inactive so I can't quite tell where the port(s) are maintained now, do you know if this is just the macOS bits or are there are other patches applied? > : > I see that SelChImpl.kill() still declares throws IOException. > > When you say "does not throw" you mean that the native implementation > for all OSes does not throw even if the signature declares it? > Would be it possible for other implementations (e.g. FreeBSD) to throw > and still comply with the signature, but be handled badly by common > OpenJDK classes? > The kill implementations of the channels in sun.nio.ch do not throw IOException (as of JDK 13). The method still declares that it throws IOException because the SCTP channel implementations (jdk.sctp module) dates from JDK 7 and haven't been modernized/aligned with the new implementations in sun.nio.ch. In any case, I think the issue you are seeing reported on FreeBSD is because SocketChannelImpl::kill shouldn't be propagating from the exception/error from close. It was fixed in JDK 13 by way of other changes. The reason you don't see this in JDK 8u is because it has the old (and problematic) async close mechanism would dup2 the socket while registered with kqueue. Selection operations would do the final cleanup/close (so couldn't fail with ECONNRESET). As I mentioned, there has been a lot of changes to his area in recent releases and fixing the issue for BSD would require back-porting a subset of the changes from JDK-8223353. -Alan. From simone.bordet at gmail.com Wed Feb 12 09:38:39 2020 From: simone.bordet at gmail.com (Simone Bordet) Date: Wed, 12 Feb 2020 10:38:39 +0100 Subject: Cancelled key may fail select() In-Reply-To: <05f51d3d-b182-a32c-c105-bef9ced733b0@oracle.com> References: <05f51d3d-b182-a32c-c105-bef9ced733b0@oracle.com> Message-ID: Alan, On Wed, Feb 12, 2020 at 9:38 AM Alan Bateman wrote: > > On 11/02/2020 22:30, Simone Bordet wrote: > > Hi, > > > > On Tue, Feb 11, 2020 at 11:11 PM Alan Bateman wrote: > >> Do you have a reproducer? > > No. It seems to happen more frequently on the FreeBSD port of OpenJDK. > The FreeBSD man page for close lists ECONNRESET as a possible error on > that platform. I don't see it in the man pages on other platforms. The > OpenJDK BSD Port Project seems to be inactive so I can't quite tell > where the port(s) are maintained now, do you know if this is just the > macOS bits or are there are other patches applied? No, I don't know. This conversation is linked to the issues I have reported, and hopefully a FreeBSD maintainer will read your comments. > In any case, I think the issue you are seeing reported on FreeBSD is > because SocketChannelImpl::kill shouldn't be propagating from the > exception/error from close. It was fixed in JDK 13 by way of other > changes. The reason you don't see this in JDK 8u is because it has the > old (and problematic) async close mechanism would dup2 the socket while > registered with kqueue. Selection operations would do the final > cleanup/close (so couldn't fail with ECONNRESET). As I mentioned, there > has been a lot of changes to his area in recent releases and fixing the > issue for BSD would require back-porting a subset of the changes from > JDK-8223353. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From jai.forums2013 at gmail.com Wed Feb 12 14:48:29 2020 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Wed, 12 Feb 2020 20:18:29 +0530 Subject: RFR: JDK-8225507: (zipfs): No space left on device not thrown with ZipFileSystem In-Reply-To: <76d82ba5-8cea-9196-3989-2d8b9fd4e4f8@oracle.com> References: <50746bf1-1d46-a0de-b9b5-2fc1aaaa5c7e@gmail.com> <76d82ba5-8cea-9196-3989-2d8b9fd4e4f8@oracle.com> Message-ID: Hello Alan, On 09/02/20 10:32 pm, Alan Bateman wrote: > On 05/02/2020 23:34, Lance Andersen wrote: >> Hi Jaikiran, >> >> I think overall the patch is correct but it would be good to get some >> extra eyes on this. > On the surface this looks okay but I think it would be useful if you > (or Jai) could write a brief summary about the state of the file > system and the underlying channel when the sync fails. I think the zip > file system state will be closed (!isOpen) so a later attempt to close > it will be a no-op. However, the state of the channel is not clear - > is it open or closed? I hadn't paid attention to this aspect previously. I checked it now. If an IOException occurs in sync(), before the channel.close() is attempted within the sync() method, then the current implementation of ZipFileSystem.close() leaves the channel open. This can happen even in the current code where the IOException is caught and swallowed in a couple of places, since there are other places within sync() which can propagate an IOException. It's not just that - the current implementation and even the proposed IOException propagation in this patch can end up leaving the current ZipFileSystem instance within the ZipFileSystemProvider#filesystems Map, since the call to provider.removeFileSystem(zfpath, this) in the close() method of ZipFileSystem ends up being skipped if the internal call to sync() ends up throwing an IOException. I don't have much knowledge of these classes, but I don't think this is a good thing, because any subsequent call to the ZipFileSystemProvider#newFileSystem(URI ...) with the exact same URI as the previous one can result in a FileSystemAlreadyExistsException and any calls to ZipFileSystemProvider#getFileSystem(URI ...) will result in returning a closed instance of the filesystem. Given that ZipFileSystemProvider is a capable of recreating new filesystems using the same URI, the current implementation in the ZipFileSystem#close() method would end up in a state where the ZipFileSystemProvider would violate the contract of the FileSystems#getFileSystem(URI...): ???? * Once a file system created by this provider is {@link FileSystem#close ???? * closed} it is provider-dependent if this method returns a reference to ???? * the closed file system or throws {@link FileSystemNotFoundException}. ???? * If the provider allows a new file system to be created with the same URI ???? * as a file system it previously created then this method throws the ???? * exception if invoked after the file system is closed (and before a new ???? * instance is created by the {@link #newFileSystem newFileSystem} method). Again, as I noted, I am not too familiar with this piece of code (in fact, having now looked at the jdk.nio.zipfs.ZipFileSystemProvider implementation, I am surprised that the newFileSystem implementation behaves completely differently when passed a URI as compared to when passed a Path), so I may be overstating the potential issue. -Jaikiran From lance.andersen at oracle.com Wed Feb 12 22:22:20 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 12 Feb 2020 17:22:20 -0500 Subject: RFR: JDK-8225507: (zipfs): No space left on device not thrown with ZipFileSystem In-Reply-To: <76d82ba5-8cea-9196-3989-2d8b9fd4e4f8@oracle.com> References: <50746bf1-1d46-a0de-b9b5-2fc1aaaa5c7e@gmail.com> <76d82ba5-8cea-9196-3989-2d8b9fd4e4f8@oracle.com> Message-ID: <0E333D93-D52E-4884-B0B2-64AC1261431C@oracle.com> A few observations from looking at: close method: Any open InputStreams are closed so there is a possibility of an IOException being thrown The channel is closed in the same try block that sync() is invoked so there is a potential that if sync() fails the channel is not closed cleanly the removal of the filesystem for URI based invocations of newFileSystem should/could be moved up after the Zip FS is moved closed I suspect we should defer the throwing of the IOException if sync() fails until after we finish the rest of close() similar as is done for an IOException occurring while deleting the temp filess sync method: Looks like if the write of a CEN entry fails the IOException would be thrown where as an issue updating the LOC will just be ignored which I am not sure why that was done Best Lance > On Feb 9, 2020, at 12:02 PM, Alan Bateman wrote: > > On 05/02/2020 23:34, Lance Andersen wrote: >> Hi Jaikiran, >> >> I think overall the patch is correct but it would be good to get some extra eyes on this. > On the surface this looks okay but I think it would be useful if you (or Jai) could write a brief summary about the state of the file system and the underlying channel when the sync fails. I think the zip file system state will be closed (!isOpen) so a later attempt to close it will be a no-op. However, the state of the channel is not clear - is it open or closed? > > -Alan Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From paul.sandoz at oracle.com Wed Feb 12 22:43:13 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 12 Feb 2020 14:43:13 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <8736bhhvst.fsf@oldenburg2.str.redhat.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <87wo8ti01g.fsf@oldenburg2.str.redhat.com> <824238F5-D399-4088-A1AA-A9AC8321577F@oracle.com> <87eev1hwxa.fsf@oldenburg2.str.redhat.com> <8736bhhvst.fsf@oldenburg2.str.redhat.com> Message-ID: <6B7BB465-EA1F-4763-8E2D-927B71EADFC6@oracle.com> I?ll let Brian decide on that from the perspective of Buffers, given IIRC it does not mention anything about overlapping regions. However, the incubating memory segment API does: https://hg.openjdk.java.net/jdk/jdk/file/tip/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryAddress.java#l98 We can capture something here first. My preference is an api note. Care to propose something? Paul. > On Feb 11, 2020, at 9:34 AM, Florian Weimer wrote: > > * Paul Sandoz: > >> Thanks for the clarification. I see what you are getting at now. >> >> Yes, under those aliased circumstances there is not much we can do. >> All we can talk about is overlapping regions in virtual memory. > > Is there a way to capture this in the specification? > > Thanks, > Florian > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Feb 12 23:02:33 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 12 Feb 2020 15:02:33 -0800 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <6B7BB465-EA1F-4763-8E2D-927B71EADFC6@oracle.com> References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <87wo8ti01g.fsf@oldenburg2.str.redhat.com> <824238F5-D399-4088-A1AA-A9AC8321577F@oracle.com> <87eev1hwxa.fsf@oldenburg2.str.redhat.com> <8736bhhvst.fsf@oldenburg2.str.redhat.com> <6B7BB465-EA1F-4763-8E2D-927B71EADFC6@oracle.com> Message-ID: <91B87AC7-2CA7-470D-BAFD-EEC53D7C2374@oracle.com> > On Feb 12, 2020, at 2:43 PM, Paul Sandoz wrote: > > I?ll let Brian decide on that from the perspective of Buffers, given IIRC it does not mention anything about overlapping regions. It only mentions about overlap when the source and target buffers are the same Buffer. That needs to be generalized to ?share the same storage? or some such. > However, the incubating memory segment API does: > > https://hg.openjdk.java.net/jdk/jdk/file/tip/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryAddress.java#l98 This gives a nice, concrete description of what is going on. In the context of buffers an intermediate temporary buffer for when the memory overlaps would be something new I think. Brian > We can capture something here first. My preference is an api note. Care to propose something? > > Paul. > >> On Feb 11, 2020, at 9:34 AM, Florian Weimer > wrote: >> >> * Paul Sandoz: >> >>> Thanks for the clarification. I see what you are getting at now. >>> >>> Yes, under those aliased circumstances there is not much we can do. >>> All we can talk about is overlapping regions in virtual memory. >> >> Is there a way to capture this in the specification? >> >> Thanks, >> Florian >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From patrick.concannon at oracle.com Thu Feb 13 14:27:39 2020 From: patrick.concannon at oracle.com (Patrick Concannon) Date: Thu, 13 Feb 2020 14:27:39 +0000 Subject: RFR[8234812]: 'Add micros for DatagramChannel send/receive' Message-ID: <49bb88c0-04be-6597-f0fa-9e49f6bf03d8@oracle.com> Hi, Could someone please review my webrev for JDK-8234812 'Add micros for DatagramChannel send/receive' ? This test adds some benchmarks for the DatagramChannel's send, receive, read and write methods. bug: https://bugs.openjdk.java.net/browse/JDK-8234812 webrev: http://cr.openjdk.java.net/~pconcannon/8234812/webrevs/webrev.00/ Kind regards, Patrick From chris.hegarty at oracle.com Thu Feb 13 16:53:49 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 13 Feb 2020 16:53:49 +0000 Subject: RFR[8234812]: 'Add micros for DatagramChannel send/receive' In-Reply-To: <49bb88c0-04be-6597-f0fa-9e49f6bf03d8@oracle.com> References: <49bb88c0-04be-6597-f0fa-9e49f6bf03d8@oracle.com> Message-ID: > On 13 Feb 2020, at 14:27, Patrick Concannon wrote: > > Hi, > > > Could someone please review my webrev for JDK-8234812 'Add micros for DatagramChannel send/receive' ? > > This test adds some benchmarks for the DatagramChannel's send, receive, read and write methods. > > > bug: https://bugs.openjdk.java.net/browse/JDK-8234812 > webrev: http://cr.openjdk.java.net/~pconcannon/8234812/webrevs/webrev.00/ > Did you intentionally use heap byte buffers, or should the test use direct buffers? -Chris. From Alan.Bateman at oracle.com Fri Feb 14 09:41:31 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 14 Feb 2020 09:41:31 +0000 Subject: RFR[8234812]: 'Add micros for DatagramChannel send/receive' In-Reply-To: References: <49bb88c0-04be-6597-f0fa-9e49f6bf03d8@oracle.com> Message-ID: <6d7f6cb0-1ec0-3302-eb9c-6dcbf3759153@oracle.com> On 13/02/2020 16:53, Chris Hegarty wrote: > : > Did you intentionally use heap byte buffers, or should the test use direct buffers? > That is a good discussion point as there is a matrix of heap vs. direct, blocking vs. non-blocking, scatter/gather, ... that could be added to create more comprehensive micros. I don't have a strong opinion on this but your suggestion for heap vs. direct would be useful to have as it would help track of the overhead of copying in or out of temporary direct buffers (this overhead has been reduced significantly in recent releases but should still be observable in micros that don't do anything but send/receive or read/write). -Alan From patrick.concannon at oracle.com Fri Feb 14 16:13:41 2020 From: patrick.concannon at oracle.com (Patrick Concannon) Date: Fri, 14 Feb 2020 16:13:41 +0000 Subject: RFR[8234812]: 'Add micros for DatagramChannel send/receive' In-Reply-To: <6d7f6cb0-1ec0-3302-eb9c-6dcbf3759153@oracle.com> References: <49bb88c0-04be-6597-f0fa-9e49f6bf03d8@oracle.com> <6d7f6cb0-1ec0-3302-eb9c-6dcbf3759153@oracle.com> Message-ID: <204e8a4d-8f60-5a3e-3d70-dadf01252be0@oracle.com> Hi, Thanks for the feedback Chris and Alan. I've added an boolean parameter to the benchmark to allow for the use of a direct ByteBuffer -- true for direct, false for heap. In interest of keeping the run time of the benchmark down, I've only included a 'true' value. However, 'false' can be added if a more comprehensive comparison is required. http://cr.openjdk.java.net/~pconcannon/8234812/webrevs/webrev.02/ Kind regards, Patrick On 14/02/2020 09:41, Alan Bateman wrote: > On 13/02/2020 16:53, Chris Hegarty wrote: >> : >> Did you intentionally use heap byte buffers, or should the test use >> direct buffers? >> > That is a good discussion point as there is a matrix of heap vs. > direct, blocking vs. non-blocking, scatter/gather, ... that could be > added to create more comprehensive micros. I don't have a strong > opinion on this but your suggestion for heap vs. direct would be > useful to have as it would help track of the overhead of copying in or > out of temporary direct buffers (this overhead has been reduced > significantly in recent releases but should still be observable in > micros that don't do anything but send/receive or read/write). > > -Alan From lance.andersen at oracle.com Fri Feb 14 20:57:36 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 14 Feb 2020 15:57:36 -0500 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: References: Message-ID: <51EF8E30-12C0-4CDA-9E2E-79F250375567@oracle.com> Hi Jaikiran, I think the changes to ZipFileSystem are OK. The test overall is good. I am going to streamline it a bit and remove the long lines (we try to keep lines to around 80 characters). I will push a revised webrev out once I do this over the next few days Best Lance > On Feb 12, 2020, at 1:39 AM, Jaikiran Pai wrote: > > I realized that the verify() method in the testcase can include a couple of more tests while dealing with the JarInputStream. So I've updated that method and created a fresh webrev which is available at https://cr.openjdk.java.net/~jpai/webrev/8211917/3/webrev/ > -Jaikiran > > On 11/02/20 10:03 pm, Jaikiran Pai wrote: >> Hello Lance, >> >> On 05/02/20 3:13 am, Lance Andersen wrote: >>> Hi Jaikiran, >>> >>> Thank you again for tackling this feature request. >>> >>> Overall, I think this looks OK. >>> >>> In ZipFileSystem.java, I would reverse the if statement given a MANiFEST.MF present is going to be the exception vs the norm. Perhaps something similar to: >>> >>> ???????????? >>> if(manifestInode == null || manifestProcessed) { >>> inode = inodeIterator.next(); >>> if(inode == manifestInode) >>> continue; >>> } else { >>> inode = manifestInode; >>> manifestProcessed = true; >>> } >>> ????????????????? >>> >> Done. >> >> >> >>> A few comments/suggestions on the test: >>> >>> I would prefer that the tests use the newer FileSystems:: newFileSystem methods for the patch to the current openjdk repository >> Done - updated the testcase to use the newer available APIs. >> >> >> >>> Please use Map.of() vs Collections.xxxMap >> Done. >> >> >> >>> We should test with STORED and DEFLATED specified for compression. >>> I would look at the CopyMoveTest and leverage the Entry class and verify method in this test. This will keep the tests looking somewhat similar >> Done - the testcase now tests the default (== DEFLATED), STORED and (explicit) DEFLATED compression methods. >> >> >> >>> Please add a test which copies the Manifest from an OS file to the JAR >> Done. New testManifestCopiedFromOSFile test method added. >> >> >> >>> I would consider adding a test which creates a JAR with a Manifest and then uses Files::copy to create a another JAR from the original JAR >> Done. New testDuplicateJar test method added. >> >> >> >>> I would consider a test which creates the JAR via the jar tool(using the ToolProvider API) and then updates the JAR via ZipFS >> Done. New testJarToolGeneratedJarWithManifest added. >> >> >> >>> You may want to consider executing the JAR if you are setting the main class attribute see the LargeEntriesTest as an example >> In my initial version, I included the Main-Class manifest attribute only to make sure the manifest file that is being verified is indeed the one we had added in the tests. I did not have any intention of testing the "Main-Class" semantics. In this newer updated version of the test case, I decided to just remove the "Main-Class" and instead use a dummy manifest attribute that I just check for equality. I decided to remove the "Main-Class" attribute since I didn't want this test to do too many things. Let me know if you prefer that the Main-Class to stay and be verified that it can be launched. >> >> All these changes are now part of the new webrev which is at https://cr.openjdk.java.net/~jpai/webrev/8211917/2/webrev/ >> -Jaikiran >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From jai.forums2013 at gmail.com Mon Feb 17 02:55:59 2020 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Mon, 17 Feb 2020 08:25:59 +0530 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: <51EF8E30-12C0-4CDA-9E2E-79F250375567@oracle.com> References: <51EF8E30-12C0-4CDA-9E2E-79F250375567@oracle.com> Message-ID: <4256fa6d-cd3b-4852-72c6-7c9acbd8b489@gmail.com> Hello Lance, On 15/02/20 2:27 am, Lance Andersen wrote: > Hi Jaikiran, > > I think the changes to ZipFileSystem are OK. > > The test overall is good. ?I am going to streamline it a bit and > remove the long lines (we try to keep lines to around 80 characters). > I'll keep that in mind for future changes. Thank you for taking care of this. -Jaikiran From linzang at tencent.com Mon Feb 17 10:37:59 2020 From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=) Date: Mon, 17 Feb 2020 10:37:59 +0000 Subject: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) References: <4b9f192a755940cb97fd87b78b7c8469@tencent.com>, , Message-ID: <9b8c922a107d4843b4bf07dc57f49067@tencent.com> From updates in?https://bugs.openjdk.java.net/browse/JDK-8239139, This is related to a test case of recent nio patch. so move this thread to nio-dev. Thanks -------------- Lin >Thanks a lot ! >-------------- >Lin >>On 2020-02-17 08:48, linzang(??) wrote: >>Hi, >> May I ask help to review one tiny fix of build failure described at https://bugs.openjdk.java.net/browse/JDK-8239139 >>???? Root cause is gcc version 8 prints warning for strncpy. >>???? The fix simply replace strncpy with snprintf. >>???? Thanks! >> Bug: https://bugs.openjdk.java.net/browse/JDK-8239139 >> webrev: http://cr.openjdk.java.net/~lzang/8239139/webrev/ >> >>BRs, >>Lin >> >>Hi Lin, >> >>This is not a build issue. I'm redirecting this to core-libs. >> >>/Magnus From Alan.Bateman at oracle.com Mon Feb 17 12:28:33 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 17 Feb 2020 12:28:33 +0000 Subject: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) In-Reply-To: <9b8c922a107d4843b4bf07dc57f49067@tencent.com> References: <4b9f192a755940cb97fd87b78b7c8469@tencent.com> <9b8c922a107d4843b4bf07dc57f49067@tencent.com> Message-ID: <7bdd4472-d1fb-2252-367a-8cd9a0fe44f1@oracle.com> On 17/02/2020 10:37, linzang(??) wrote: > From updates in?https://bugs.openjdk.java.net/browse/JDK-8239139, > This is related to a test case of recent nio patch. > so move this thread to nio-dev. > Yes, this test was expanded recently to run the launcher with stdin connected to a Unix domain socket. The len parameter should have been sizeof(addr.sun_path) but changing this to snprintf looks okay too. -Alan. From christoph.langer at sap.com Mon Feb 17 16:26:29 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 17 Feb 2020 16:26:29 +0000 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: <4256fa6d-cd3b-4852-72c6-7c9acbd8b489@gmail.com> References: <51EF8E30-12C0-4CDA-9E2E-79F250375567@oracle.com> <4256fa6d-cd3b-4852-72c6-7c9acbd8b489@gmail.com> Message-ID: Hi Jaikiran, Lance, I'm a bit late to the game. It occurred to me, whether we could maybe use the streams API to process the inodes. Something like this: diff --git a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java @@ -56,6 +56,7 @@ import java.util.jar.Attributes; import java.util.jar.Manifest; import java.util.regex.Pattern; +import java.util.stream.Stream; import java.util.zip.CRC32; import java.util.zip.Deflater; import java.util.zip.DeflaterOutputStream; @@ -1730,7 +1731,17 @@ Entry e; // write loc - for (IndexNode inode : inodes.values()) { + + // write the manifest inode (if any) first in the loc so that the + // java.util.jar.JarInputStream can find it, since it expects it to be + // the first or second entry in the jar + final IndexNode manifestInode = getInode(getBytes("/META-INF/MANIFEST.MF")); + final Stream inodeStream = manifestInode == null ? + inodes.values().stream() : + Stream.concat(Stream.of(manifestInode), + inodes.values().stream().filter(node -> !manifestInode.equals(node))); + + for (IndexNode inode : (Iterable)inodeStream::iterator) { if (inode instanceof Entry) { // an updated inode e = (Entry)inode; try { What do you think? I didn't have time yet to have a closer look to the testcase. Will do tomorrow. Cheers Christoph > -----Original Message----- > From: nio-dev On Behalf Of Jaikiran > Pai > Sent: Montag, 17. Februar 2020 03:56 > To: Lance Andersen > Cc: nio-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > Subject: Re: RFR: 8211917: (zipfs) Creating or updating a JAR file system > should put the MANIFEST.MF at the start > > Hello Lance, > > On 15/02/20 2:27 am, Lance Andersen wrote: > > Hi Jaikiran, > > > > I think the changes to ZipFileSystem are OK. > > > > The test overall is good. ?I am going to streamline it a bit and > > remove the long lines (we try to keep lines to around 80 characters). > > > I'll keep that in mind for future changes. Thank you for taking care of > this. > > -Jaikiran From linzang at tencent.com Tue Feb 18 00:27:19 2020 From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=) Date: Tue, 18 Feb 2020 00:27:19 +0000 Subject: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) References: <4b9f192a755940cb97fd87b78b7c8469@tencent.com>, , , <9b8c922a107d4843b4bf07dc57f49067@tencent.com>, <7bdd4472-d1fb-2252-367a-8cd9a0fe44f1@oracle.com> Message-ID: <34dd37811567470882af8165dc9ad875@tencent.com> Thanks Alan.? Hi all,? ? ? May I get more review about this tiny patch? Thanks! -------------- Lin >On 17/02/2020 10:37, linzang(??) wrote: >> From updates in?https://bugs.openjdk.java.net/browse/JDK-8239139, >> This is related to a test case of recent nio patch. >> so move this thread to nio-dev. >> >Yes, this test was expanded recently to run the launcher with stdin >connected to a Unix domain socket. The len parameter should have been >sizeof(addr.sun_path) but changing this to snprintf looks okay too. > >-Alan. > From vtewar26 at in.ibm.com Tue Feb 18 04:42:16 2020 From: vtewar26 at in.ibm.com (Vyom Tewari26) Date: Tue, 18 Feb 2020 04:42:16 +0000 Subject: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) In-Reply-To: <34dd37811567470882af8165dc9ad875@tencent.com> References: <34dd37811567470882af8165dc9ad875@tencent.com>, <4b9f192a755940cb97fd87b78b7c8469@tencent.com>, , , <9b8c922a107d4843b4bf07dc57f49067@tencent.com>, <7bdd4472-d1fb-2252-367a-8cd9a0fe44f1@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From linzang at tencent.com Tue Feb 18 06:05:37 2020 From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=) Date: Tue, 18 Feb 2020 06:05:37 +0000 Subject: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) References: <34dd37811567470882af8165dc9ad875@tencent.com>, <4b9f192a755940cb97fd87b78b7c8469@tencent.com>, , , <9b8c922a107d4843b4bf07dc57f49067@tencent.com>, <7bdd4472-d1fb-2252-367a-8cd9a0fe44f1@oracle.com>, Message-ID: <57a1b181ff624aaa8bb915dc11fcc3ac@tencent.com> Dear Vyom,? ? ? ? ? Your change looks good to me.? ? ? ? ? And I have updated it to?http://cr.openjdk.java.net/~lzang/8239139/webrev02/. Thanks,? Lin >Hi Lin, > >thanks for? fixing this issue, i think your changes will have similar warnings with -Wformat-truncation flag. What do you think about below patch ?. > >Thanks, >Vyom > >--- a/test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/libInheritedChannel.c??? Mon Feb 17 18:52:50 2020 -0800 >+++ b/test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/libInheritedChannel.c??? Tue Feb 18 10:09:50 2020 +0530 >@@ -232,11 +232,12 @@ > { >???? struct sockaddr_un addr; >???? const char *nameUtf = (*env)->GetStringUTFChars(env, name, NULL); >-??? int ret = -1; >+??? int ret = -1, length=sizeof(addr.sun_path); >???? unlink(nameUtf); >???? memset(&addr, 0, sizeof(addr)); >???? addr.sun_family = AF_UNIX; >-??? strncpy(addr.sun_path, nameUtf, strlen(nameUtf)); >+??? strncpy(addr.sun_path, nameUtf, length); >+??? addr.sun_path[length - 1] = '\0'; >???? ret = bind(sock, (const struct sockaddr*)&addr, sizeof(addr)); >???? if (ret == -1) { >???????? ThrowException(env, "java/io/IOException", "socket bind error"); > >----- Original message ----- >From: "linzang(??)" >Sent by: "nio-dev" >To: Alan Bateman , nio-dev >Cc: >Subject: [EXTERNAL] Re: Re: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) >Date: Tue, Feb 18, 2020 5:57 AM >? >Thanks Alan. > >Hi all, >??? May I get more review about this tiny patch? Thanks! > >-------------- >Lin >>On 17/02/2020 10:37, linzang(??) wrote: >>> From updates in https://bugs.openjdk.java.net/browse/JDK-8239139 , >>> This is related to a test case of recent nio patch. >>> so move this thread to nio-dev. >>> >>Yes, this test was expanded recently to run the launcher with stdin >>connected to a Unix domain socket. The len parameter should have been >>sizeof(addr.sun_path) but changing this to snprintf looks okay too. >> >>-Alan. >> > > From jai.forums2013 at gmail.com Tue Feb 18 08:29:05 2020 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Tue, 18 Feb 2020 13:59:05 +0530 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: References: <51EF8E30-12C0-4CDA-9E2E-79F250375567@oracle.com> <4256fa6d-cd3b-4852-72c6-7c9acbd8b489@gmail.com> Message-ID: Hello Christoph, On 17/02/20 9:56 pm, Langer, Christoph wrote: > Hi Jaikiran, Lance, > > I'm a bit late to the game. It occurred to me, whether we could maybe use the streams API to process the inodes. Something like this: > > diff --git a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java > --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java > +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java > @@ -56,6 +56,7 @@ > import java.util.jar.Attributes; > import java.util.jar.Manifest; > import java.util.regex.Pattern; > +import java.util.stream.Stream; > import java.util.zip.CRC32; > import java.util.zip.Deflater; > import java.util.zip.DeflaterOutputStream; > @@ -1730,7 +1731,17 @@ > Entry e; > > // write loc > - for (IndexNode inode : inodes.values()) { > + > + // write the manifest inode (if any) first in the loc so that the > + // java.util.jar.JarInputStream can find it, since it expects it to be > + // the first or second entry in the jar > + final IndexNode manifestInode = getInode(getBytes("/META-INF/MANIFEST.MF")); > + final Stream inodeStream = manifestInode == null ? > + inodes.values().stream() : > + Stream.concat(Stream.of(manifestInode), > + inodes.values().stream().filter(node -> !manifestInode.equals(node))); > + > + for (IndexNode inode : (Iterable)inodeStream::iterator) { I don't have any practical experience with Stream and how it performs. Would using Stream here introduce any kind of performance penalty, especially when the number of entries in the jar is high? -Jaikiran From gnu.andrew at redhat.com Tue Feb 18 09:20:51 2020 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Tue, 18 Feb 2020 09:20:51 +0000 Subject: RFR(8u): 8229872: (fs) Increase buffer size used with getmntent In-Reply-To: References: <5CCC5140-4C75-45CD-A3A2-CC0237DD7AD9@azul.com> <7B2B9DF5-F845-42FC-949C-102B72282975@amazon.com> Message-ID: <470ddc7c-fc1b-2dc5-25be-97d024eef061@redhat.com> On 28/10/2019 09:20, Vladimir Kempik wrote: > Hello > Can anyone please take a look at this patch? > The difference between 11 and 8 is pretty simple: move one method from superclass to subclass to prevent it being compiled on every Unix system(it can?t be compiled on solaris). This code is only used on Linux so far. > > Thanks, Vladimir > > 21 ???. 2019 ?., ? 19:23, Hohensee, Paul > ???????(?): > > +jdk8u-dev > > Paul > > From: nio-dev > on behalf of Vladimir Kempik > > Date: Monday, October 21, 2019 at 2:47 AM > To: "nio-dev at openjdk.java.net" > > Subject: RFR(8u): 8229872: (fs) Increase buffer size used with getmntent > > Hello > > Please review backport of https://bugs.openjdk.java.net/browse/JDK-8229872 to 8u: > http://cr.openjdk.java.net/~vkempik/8229872/ojdk8/webrev.00/ > > Difference between 14/13/11 fix: getline is missing on solaris10, so getlinelen method was moved from UnixNativeDispatcher to LinuxNativeDispatcher. > Original fix: https://hg.openjdk.java.net/jdk/jdk/rev/2d40e6a7ce8e > > Such fix was already included into azul?s october psu for openjdk8 and passed all release cycle tests. > > Thanks, Vladimir > An addition to the comments above getlinelen in LinuxNativeDispatcher.c as to why it is located there rather than in UnixNativeDispatcher.c would be good for anyone trying to understand this difference in future (no need for a new webrev, just add before push). Approved for 8u. Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: OpenPGP digital signature URL: From vtewar26 at in.ibm.com Tue Feb 18 13:10:17 2020 From: vtewar26 at in.ibm.com (Vyom Tewari26) Date: Tue, 18 Feb 2020 13:10:17 +0000 Subject: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) In-Reply-To: <57a1b181ff624aaa8bb915dc11fcc3ac@tencent.com> References: <57a1b181ff624aaa8bb915dc11fcc3ac@tencent.com>, <34dd37811567470882af8165dc9ad875@tencent.com>, <4b9f192a755940cb97fd87b78b7c8469@tencent.com>, , , <9b8c922a107d4843b4bf07dc57f49067@tencent.com>, <7bdd4472-d1fb-2252-367a-8cd9a0fe44f1@oracle.com>, Message-ID: An HTML attachment was scrubbed... URL: From linzang at tencent.com Wed Feb 19 01:31:44 2020 From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=) Date: Wed, 19 Feb 2020 01:31:44 +0000 Subject: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) References: <57a1b181ff624aaa8bb915dc11fcc3ac@tencent.com>, <34dd37811567470882af8165dc9ad875@tencent.com>, <4b9f192a755940cb97fd87b78b7c8469@tencent.com>, , , <9b8c922a107d4843b4bf07dc57f49067@tencent.com>, <7bdd4472-d1fb-2252-367a-8cd9a0fe44f1@oracle.com>, , Message-ID: Thanks! So can this change be merge ? or it needs more review? BRs, Lin >Looks good to me. >Vyom > >----- Original message ----- >From: "linzang(??)" >To: Vyom Tewari26 >Cc: "linzang(??)" , nio-dev >Subject: [EXTERNAL] Re: RE: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) >Date: Tue, Feb 18, 2020 11:35 AM >? >Dear Vyom, >??????? Your change looks good to me. > And I have updated it to http://cr.openjdk.java.net/~lzang/8239139/webrev02/ . > >Thanks, >Lin > >>Hi Lin, >> >>thanks for? fixing this issue, i think your changes will have similar warnings with -Wformat-truncation flag. What do you think about below patch ?. >> >>Thanks, >>Vyom >> >>--- a/test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/libInheritedChannel.c??? Mon Feb 17 18:52:50 2020 -0800 >>+++ b/test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/libInheritedChannel.c??? Tue Feb 18 10:09:50 2020 +0530 >>@@ -232,11 +232,12 @@ >> { >>???? struct sockaddr_un addr; >>???? const char *nameUtf = (*env)->GetStringUTFChars(env, name, NULL); >>-??? int ret = -1; >>+??? int ret = -1, length=sizeof(addr.sun_path); >>???? unlink(nameUtf); >>???? memset(&addr, 0, sizeof(addr)); >>???? addr.sun_family = AF_UNIX; >>-??? strncpy(addr.sun_path, nameUtf, strlen(nameUtf)); >>+??? strncpy(addr.sun_path, nameUtf, length); >>+??? addr.sun_path[length - 1] = '\0'; >>???? ret = bind(sock, (const struct sockaddr*)&addr, sizeof(addr)); >>???? if (ret == -1) { >>???????? ThrowException(env, "java/io/IOException", "socket bind error"); >> >>----- Original message ----- >>From: "linzang(??)" >>Sent by: "nio-dev" >>To: Alan Bateman , nio-dev >>Cc: >>Subject: [EXTERNAL] Re: Re: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) >>Date: Tue, Feb 18, 2020 5:57 AM >> >>Thanks Alan. >> >>Hi all, >>??? May I get more review about this tiny patch? Thanks! >> >>-------------- >>Lin >>>On 17/02/2020 10:37, linzang(??) wrote: >>>> From updates in https://bugs.openjdk.java.net/browse/JDK-8239139 , >>>> This is related to a test case of recent nio patch. >>>> so move this thread to nio-dev. >>>> >>>Yes, this test was expanded recently to run the launcher with stdin >>>connected to a Unix domain socket. The len parameter should have been >>>sizeof(addr.sun_path) but changing this to snprintf looks okay too. >>> >>>-Alan. >>> >> >> > > From lance.andersen at oracle.com Wed Feb 19 01:42:23 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 18 Feb 2020 20:42:23 -0500 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: References: <51EF8E30-12C0-4CDA-9E2E-79F250375567@oracle.com> <4256fa6d-cd3b-4852-72c6-7c9acbd8b489@gmail.com> Message-ID: <3B2A5184-B592-4F3F-80AF-5FCF8AF7035C@oracle.com> Hi Christoph > On Feb 17, 2020, at 11:26 AM, Langer, Christoph wrote: > > Hi Jaikiran, Lance, > > I'm a bit late to the game. It occurred to me, whether we could maybe use the streams API to process the inodes. Something like this: > > diff --git a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java > --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java > +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java > @@ -56,6 +56,7 @@ > import java.util.jar.Attributes; > import java.util.jar.Manifest; > import java.util.regex.Pattern; > +import java.util.stream.Stream; > import java.util.zip.CRC32; > import java.util.zip.Deflater; > import java.util.zip.DeflaterOutputStream; > @@ -1730,7 +1731,17 @@ > Entry e; > > // write loc > - for (IndexNode inode : inodes.values()) { > + > + // write the manifest inode (if any) first in the loc so that the > + // java.util.jar.JarInputStream can find it, since it expects it to be > + // the first or second entry in the jar > + final IndexNode manifestInode = getInode(getBytes("/META-INF/MANIFEST.MF")); > + final Stream inodeStream = manifestInode == null ? > + inodes.values().stream() : > + Stream.concat(Stream.of(manifestInode), > + inodes.values().stream().filter(node -> !manifestInode.equals(node))); > + > + for (IndexNode inode : (Iterable)inodeStream::iterator) { > if (inode instanceof Entry) { // an updated inode > e = (Entry)inode; > try { > > What do you think? I am not sure the use of Streams will help us too much here as in your proposed example I believe you would end out traversing the entire list of IndexNodes twice in the case of a Manifest existing. Once when building the Stream and the other when you are walking through the IndexNodes to write them out. Best Lance > > I didn't have time yet to have a closer look to the testcase. Will do tomorrow. > > Cheers > Christoph > >> -----Original Message----- >> From: nio-dev On Behalf Of Jaikiran >> Pai >> Sent: Montag, 17. Februar 2020 03:56 >> To: Lance Andersen >> Cc: nio-dev at openjdk.java.net; core-libs-dev at openjdk.java.net >> Subject: Re: RFR: 8211917: (zipfs) Creating or updating a JAR file system >> should put the MANIFEST.MF at the start >> >> Hello Lance, >> >> On 15/02/20 2:27 am, Lance Andersen wrote: >>> Hi Jaikiran, >>> >>> I think the changes to ZipFileSystem are OK. >>> >>> The test overall is good. I am going to streamline it a bit and >>> remove the long lines (we try to keep lines to around 80 characters). >>> >> I'll keep that in mind for future changes. Thank you for taking care of >> this. >> >> -Jaikiran > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From vtewar26 at in.ibm.com Wed Feb 19 05:51:08 2020 From: vtewar26 at in.ibm.com (Vyom Tewari26) Date: Wed, 19 Feb 2020 05:51:08 +0000 Subject: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) In-Reply-To: References: , <57a1b181ff624aaa8bb915dc11fcc3ac@tencent.com>, <34dd37811567470882af8165dc9ad875@tencent.com>, <4b9f192a755940cb97fd87b78b7c8469@tencent.com>, , , <9b8c922a107d4843b4bf07dc57f49067@tencent.com>, <7bdd4472-d1fb-2252-367a-8cd9a0fe44f1@oracle.com>, , Message-ID: An HTML attachment was scrubbed... URL: From linzang at tencent.com Wed Feb 19 05:54:30 2020 From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=) Date: Wed, 19 Feb 2020 05:54:30 +0000 Subject: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) References: , <57a1b181ff624aaa8bb915dc11fcc3ac@tencent.com>, <34dd37811567470882af8165dc9ad875@tencent.com>, <4b9f192a755940cb97fd87b78b7c8469@tencent.com>, , , <9b8c922a107d4843b4bf07dc57f49067@tencent.com>, <7bdd4472-d1fb-2252-367a-8cd9a0fe44f1@oracle.com>, , , Message-ID: Thanks Vyom!? Dear All, ? ? May I ask your help to review this tiny patch?? ? ??http://cr.openjdk.java.net/~lzang/8239139/webrev02/? Thanks, Lin >I am not the official OpenJDK reviewer, i think you need at least one more review. >Vyom > >----- Original message ----- >From: "linzang(??)" >To: Vyom Tewari26 , nio-dev >Cc: "linzang(??)" >Subject: [EXTERNAL] Re: RE: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) >Date: Wed, Feb 19, 2020 7:01 AM >? >Thanks! > >So can this change be merge ? or it needs more review? > >BRs, >Lin >>Looks good to me. >>Vyom >> >>----- Original message ----- >>From: "linzang(??)" >>To: Vyom Tewari26 >>Cc: "linzang(??)" , nio-dev >>Subject: [EXTERNAL] Re: RE: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) >>Date: Tue, Feb 18, 2020 11:35 AM >> >>Dear Vyom, >>??????? Your change looks good to me. >> And I have updated it to http://cr.openjdk.java.net/~lzang/8239139/webrev02/ . >> >>Thanks, >>Lin >> >>>Hi Lin, >>> >>>thanks for? fixing this issue, i think your changes will have similar warnings with -Wformat-truncation flag. What do you think about below patch ?. >>> >>>Thanks, >>>Vyom >>> >>>--- a/test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/libInheritedChannel.c??? Mon Feb 17 18:52:50 2020 -0800 >>>+++ b/test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/libInheritedChannel.c??? Tue Feb 18 10:09:50 2020 +0530 >>>@@ -232,11 +232,12 @@ >>> { >>>???? struct sockaddr_un addr; >>>???? const char *nameUtf = (*env)->GetStringUTFChars(env, name, NULL); >>>-??? int ret = -1; >>>+??? int ret = -1, length=sizeof(addr.sun_path); >>>???? unlink(nameUtf); >>>???? memset(&addr, 0, sizeof(addr)); >>>???? addr.sun_family = AF_UNIX; >>>-??? strncpy(addr.sun_path, nameUtf, strlen(nameUtf)); >>>+??? strncpy(addr.sun_path, nameUtf, length); >>>+??? addr.sun_path[length - 1] = '\0'; >>>???? ret = bind(sock, (const struct sockaddr*)&addr, sizeof(addr)); >>>???? if (ret == -1) { >>>???????? ThrowException(env, "java/io/IOException", "socket bind error"); >>> >>>----- Original message ----- >>>From: "linzang(??)" >>>Sent by: "nio-dev" >>>To: Alan Bateman , nio-dev >>>Cc: >>>Subject: [EXTERNAL] Re: Re: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) >>>Date: Tue, Feb 18, 2020 5:57 AM >>> >>>Thanks Alan. >>> >>>Hi all, >>>??? May I get more review about this tiny patch? Thanks! >>> >>>-------------- >>>Lin >>>>On 17/02/2020 10:37, linzang(??) wrote: >>>>> From updates in https://bugs.openjdk.java.net/browse/JDK-8239139 , >>>>> This is related to a test case of recent nio patch. >>>>> so move this thread to nio-dev. >>>>> >>>>Yes, this test was expanded recently to run the launcher with stdin >>>>connected to a Unix domain socket. The len parameter should have been >>>>sizeof(addr.sun_path) but changing this to snprintf looks okay too. >>>> >>>>-Alan. >>>> >>> >>> >> >> > > From michael.x.mcmahon at oracle.com Wed Feb 19 08:38:18 2020 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Wed, 19 Feb 2020 08:38:18 +0000 Subject: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) In-Reply-To: References: <57a1b181ff624aaa8bb915dc11fcc3ac@tencent.com> <34dd37811567470882af8165dc9ad875@tencent.com> <4b9f192a755940cb97fd87b78b7c8469@tencent.com> <9b8c922a107d4843b4bf07dc57f49067@tencent.com> <7bdd4472-d1fb-2252-367a-8cd9a0fe44f1@oracle.com> Message-ID: <5a7323c1-7e0d-da31-2a08-3d8fa7577128@oracle.com> Hi, I think the patch looks fine. I presume you have checked that it does not cause a warning with gcc 8.3.1? I will check it on other platforms and sponsor it for you. Thanks, Michael. On 19/02/2020 05:54, linzang(??) wrote: > Thanks Vyom! > > Dear All, > ? ? May I ask your help to review this tiny patch? > ? ??http://cr.openjdk.java.net/~lzang/8239139/webrev02/ > > Thanks, > Lin >> I am not the official OpenJDK reviewer, i think you need at least one more review. >> Vyom >> >> ----- Original message ----- >> From: "linzang(??)" >> To: Vyom Tewari26 , nio-dev >> Cc: "linzang(??)" >> Subject: [EXTERNAL] Re: RE: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) >> Date: Wed, Feb 19, 2020 7:01 AM >> >> Thanks! >> >> So can this change be merge ? or it needs more review? >> >> BRs, >> Lin >>> Looks good to me. >>> Vyom >>> >>> ----- Original message ----- >>> From: "linzang(??)" >>> To: Vyom Tewari26 >>> Cc: "linzang(??)" , nio-dev >>> Subject: [EXTERNAL] Re: RE: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) >>> Date: Tue, Feb 18, 2020 11:35 AM >>> >>> Dear Vyom, >>> ??????? Your change looks good to me. >>> And I have updated it to http://cr.openjdk.java.net/~lzang/8239139/webrev02/ . >>> >>> Thanks, >>> Lin >>> >>>> Hi Lin, >>>> >>>> thanks for? fixing this issue, i think your changes will have similar warnings with -Wformat-truncation flag. What do you think about below patch ?. >>>> >>>> Thanks, >>>> Vyom >>>> >>>> --- a/test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/libInheritedChannel.c??? Mon Feb 17 18:52:50 2020 -0800 >>>> +++ b/test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/libInheritedChannel.c??? Tue Feb 18 10:09:50 2020 +0530 >>>> @@ -232,11 +232,12 @@ >>>> { >>>> ???? struct sockaddr_un addr; >>>> ???? const char *nameUtf = (*env)->GetStringUTFChars(env, name, NULL); >>>> -??? int ret = -1; >>>> +??? int ret = -1, length=sizeof(addr.sun_path); >>>> ???? unlink(nameUtf); >>>> ???? memset(&addr, 0, sizeof(addr)); >>>> ???? addr.sun_family = AF_UNIX; >>>> -??? strncpy(addr.sun_path, nameUtf, strlen(nameUtf)); >>>> +??? strncpy(addr.sun_path, nameUtf, length); >>>> +??? addr.sun_path[length - 1] = '\0'; >>>> ???? ret = bind(sock, (const struct sockaddr*)&addr, sizeof(addr)); >>>> ???? if (ret == -1) { >>>> ???????? ThrowException(env, "java/io/IOException", "socket bind error"); >>>> >>>> ----- Original message ----- >>>> From: "linzang(??)" >>>> Sent by: "nio-dev" >>>> To: Alan Bateman , nio-dev >>>> Cc: >>>> Subject: [EXTERNAL] Re: Re: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) >>>> Date: Tue, Feb 18, 2020 5:57 AM >>>> >>>> Thanks Alan. >>>> >>>> Hi all, >>>> ??? May I get more review about this tiny patch? Thanks! >>>> >>>> -------------- >>>> Lin >>>>> On 17/02/2020 10:37, linzang(??) wrote: >>>>>> From updates in https://bugs.openjdk.java.net/browse/JDK-8239139 , >>>>>> This is related to a test case of recent nio patch. >>>>>> so move this thread to nio-dev. >>>>>> >>>>> Yes, this test was expanded recently to run the launcher with stdin >>>>> connected to a Unix domain socket. The len parameter should have been >>>>> sizeof(addr.sun_path) but changing this to snprintf looks okay too. >>>>> >>>>> -Alan. >>>>> >>>> >>> > > From linzang at tencent.com Wed Feb 19 09:05:53 2020 From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=) Date: Wed, 19 Feb 2020 09:05:53 +0000 Subject: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) References: , <57a1b181ff624aaa8bb915dc11fcc3ac@tencent.com>, <34dd37811567470882af8165dc9ad875@tencent.com>, <4b9f192a755940cb97fd87b78b7c8469@tencent.com>, , , <9b8c922a107d4843b4bf07dc57f49067@tencent.com>, <7bdd4472-d1fb-2252-367a-8cd9a0fe44f1@oracle.com>, , , , , <5a7323c1-7e0d-da31-2a08-3d8fa7577128@oracle.com> Message-ID: Dear Michael,? ? ? ? ?Yes it compiled successfully without any warning on my machine with gcc 8.3.1. ? ? ? ?And thanks a lot for sponsoring. BRs, Lin >Hi, > >I think the patch looks fine. I presume you have checked that it does >not cause a warning with gcc 8.3.1? > >I will check it on other platforms and sponsor it for you. > >Thanks, > >Michael. > >On 19/02/2020 05:54, linzang(??) wrote: >> Thanks Vyom! >> >> Dear All, >>? ? ? May I ask your help to review this tiny patch? >> ? ??http://cr.openjdk.java.net/~lzang/8239139/webrev02/ >> >> Thanks, >> Lin >>> I am not the official OpenJDK reviewer, i think you need at least one more review. >>> Vyom >>> >>> ----- Original message ----- >>> From: "linzang(??)" >>> To: Vyom Tewari26 , nio-dev >>> Cc: "linzang(??)" >>> Subject: [EXTERNAL] Re: RE: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) >>> Date: Wed, Feb 19, 2020 7:01 AM >>>?? >>> Thanks! >>> >>> So can this change be merge ? or it needs more review? >>> >>> BRs, >>> Lin >>>> Looks good to me. >>>> Vyom >>>> >>>> ----- Original message ----- >>>> From: "linzang(??)" >>>> To: Vyom Tewari26 >>>> Cc: "linzang(??)" , nio-dev >>>> Subject: [EXTERNAL] Re: RE: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) >>>> Date: Tue, Feb 18, 2020 11:35 AM >>>> >>>> Dear Vyom, >>>>? ??????? Your change looks good to me. >>>> And I have updated it to http://cr.openjdk.java.net/~lzang/8239139/webrev02/ . >>>> >>>> Thanks, >>>> Lin >>>> >>>>> Hi Lin, >>>>> >>>>> thanks for? fixing this issue, i think your changes will have similar warnings with -Wformat-truncation flag. What do you think about below patch ?. >>>>> >>>>> Thanks, >>>>> Vyom >>>>> >>>>> --- a/test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/libInheritedChannel.c??? Mon Feb 17 18:52:50 2020 -0800 >>>>> +++ b/test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/libInheritedChannel.c??? Tue Feb 18 10:09:50 2020 +0530 >>>>> @@ -232,11 +232,12 @@ >>>>> { >>>>>? ???? struct sockaddr_un addr; >>>>>? ???? const char *nameUtf = (*env)->GetStringUTFChars(env, name, NULL); >>>>> -??? int ret = -1; >>>>> +??? int ret = -1, length=sizeof(addr.sun_path); >>>>>? ???? unlink(nameUtf); >>>>>? ???? memset(&addr, 0, sizeof(addr)); >>>>>? ???? addr.sun_family = AF_UNIX; >>>>> -??? strncpy(addr.sun_path, nameUtf, strlen(nameUtf)); >>>>> +??? strncpy(addr.sun_path, nameUtf, length); >>>>> +??? addr.sun_path[length - 1] = '\0'; >>>>>? ???? ret = bind(sock, (const struct sockaddr*)&addr, sizeof(addr)); >>>>>? ???? if (ret == -1) { >>>>>? ???????? ThrowException(env, "java/io/IOException", "socket bind error"); >>>>> >>>>> ----- Original message ----- >>>>> From: "linzang(??)" >>>>> Sent by: "nio-dev" >>>>> To: Alan Bateman , nio-dev >>>>> Cc: >>>>> Subject: [EXTERNAL] Re: Re: RFR: JDK-8239139 testmake fail with warning about strncpy using gcc version 8(Internet mail) >>>>> Date: Tue, Feb 18, 2020 5:57 AM >>>>> >>>>> Thanks Alan. >>>>> >>>>> Hi all, >>>>>? ??? May I get more review about this tiny patch? Thanks! >>>>> >>>>> -------------- >>>>> Lin >>>>>> On 17/02/2020 10:37, linzang(??) wrote: >>>>>>> From updates in https://bugs.openjdk.java.net/browse/JDK-8239139 , >>>>>>> This is related to a test case of recent nio patch. >>>>>>> so move this thread to nio-dev. >>>>>>> >>>>>> Yes, this test was expanded recently to run the launcher with stdin >>>>>> connected to a Unix domain socket. The len parameter should have been >>>>>> sizeof(addr.sun_path) but changing this to snprintf looks okay too. >>>>>> >>>>>> -Alan. >>>>>> >>>>> >>>> >> > > From christoph.langer at sap.com Wed Feb 19 09:55:07 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 19 Feb 2020 09:55:07 +0000 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: <3B2A5184-B592-4F3F-80AF-5FCF8AF7035C@oracle.com> References: <51EF8E30-12C0-4CDA-9E2E-79F250375567@oracle.com> <4256fa6d-cd3b-4852-72c6-7c9acbd8b489@gmail.com> <3B2A5184-B592-4F3F-80AF-5FCF8AF7035C@oracle.com> Message-ID: Hi Lance, Jaikiran, you?re probably right. Both, the gut feeling that using streams at all here will bring some performance penalties as well as the potential double traversal of the value map speak against the suggestion I made. I was actually not really convinced myself ? however, I was looking for a way to write the coding in a more concise manner (without performance penalties, of course). So, going back to Jaikiran?s latest proposal, I only have a few minor performance tweaks. Please look at this webrev I generated: http://cr.openjdk.java.net/~clanger/webrevs/8211917.0/ First, the manifest node should be looked up by the statement ?inodes.get(IndexNode.keyOf(getBytes("/META-INF/MANIFEST.MF")))?. Using ?getInode(getBytes("/META-INF/MANIFEST.MF"))? would incur an unnecessary Object.requireNonNull() check and would also call the ?entryLookup? function that is needed for mapped entries in MR jars. But even in MR jars, the manifest would not be mapped to some other versioned node. And then within the loop body I restructured the if statements such that in the case of no manifest (probably the most common case), just one ?if (manifestInode == null)? check is performed and not another ?if (inode == manifestInode)? after the call to inodeIterator.next(). So, just minor tweaks but probably still worthwile doing. ?? Note that I also added a space after ?while? to be consistent with the rest of the file. Other than that I?m good with the change. The testcase looks comprising and zipfs tests are passing. Best regards Christoph From: Lance Andersen Sent: Mittwoch, 19. Februar 2020 02:42 To: Langer, Christoph Cc: Jaikiran Pai ; nio-dev at openjdk.java.net; core-libs-dev at openjdk.java.net Subject: Re: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start Hi Christoph On Feb 17, 2020, at 11:26 AM, Langer, Christoph > wrote: Hi Jaikiran, Lance, I'm a bit late to the game. It occurred to me, whether we could maybe use the streams API to process the inodes. Something like this: diff --git a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java @@ -56,6 +56,7 @@ import java.util.jar.Attributes; import java.util.jar.Manifest; import java.util.regex.Pattern; +import java.util.stream.Stream; import java.util.zip.CRC32; import java.util.zip.Deflater; import java.util.zip.DeflaterOutputStream; @@ -1730,7 +1731,17 @@ Entry e; // write loc - for (IndexNode inode : inodes.values()) { + + // write the manifest inode (if any) first in the loc so that the + // java.util.jar.JarInputStream can find it, since it expects it to be + // the first or second entry in the jar + final IndexNode manifestInode = getInode(getBytes("/META-INF/MANIFEST.MF")); + final Stream inodeStream = manifestInode == null ? + inodes.values().stream() : + Stream.concat(Stream.of(manifestInode), + inodes.values().stream().filter(node -> !manifestInode.equals(node))); + + for (IndexNode inode : (Iterable)inodeStream::iterator) { if (inode instanceof Entry) { // an updated inode e = (Entry)inode; try { What do you think? I am not sure the use of Streams will help us too much here as in your proposed example I believe you would end out traversing the entire list of IndexNodes twice in the case of a Manifest existing. Once when building the Stream and the other when you are walking through the IndexNodes to write them out. Best Lance I didn't have time yet to have a closer look to the testcase. Will do tomorrow. Cheers Christoph -----Original Message----- From: nio-dev > On Behalf Of Jaikiran Pai Sent: Montag, 17. Februar 2020 03:56 To: Lance Andersen > Cc: nio-dev at openjdk.java.net; core-libs-dev at openjdk.java.net Subject: Re: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start Hello Lance, On 15/02/20 2:27 am, Lance Andersen wrote: Hi Jaikiran, I think the changes to ZipFileSystem are OK. The test overall is good. I am going to streamline it a bit and remove the long lines (we try to keep lines to around 80 characters). I'll keep that in mind for future changes. Thank you for taking care of this. -Jaikiran [cid:image001.gif at 01D5E713.0B83F590] Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 658 bytes Desc: image001.gif URL: From christoph.langer at sap.com Wed Feb 19 10:14:16 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 19 Feb 2020 10:14:16 +0000 Subject: RFR: JDK-8225507: (zipfs): No space left on device not thrown with ZipFileSystem In-Reply-To: <0E333D93-D52E-4884-B0B2-64AC1261431C@oracle.com> References: <50746bf1-1d46-a0de-b9b5-2fc1aaaa5c7e@gmail.com> <76d82ba5-8cea-9196-3989-2d8b9fd4e4f8@oracle.com> <0E333D93-D52E-4884-B0B2-64AC1261431C@oracle.com> Message-ID: Hi, I just had a look at the implementations of close() and sync() and from what I can see there should probably be a lot more done to make sure the ZipFileSystem instance is in a closed and clean state when an IOException was encountered along the path of writing it. In sync, we should probably move some stuff to a finally block (e.g. closing of the channel, make sure the tmpFile is removed etc.). I'm also not sure about the exChClosers. I don't fully understand yet what this is needed for. I guess this should be examined, too. In close(), there are probably also quite some steps that could be moved to a finally block, even if sync() came back with an Exception. Let me see if I find the time to come up with some prototype soon... But overall, I agree, these IOExceptions definitely shouldn't be swallowed and rather be propagated to the caller. Best regards Christoph From: nio-dev On Behalf Of Lance Andersen Sent: Mittwoch, 12. Februar 2020 23:22 To: Alan Bateman Cc: nio-dev at openjdk.java.net Subject: Re: RFR: JDK-8225507: (zipfs): No space left on device not thrown with ZipFileSystem A few observations from looking at: close method: * Any open InputStreams are closed so there is a possibility of an IOException being thrown * The channel is closed in the same try block that sync() is invoked so there is a potential that if sync() fails the channel is not closed cleanly * the removal of the filesystem for URI based invocations of newFileSystem should/could be moved up after the Zip FS is moved closed * I suspect we should defer the throwing of the IOException if sync() fails until after we finish the rest of close() similar as is done for an IOException occurring while deleting the temp filess sync method: * Looks like if the write of a CEN entry fails the IOException would be thrown where as an issue updating the LOC will just be ignored which I am not sure why that was done Best Lance On Feb 9, 2020, at 12:02 PM, Alan Bateman > wrote: On 05/02/2020 23:34, Lance Andersen wrote: Hi Jaikiran, I think overall the patch is correct but it would be good to get some extra eyes on this. On the surface this looks okay but I think it would be useful if you (or Jai) could write a brief summary about the state of the file system and the underlying channel when the sync fails. I think the zip file system state will be closed (!isOpen) so a later attempt to close it will be a no-op. However, the state of the channel is not clear - is it open or closed? -Alan [cid:image001.gif at 01D5E715.1DDB2BD0] Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 658 bytes Desc: image001.gif URL: From akashche at redhat.com Wed Feb 19 11:42:16 2020 From: akashche at redhat.com (Alex Kashchenko) Date: Wed, 19 Feb 2020 11:42:16 +0000 Subject: [8u] RFR: 8216472: (se) Stack overflow during selection operation leads to crash (win) Message-ID: Please review the backport of JDK-8216472 to 8u: Bug: https://bugs.openjdk.java.net/browse/JDK-8216472 Original review thread: https://mail.openjdk.java.net/pipermail/nio-dev/2019-October/006684.html Original review thread (continuation): https://mail.openjdk.java.net/pipermail/nio-dev/2019-November/006777.html Original change: https://hg.openjdk.java.net/jdk/jdk/rev/6bc29ebe053e 11u review thread: http://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020-January/002417.html 11u change: https://hg.openjdk.java.net/jdk-updates/jdk11u/rev/3f41514eef8a 8u webrev: http://cr.openjdk.java.net/~akasko/jdk8u/8216472/webrev.00/ 8u change is the same as in 11u, it doesn't apply cleanly to 8u because of different imports and minor code differences in java part. Testing: 8u change in this form was included with Red Hat 8u242-windows release and passed usual release testing. -- -Alex From lance.andersen at oracle.com Wed Feb 19 20:06:08 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 19 Feb 2020 15:06:08 -0500 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: References: <51EF8E30-12C0-4CDA-9E2E-79F250375567@oracle.com> <4256fa6d-cd3b-4852-72c6-7c9acbd8b489@gmail.com> <3B2A5184-B592-4F3F-80AF-5FCF8AF7035C@oracle.com> Message-ID: <1AE7AAAE-DA56-44AD-B932-AF7EF9F00BA9@oracle.com> Hi Christoph I think your changes to ZipFileSystem look fine. I am running it through mach5 now. I am still planning to clean up the test a bit and will create a web rev when I do. > On Feb 19, 2020, at 4:55 AM, Langer, Christoph wrote: > > Hi Lance, Jaikiran, > > you?re probably right. Both, the gut feeling that using streams at all here will bring some performance penalties as well as the potential double traversal of the value map speak against the suggestion I made. I was actually not really convinced myself ? however, I was looking for a way to write the coding in a more concise manner (without performance penalties, of course). Understand I did not come up with anything but if we do we can always update it later. > > So, going back to Jaikiran?s latest proposal, I only have a few minor performance tweaks. Please look at this webrev I generated:http://cr.openjdk.java.net/~clanger/webrevs/8211917.0/ > > First, the manifest node should be looked up by the statement ?inodes.get(IndexNode.keyOf(getBytes("/META-INF/MANIFEST.MF")))?. Using ?getInode(getBytes("/META-INF/MANIFEST.MF"))? would incur an unnecessary Object.requireNonNull() check and would also call the ?entryLookup? function that is needed for mapped entries in MR jars. But even in MR jars, the manifest would not be mapped to some other versioned node. > > And then within the loop body I restructured the if statements such that in the case of no manifest (probably the most common case), just one ?if (manifestInode == null)? check is performed and not another ?if (inode == manifestInode)? after the call to inodeIterator.next(). > > So, just minor tweaks but probably still worthwile doing. ?? All good > > Note that I also added a space after ?while? to be consistent with the rest of the file. > > Other than that I?m good with the change. The testcase looks comprising and zipfs tests are passing. Best Lance > > Best regards > Christoph > > > From: Lance Andersen > > Sent: Mittwoch, 19. Februar 2020 02:42 > To: Langer, Christoph > > Cc: Jaikiran Pai >; nio-dev at openjdk.java.net ; core-libs-dev at openjdk.java.net > Subject: Re: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start > > Hi Christoph > > > On Feb 17, 2020, at 11:26 AM, Langer, Christoph > wrote: > > Hi Jaikiran, Lance, > > I'm a bit late to the game. It occurred to me, whether we could maybe use the streams API to process the inodes. Something like this: > > diff --git a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java > --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java > +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java > @@ -56,6 +56,7 @@ > import java.util.jar.Attributes; > import java.util.jar.Manifest; > import java.util.regex.Pattern; > +import java.util.stream.Stream; > import java.util.zip.CRC32; > import java.util.zip.Deflater; > import java.util.zip.DeflaterOutputStream; > @@ -1730,7 +1731,17 @@ > Entry e; > > // write loc > - for (IndexNode inode : inodes.values()) { > + > + // write the manifest inode (if any) first in the loc so that the > + // java.util.jar.JarInputStream can find it, since it expects it to be > + // the first or second entry in the jar > + final IndexNode manifestInode = getInode(getBytes("/META-INF/MANIFEST.MF")); > + final Stream inodeStream = manifestInode == null ? > + inodes.values().stream() : > + Stream.concat(Stream.of(manifestInode), > + inodes.values().stream().filter(node -> !manifestInode.equals(node))); > + > + for (IndexNode inode : (Iterable)inodeStream::iterator) { > if (inode instanceof Entry) { // an updated inode > e = (Entry)inode; > try { > > What do you think? > > I am not sure the use of Streams will help us too much here as in your proposed example I believe you would end out traversing the entire list of IndexNodes twice in the case of a Manifest existing. Once when building the Stream and the other when you are walking through the IndexNodes to write them out. > > Best > Lance > > > I didn't have time yet to have a closer look to the testcase. Will do tomorrow. > > Cheers > Christoph > > > -----Original Message----- > From: nio-dev > On Behalf Of Jaikiran > Pai > Sent: Montag, 17. Februar 2020 03:56 > To: Lance Andersen > > Cc: nio-dev at openjdk.java.net ; core-libs-dev at openjdk.java.net > Subject: Re: RFR: 8211917: (zipfs) Creating or updating a JAR file system > should put the MANIFEST.MF at the start > > Hello Lance, > > On 15/02/20 2:27 am, Lance Andersen wrote: > > Hi Jaikiran, > > I think the changes to ZipFileSystem are OK. > > The test overall is good. I am going to streamline it a bit and > remove the long lines (we try to keep lines to around 80 characters). > > I'll keep that in mind for future changes. Thank you for taking care of > this. > > -Jaikiran > > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From gnu.andrew at redhat.com Thu Feb 20 04:11:06 2020 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Thu, 20 Feb 2020 04:11:06 +0000 Subject: [8u] RFR: 8216472: (se) Stack overflow during selection operation leads to crash (win) In-Reply-To: References: Message-ID: <1b4ac8d6-7163-2bad-8246-9e163e723749@redhat.com> On 19/02/2020 11:42, Alex Kashchenko wrote: > Please review the backport of JDK-8216472 to 8u: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8216472 > > Original review thread: > https://mail.openjdk.java.net/pipermail/nio-dev/2019-October/006684.html > > Original review thread (continuation): > https://mail.openjdk.java.net/pipermail/nio-dev/2019-November/006777.html > > Original change: https://hg.openjdk.java.net/jdk/jdk/rev/6bc29ebe053e > > 11u review thread: > http://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020-January/002417.html > > > 11u change: https://hg.openjdk.java.net/jdk-updates/jdk11u/rev/3f41514eef8a > > 8u webrev: http://cr.openjdk.java.net/~akasko/jdk8u/8216472/webrev.00/ > > 8u change is the same as in 11u, it doesn't apply cleanly to 8u because > of different imports and minor code differences in java part. > > Testing: 8u change in this form was included with Red Hat 8u242-windows > release and passed usual release testing. > This looks good to me. I'd still say you should use your OpenJDK username on this commit, even if you're not technically an author for this particular JDK project. Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: OpenPGP digital signature URL: From christoph.langer at sap.com Thu Feb 20 13:33:04 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 20 Feb 2020 13:33:04 +0000 Subject: RFR 8239556: (zipfs) remove ExistingChannelCloser facility in zipfs implementation Message-ID: Hi, please review this cleanup change to remove the ExistingChannelCloser facility in zipfs. Some technical details about why this existed can be found in this mailing list thread, where Sherman explained its original purpose: https://mail.openjdk.java.net/pipermail/core-libs-dev/2019-April/059839.html. At that time we didn't dare to remove it and deferred to further investigation. Now, when looking at the close() implementation of ZipFileSystem during review of JDK-8225507, I had a closer look to exChannelCloser as well and found that it should definitely be removed. Its original purpose was to keep a backing file and channel for InputStreams that were still open while the sync() method was called to update a zip file on disk. However, in its current state, a zip file is only synced to disk by sync() during close(), at a time when all InputStreams should already be closed. Closing of the streams is done at the beginning of method close(), in line 466ff. Closing of any open InputStream would remove it from the list named "streams". After that, it must be assured that no InputStreams get created, of course. This is accomplished by calling ensureOpen() under readLock, whenever an InputStream shall be created. The first step of close() though is to set the ZipFileSystem to state "closed", so these checks should all fail. I have, however, found one location where this ensureOpen check was missing and added it. jdk/nio/zipfs tests still pass. Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8239556.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8239556 Thanks Christoph -------------- next part -------------- An HTML attachment was scrubbed... URL: From lance.andersen at oracle.com Thu Feb 20 19:00:01 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 20 Feb 2020 14:00:01 -0500 Subject: RFR 8239556: (zipfs) remove ExistingChannelCloser facility in zipfs implementation In-Reply-To: References: Message-ID: Hi Christoph, This looks good and thank you for tackling. We will need to regenerate the web-rev for the Manifest fix after this gets pushed I am running Mach5 tiers 1-3 now and will let you know when it completes > On Feb 20, 2020, at 8:33 AM, Langer, Christoph wrote: > > Hi, > > please review this cleanup change to remove the ExistingChannelCloser facility in zipfs. > > Some technical details about why this existed can be found in this mailing list thread, where Sherman explained its original purpose: https://mail.openjdk.java.net/pipermail/core-libs-dev/2019-April/059839.html. At that time we didn't dare to remove it and deferred to further investigation. > > Now, when looking at the close() implementation of ZipFileSystem during review of JDK-8225507, I had a closer look to exChannelCloser as well and found that it should definitely be removed. Its original purpose was to keep a backing file and channel for InputStreams that were still open while the sync() method was called to update a zip file on disk. However, in its current state, a zip file is only synced to disk by sync() during close(), at a time when all InputStreams should already be closed. > > Closing of the streams is done at the beginning of method close(), in line 466ff. Closing of any open InputStream would remove it from the list named "streams". After that, it must be assured that no InputStreams get created, of course. This is accomplished by calling ensureOpen() under readLock, whenever an InputStream shall be created. The first step of close() though is to set the ZipFileSystem to state "closed", so these checks should all fail. I have, however, found one location where this ensureOpen check was missing and added it. > > jdk/nio/zipfs tests still pass. > > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8239556.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8239556 > > Thanks > Christoph > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From akashche at redhat.com Thu Feb 20 19:13:17 2020 From: akashche at redhat.com (Alex Kashchenko) Date: Thu, 20 Feb 2020 19:13:17 +0000 Subject: [8u] RFR: 8216472: (se) Stack overflow during selection operation leads to crash (win) In-Reply-To: <1b4ac8d6-7163-2bad-8246-9e163e723749@redhat.com> References: <1b4ac8d6-7163-2bad-8246-9e163e723749@redhat.com> Message-ID: Hi Andrew, On 02/20/2020 04:11 AM, Andrew Hughes wrote: > > > On 19/02/2020 11:42, Alex Kashchenko wrote: >> Please review the backport of JDK-8216472 to 8u: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8216472 >> >> Original review thread: >> https://mail.openjdk.java.net/pipermail/nio-dev/2019-October/006684.html >> >> Original review thread (continuation): >> https://mail.openjdk.java.net/pipermail/nio-dev/2019-November/006777.html >> >> Original change: https://hg.openjdk.java.net/jdk/jdk/rev/6bc29ebe053e >> >> 11u review thread: >> http://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020-January/002417.html >> >> >> 11u change: https://hg.openjdk.java.net/jdk-updates/jdk11u/rev/3f41514eef8a >> >> 8u webrev: http://cr.openjdk.java.net/~akasko/jdk8u/8216472/webrev.00/ >> >> 8u change is the same as in 11u, it doesn't apply cleanly to 8u because >> of different imports and minor code differences in java part. >> >> Testing: 8u change in this form was included with Red Hat 8u242-windows >> release and passed usual release testing. >> > > This looks good to me. > > I'd still say you should use your OpenJDK username on this commit, even > if you're not technically an author for this particular JDK project. Thanks for the review, I've added 8u fix request to the issue. -- -Alex From christoph.langer at sap.com Fri Feb 21 08:11:16 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 21 Feb 2020 08:11:16 +0000 Subject: RFR 8239556: (zipfs) remove ExistingChannelCloser facility in zipfs implementation In-Reply-To: References: Message-ID: Hi Lance, thanks for reviewing. Any results from Mach5? I'll push it after I get your ok. The manifest fix will still apply after this - but you wanted to update the webrev anyway with some test updates, right? When both of these fixes are in, I'll look at augmenting Jai's current proposal for JDK-8225507 to overhaul exception handling in close()&sync(). Cheers Christoph From: Lance Andersen Sent: Donnerstag, 20. Februar 2020 20:00 To: Langer, Christoph Cc: nio-dev at openjdk.java.net; core-libs-dev at openjdk.java.net Subject: Re: RFR 8239556: (zipfs) remove ExistingChannelCloser facility in zipfs implementation Hi Christoph, This looks good and thank you for tackling. We will need to regenerate the web-rev for the Manifest fix after this gets pushed I am running Mach5 tiers 1-3 now and will let you know when it completes On Feb 20, 2020, at 8:33 AM, Langer, Christoph > wrote: Hi, please review this cleanup change to remove the ExistingChannelCloser facility in zipfs. Some technical details about why this existed can be found in this mailing list thread, where Sherman explained its original purpose: https://mail.openjdk.java.net/pipermail/core-libs-dev/2019-April/059839.html. At that time we didn't dare to remove it and deferred to further investigation. Now, when looking at the close() implementation of ZipFileSystem during review of JDK-8225507, I had a closer look to exChannelCloser as well and found that it should definitely be removed. Its original purpose was to keep a backing file and channel for InputStreams that were still open while the sync() method was called to update a zip file on disk. However, in its current state, a zip file is only synced to disk by sync() during close(), at a time when all InputStreams should already be closed. Closing of the streams is done at the beginning of method close(), in line 466ff. Closing of any open InputStream would remove it from the list named "streams". After that, it must be assured that no InputStreams get created, of course. This is accomplished by calling ensureOpen() under readLock, whenever an InputStream shall be created. The first step of close() though is to set the ZipFileSystem to state "closed", so these checks should all fail. I have, however, found one location where this ensureOpen check was missing and added it. jdk/nio/zipfs tests still pass. Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8239556.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8239556 Thanks Christoph [cid:image001.gif at 01D5E896.DE3E9740] Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 658 bytes Desc: image001.gif URL: From lance.andersen at oracle.com Fri Feb 21 11:34:02 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 21 Feb 2020 06:34:02 -0500 Subject: RFR 8239556: (zipfs) remove ExistingChannelCloser facility in zipfs implementation In-Reply-To: References: Message-ID: <5F86603B-D962-44A4-B0F9-BF7AD3C3FE14@oracle.com> Morning Christoph, > On Feb 21, 2020, at 3:11 AM, Langer, Christoph wrote: > > Hi Lance, > > thanks for reviewing. Any results from Mach5? I?ll push it after I get your ok. It finished late last night and it ran clean > > The manifest fix will still apply after this ? but you wanted to update the webrev anyway with some test updates, right? Yes, I wanted to clean up the test some. > > When both of these fixes are in, I?ll look at augmenting Jai?s current proposal for JDK-8225507 to overhaul exception handling in close()&sync(). Assume your last webrev on top of this fix is the lucky winner while you take a stab at this clean up :-) Best Lance > > Cheers > Christoph > > > > From: Lance Andersen > > Sent: Donnerstag, 20. Februar 2020 20:00 > To: Langer, Christoph > > Cc: nio-dev at openjdk.java.net ; core-libs-dev at openjdk.java.net > Subject: Re: RFR 8239556: (zipfs) remove ExistingChannelCloser facility in zipfs implementation > > Hi Christoph, > > This looks good and thank you for tackling. We will need to regenerate the web-rev for the Manifest fix after this gets pushed > > I am running Mach5 tiers 1-3 now and will let you know when it completes > > > > > On Feb 20, 2020, at 8:33 AM, Langer, Christoph > wrote: > > Hi, > > please review this cleanup change to remove the ExistingChannelCloser facility in zipfs. > > Some technical details about why this existed can be found in this mailing list thread, where Sherman explained its original purpose: https://mail.openjdk.java.net/pipermail/core-libs-dev/2019-April/059839.html . At that time we didn't dare to remove it and deferred to further investigation. > > Now, when looking at the close() implementation of ZipFileSystem during review of JDK-8225507, I had a closer look to exChannelCloser as well and found that it should definitely be removed. Its original purpose was to keep a backing file and channel for InputStreams that were still open while the sync() method was called to update a zip file on disk. However, in its current state, a zip file is only synced to disk by sync() during close(), at a time when all InputStreams should already be closed. > > Closing of the streams is done at the beginning of method close(), in line 466ff. Closing of any open InputStream would remove it from the list named "streams". After that, it must be assured that no InputStreams get created, of course. This is accomplished by calling ensureOpen() under readLock, whenever an InputStream shall be created. The first step of close() though is to set the ZipFileSystem to state "closed", so these checks should all fail. I have, however, found one location where this ensureOpen check was missing and added it. > > jdk/nio/zipfs tests still pass. > > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8239556.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8239556 > > Thanks > Christoph > > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From christoph.langer at sap.com Fri Feb 21 17:27:57 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 21 Feb 2020 17:27:57 +0000 Subject: RFR 8239556: (zipfs) remove ExistingChannelCloser facility in zipfs implementation In-Reply-To: <5F86603B-D962-44A4-B0F9-BF7AD3C3FE14@oracle.com> References: <5F86603B-D962-44A4-B0F9-BF7AD3C3FE14@oracle.com> Message-ID: Hi Lance, thanks, I?ve pushed it now: http://hg.openjdk.java.net/jdk/jdk/rev/c22b369d40b2 I?ll be out next week, so I?ll probably get to JDK-8225507 the week after. Cheers Christoph From: Lance Andersen Sent: Freitag, 21. Februar 2020 12:34 To: Langer, Christoph Cc: nio-dev at openjdk.java.net; core-libs-dev at openjdk.java.net Subject: Re: RFR 8239556: (zipfs) remove ExistingChannelCloser facility in zipfs implementation Morning Christoph, On Feb 21, 2020, at 3:11 AM, Langer, Christoph > wrote: Hi Lance, thanks for reviewing. Any results from Mach5? I?ll push it after I get your ok. It finished late last night and it ran clean The manifest fix will still apply after this ? but you wanted to update the webrev anyway with some test updates, right? Yes, I wanted to clean up the test some. When both of these fixes are in, I?ll look at augmenting Jai?s current proposal for JDK-8225507 to overhaul exception handling in close()&sync(). Assume your last webrev on top of this fix is the lucky winner while you take a stab at this clean up :-) Best Lance Cheers Christoph From: Lance Andersen > Sent: Donnerstag, 20. Februar 2020 20:00 To: Langer, Christoph > Cc: nio-dev at openjdk.java.net; core-libs-dev at openjdk.java.net Subject: Re: RFR 8239556: (zipfs) remove ExistingChannelCloser facility in zipfs implementation Hi Christoph, This looks good and thank you for tackling. We will need to regenerate the web-rev for the Manifest fix after this gets pushed I am running Mach5 tiers 1-3 now and will let you know when it completes On Feb 20, 2020, at 8:33 AM, Langer, Christoph > wrote: Hi, please review this cleanup change to remove the ExistingChannelCloser facility in zipfs. Some technical details about why this existed can be found in this mailing list thread, where Sherman explained its original purpose: https://mail.openjdk.java.net/pipermail/core-libs-dev/2019-April/059839.html. At that time we didn't dare to remove it and deferred to further investigation. Now, when looking at the close() implementation of ZipFileSystem during review of JDK-8225507, I had a closer look to exChannelCloser as well and found that it should definitely be removed. Its original purpose was to keep a backing file and channel for InputStreams that were still open while the sync() method was called to update a zip file on disk. However, in its current state, a zip file is only synced to disk by sync() during close(), at a time when all InputStreams should already be closed. Closing of the streams is done at the beginning of method close(), in line 466ff. Closing of any open InputStream would remove it from the list named "streams". After that, it must be assured that no InputStreams get created, of course. This is accomplished by calling ensureOpen() under readLock, whenever an InputStream shall be created. The first step of close() though is to set the ZipFileSystem to state "closed", so these checks should all fail. I have, however, found one location where this ensureOpen check was missing and added it. jdk/nio/zipfs tests still pass. Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8239556.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8239556 Thanks Christoph Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com [cid:image001.gif at 01D5E8E4.A340D420] Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 658 bytes Desc: image001.gif URL: From fweimer at redhat.com Thu Feb 27 12:58:04 2020 From: fweimer at redhat.com (Florian Weimer) Date: Thu, 27 Feb 2020 13:58:04 +0100 Subject: 8219014: (bf) Add absolute bulk put methods which accept a source Buffer In-Reply-To: <6B7BB465-EA1F-4763-8E2D-927B71EADFC6@oracle.com> (Paul Sandoz's message of "Wed, 12 Feb 2020 14:43:13 -0800") References: <5F6F4823-8B99-48DB-9B7A-44CF2A3DADBD@oracle.com> <45CBA103-32F0-46FE-8444-35828A990733@oracle.com> <87wo8ti01g.fsf@oldenburg2.str.redhat.com> <824238F5-D399-4088-A1AA-A9AC8321577F@oracle.com> <87eev1hwxa.fsf@oldenburg2.str.redhat.com> <8736bhhvst.fsf@oldenburg2.str.redhat.com> <6B7BB465-EA1F-4763-8E2D-927B71EADFC6@oracle.com> Message-ID: <87imjsxk4j.fsf@oldenburg2.str.redhat.com> * Paul Sandoz: > I?ll let Brian decide on that from the perspective of Buffers, given > IIRC it does not mention anything about overlapping regions. > > However, the incubating memory segment API does: > > https://hg.openjdk.java.net/jdk/jdk/file/tip/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryAddress.java#l98 > > We can capture something here first. My preference is an api note. > Care to propose something? The existing text: | Perform bulk copy from source address to target address. More Probably: Performs a bulk copy | specifically, the bytes at addresses {@code src} through | {@code src.addOffset(bytes - 1)} are copied into addresses | {@code dst} through {@code dst.addOffset(bytes - 1)}. | If the source and address ranges overlap, then the copying is Probably: source and target address ranges | performed as if the bytes at addresses {@code src} | through {@code src.addOffset(bytes - 1)} were first copied into | a temporary segment with size {@code bytes}, and then the contents | of the temporary segment were copied into the bytes at addresses | {@code dst} through {@code dst.addOffset(bytes - 1)}. Maybe add this: | If the source and target address ranges do not overlap, but refer to | the same underlying backing storage using different addresses, the | contents of the target address range is unspecified after the copy | operation. The intent is to make this a bounded error. You may get random data, but the operation itself is not erroneous. Thanks, Florian From lance.andersen at oracle.com Thu Feb 27 21:11:45 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 27 Feb 2020 16:11:45 -0500 Subject: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start In-Reply-To: <1AE7AAAE-DA56-44AD-B932-AF7EF9F00BA9@oracle.com> References: <51EF8E30-12C0-4CDA-9E2E-79F250375567@oracle.com> <4256fa6d-cd3b-4852-72c6-7c9acbd8b489@gmail.com> <3B2A5184-B592-4F3F-80AF-5FCF8AF7035C@oracle.com> <1AE7AAAE-DA56-44AD-B932-AF7EF9F00BA9@oracle.com> Message-ID: <02FEC2C4-365E-4EB1-BA4F-37D0C0CD13DD@oracle.com> Hi Christoph, I have cleaned up, re-vamped and added more coverage to the test. As part of this I started to lay the foundation for removing some of the duplicate code as part of continued clean up and enhanced coverage going forward The revised webrev can be found at: http://cr.openjdk.java.net/~lancea/8211917/webrev.00/index.html Best Lance > On Feb 19, 2020, at 3:06 PM, Lance Andersen > wrote: > > Hi Christoph > > I think your changes to ZipFileSystem look fine. I am running it through mach5 now. I am still planning to clean up the test a bit and will create a web rev when I do. > > > >> On Feb 19, 2020, at 4:55 AM, Langer, Christoph > wrote: >> >> Hi Lance, Jaikiran, >> >> you?re probably right. Both, the gut feeling that using streams at all here will bring some performance penalties as well as the potential double traversal of the value map speak against the suggestion I made. I was actually not really convinced myself ? however, I was looking for a way to write the coding in a more concise manner (without performance penalties, of course). > Understand I did not come up with anything but if we do we can always update it later. >> >> So, going back to Jaikiran?s latest proposal, I only have a few minor performance tweaks. Please look at this webrev I generated:http://cr.openjdk.java.net/~clanger/webrevs/8211917.0/ >> >> First, the manifest node should be looked up by the statement ?inodes.get(IndexNode.keyOf(getBytes("/META-INF/MANIFEST.MF")))?. Using ?getInode(getBytes("/META-INF/MANIFEST.MF"))? would incur an unnecessary Object.requireNonNull() check and would also call the ?entryLookup? function that is needed for mapped entries in MR jars. But even in MR jars, the manifest would not be mapped to some other versioned node. >> >> And then within the loop body I restructured the if statements such that in the case of no manifest (probably the most common case), just one ?if (manifestInode == null)? check is performed and not another ?if (inode == manifestInode)? after the call to inodeIterator.next(). >> >> So, just minor tweaks but probably still worthwile doing. ?? > > All good >> >> Note that I also added a space after ?while? to be consistent with the rest of the file. >> >> Other than that I?m good with the change. The testcase looks comprising and zipfs tests are passing. > > Best > Lance >> >> Best regards >> Christoph >> >> >> From: Lance Andersen > >> Sent: Mittwoch, 19. Februar 2020 02:42 >> To: Langer, Christoph > >> Cc: Jaikiran Pai >; nio-dev at openjdk.java.net ; core-libs-dev at openjdk.java.net >> Subject: Re: RFR: 8211917: (zipfs) Creating or updating a JAR file system should put the MANIFEST.MF at the start >> >> Hi Christoph >> >> >> On Feb 17, 2020, at 11:26 AM, Langer, Christoph > wrote: >> >> Hi Jaikiran, Lance, >> >> I'm a bit late to the game. It occurred to me, whether we could maybe use the streams API to process the inodes. Something like this: >> >> diff --git a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java >> --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java >> +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java >> @@ -56,6 +56,7 @@ >> import java.util.jar.Attributes; >> import java.util.jar.Manifest; >> import java.util.regex.Pattern; >> +import java.util.stream.Stream; >> import java.util.zip.CRC32; >> import java.util.zip.Deflater; >> import java.util.zip.DeflaterOutputStream; >> @@ -1730,7 +1731,17 @@ >> Entry e; >> >> // write loc >> - for (IndexNode inode : inodes.values()) { >> + >> + // write the manifest inode (if any) first in the loc so that the >> + // java.util.jar.JarInputStream can find it, since it expects it to be >> + // the first or second entry in the jar >> + final IndexNode manifestInode = getInode(getBytes("/META-INF/MANIFEST.MF")); >> + final Stream inodeStream = manifestInode == null ? >> + inodes.values().stream() : >> + Stream.concat(Stream.of(manifestInode), >> + inodes.values().stream().filter(node -> !manifestInode.equals(node))); >> + >> + for (IndexNode inode : (Iterable)inodeStream::iterator) { >> if (inode instanceof Entry) { // an updated inode >> e = (Entry)inode; >> try { >> >> What do you think? >> >> I am not sure the use of Streams will help us too much here as in your proposed example I believe you would end out traversing the entire list of IndexNodes twice in the case of a Manifest existing. Once when building the Stream and the other when you are walking through the IndexNodes to write them out. >> >> Best >> Lance >> >> >> I didn't have time yet to have a closer look to the testcase. Will do tomorrow. >> >> Cheers >> Christoph >> >> >> -----Original Message----- >> From: nio-dev > On Behalf Of Jaikiran >> Pai >> Sent: Montag, 17. Februar 2020 03:56 >> To: Lance Andersen > >> Cc: nio-dev at openjdk.java.net ; core-libs-dev at openjdk.java.net >> Subject: Re: RFR: 8211917: (zipfs) Creating or updating a JAR file system >> should put the MANIFEST.MF at the start >> >> Hello Lance, >> >> On 15/02/20 2:27 am, Lance Andersen wrote: >> >> Hi Jaikiran, >> >> I think the changes to ZipFileSystem are OK. >> >> The test overall is good. I am going to streamline it a bit and >> remove the long lines (we try to keep lines to around 80 characters). >> >> I'll keep that in mind for future changes. Thank you for taking care of >> this. >> >> -Jaikiran >> >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> >> > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From vladimir.a.ivanov at intel.com Fri Feb 28 18:40:29 2020 From: vladimir.a.ivanov at intel.com (Ivanov, Vladimir A) Date: Fri, 28 Feb 2020 18:40:29 +0000 Subject: Status of the RDMA code Message-ID: Hi, we finished with patch for the RDMA: JEP https://openjdk.java.net/jeps/337; Enhancement https://bugs.openjdk.java.net/browse/JDK-8195160 The webrev against default HEAD for this patch available as http://cr.openjdk.java.net/~sviswanathan/Vladimir/RDMA/webrev.00/ For latest binaries and configuration all JDK tests for the rsockets passed for 1000 iterations. The tests run results for one iteration looks as: ===================================================================== [rsocket-branch]# time ./run_r_tests.sh openjdk version "15-internal" 2020-09-15 OpenJDK Runtime Environment (build 15-internal+0) OpenJDK 64-Bit Server VM (build 15-internal+0, mixed mode, sharing) Directory "JTwork" not found: creating Directory "JTreport" not found: creating Passed: jdk/net/RdmaSockets/rsocket/Selector/BasicAccept.java Passed: jdk/net/RdmaSockets/rsocket/Selector/BasicConnect.java Passed: jdk/net/RdmaSockets/rsocket/Selector/Connect.java Passed: jdk/net/RdmaSockets/rsocket/Selector/SelectorTest.java Passed: jdk/net/RdmaSockets/rsocket/ServerSocketChannel/Basic.java Passed: jdk/net/RdmaSockets/rsocket/ServerSocketChannel/SocketOptionTests.java Passed: jdk/net/RdmaSockets/rsocket/ServerSocketChannel/SSCConfigureBlocking.java Passed: jdk/net/RdmaSockets/rsocket/Socket/BasicSocketTest.java Passed: jdk/net/RdmaSockets/rsocket/Socket/GetLocalAddress.java Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/BasicSocketChannelTest.java Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/CloseDuringWrite.java Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/Connect.java Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/FinishConnect.java Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/IOExchanges.java Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/SCConfigureBlocking.java Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/SocketOptionTests.java Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/Stream.java Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/VectorIO.java Passed: jdk/net/RdmaSockets/rsocket/SocketOption/OptionsTest.java Passed: jdk/net/RdmaSockets/rsocket/SocketOption/UnsupportedOptionsTest.java Passed: jdk/net/RdmaSockets/rsocket/Exceptions.java Passed: jdk/net/RdmaSockets/rsocket/NullBind.java Passed: jdk/net/RdmaSockets/rsocket/RdmaSelectorProvider.java Passed: jdk/net/RdmaSockets/rsocket/RsocketTest.java Test results: passed: 24 Report written to /home/workload/rsocket-branch/JTreport/html/report.html Results written to /home/workload/rsocket-branch/JTwork real 4m8.510s user 1m36.642s sys 0m9.726s ===================================================================== Note, this result includes all 24 test cases for the IOExchanges.java test: [rsocket-branch]# cat JTwork/jdk/net/RdmaSockets/rsocket/SocketChannel/IOExchanges.jtr |grep Total Total tests run: 24, Failures: 0, Skips: 0 Used versions are: JDK: sandbox/ rsocket-branch (https://hg.openjdk.java.net/jdk/sandbox/shortlog/af6c77705bc3) RDMA libs: rdma-core-27.0 + patch https://github.com/linux-rdma/rdma-core/pull/684 + patch https://github.com/linux-rdma/rdma-core/pull/707 Both these patches were merged to tip. RDMA driver: # modinfo irdma filename: /lib/modules/4.18.0-80.el8.x86_64/updates/drivers/infiniband/hw/irdma/irdma.ko version: 0.15.1 NICs driver (Columbiaville NIC): # modinfo ice filename: /lib/modules/4.18.0-80.el8.x86_64/updates/drivers/net/ethernet/intel/ice/ice.ko firmware: intel/ice/ddp/ice.pkg version: 0.11.2_rc36 OS: NAME="Red Hat Enterprise Linux" VERSION="8.0 (Ootpa)" Thanks, Vladimir -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Sat Feb 29 10:48:31 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sat, 29 Feb 2020 10:48:31 +0000 Subject: Status of the RDMA code In-Reply-To: References: Message-ID: <06BAF5A0-6238-40C2-BA3B-539AC815C0D2@oracle.com> Thank you for sending this update Ivanov. Looks like very good progress, and nice to see the PR?s have been accepted and merged into rdma-core. -Chris. > On 28 Feb 2020, at 18:40, Ivanov, Vladimir A wrote: > > Hi, > we finished with patch for the RDMA: JEP https://openjdk.java.net/jeps/337 ; Enhancementhttps://bugs.openjdk.java.net/browse/JDK-8195160 > > The webrev against default HEAD for this patch available as http://cr.openjdk.java.net/~sviswanathan/Vladimir/RDMA/webrev.00/ > > For latest binaries and configuration all JDK tests for the rsockets passed for 1000 iterations. > The tests run results for one iteration looks as: > ===================================================================== > [rsocket-branch]# time ./run_r_tests.sh > openjdk version "15-internal" 2020-09-15 > OpenJDK Runtime Environment (build 15-internal+0) > OpenJDK 64-Bit Server VM (build 15-internal+0, mixed mode, sharing) > Directory "JTwork" not found: creating > Directory "JTreport" not found: creating > Passed: jdk/net/RdmaSockets/rsocket/Selector/BasicAccept.java > Passed: jdk/net/RdmaSockets/rsocket/Selector/BasicConnect.java > Passed: jdk/net/RdmaSockets/rsocket/Selector/Connect.java > Passed: jdk/net/RdmaSockets/rsocket/Selector/SelectorTest.java > Passed: jdk/net/RdmaSockets/rsocket/ServerSocketChannel/Basic.java > Passed: jdk/net/RdmaSockets/rsocket/ServerSocketChannel/SocketOptionTests.java > Passed: jdk/net/RdmaSockets/rsocket/ServerSocketChannel/SSCConfigureBlocking.java > Passed: jdk/net/RdmaSockets/rsocket/Socket/BasicSocketTest.java > Passed: jdk/net/RdmaSockets/rsocket/Socket/GetLocalAddress.java > Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/BasicSocketChannelTest.java > Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/CloseDuringWrite.java > Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/Connect.java > Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/FinishConnect.java > Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/IOExchanges.java > Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/SCConfigureBlocking.java > Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/SocketOptionTests.java > Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/Stream.java > Passed: jdk/net/RdmaSockets/rsocket/SocketChannel/VectorIO.java > Passed: jdk/net/RdmaSockets/rsocket/SocketOption/OptionsTest.java > Passed: jdk/net/RdmaSockets/rsocket/SocketOption/UnsupportedOptionsTest.java > Passed: jdk/net/RdmaSockets/rsocket/Exceptions.java > Passed: jdk/net/RdmaSockets/rsocket/NullBind.java > Passed: jdk/net/RdmaSockets/rsocket/RdmaSelectorProvider.java > Passed: jdk/net/RdmaSockets/rsocket/RsocketTest.java > Test results: passed: 24 > Report written to /home/workload/rsocket-branch/JTreport/html/report.html > Results written to /home/workload/rsocket-branch/JTwork > > real 4m8.510s > user 1m36.642s > sys 0m9.726s > ===================================================================== > > Note, this result includes all 24 test cases for the IOExchanges.java test: > [rsocket-branch]# cat JTwork/jdk/net/RdmaSockets/rsocket/SocketChannel/IOExchanges.jtr |grep Total > Total tests run: 24, Failures: 0, Skips: 0 > > Used versions are: > JDK: sandbox/ rsocket-branch (https://hg.openjdk.java.net/jdk/sandbox/shortlog/af6c77705bc3 ) > RDMA libs: rdma-core-27.0 + patch https://github.com/linux-rdma/rdma-core/pull/684 + patch https://github.com/linux-rdma/rdma-core/pull/707 > Both these patches were merged to tip. > > RDMA driver: > # modinfo irdma > filename: /lib/modules/4.18.0-80.el8.x86_64/updates/drivers/infiniband/hw/irdma/irdma.ko > version: 0.15.1 > > NICs driver (Columbiaville NIC): > # modinfo ice > filename: /lib/modules/4.18.0-80.el8.x86_64/updates/drivers/net/ethernet/intel/ice/ice.ko > firmware: intel/ice/ddp/ice.pkg > version: 0.11.2_rc36 > > OS: > NAME="Red Hat Enterprise Linux" > VERSION="8.0 (Ootpa)" > > Thanks, Vladimir -------------- next part -------------- An HTML attachment was scrubbed... URL: