From Alan.Bateman at oracle.com Tue Jan 3 08:51:54 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 3 Jan 2017 08:51:54 +0000 Subject: JDK 9 RFR of 8171452: (ch) linux io_util_md: Operation not supported exception after 8168628 In-Reply-To: <5C68B212-D633-464C-A206-9787D6CC579C@oracle.com> References: <31751D55-17F9-465B-8657-2A7C39114E1B@oracle.com> <5C68B212-D633-464C-A206-9787D6CC579C@oracle.com> Message-ID: On 20/12/2016 20:16, Brian Burkhalter wrote: > : > > As noted below, I wonder whether the code should not just fall back > for all unsuccessful fallocate64() calls. The fallback is to the same > behavior which was there prior to the fix for JDK-8168628 so this > seems safe. We?ll see whether a JDK 9 Reviewer has any comment on this > point. > Falling back when the error is ENOSYS or EOPNOTSUPP seems right, I don't see a reasonable to fallback for other errors. As to whether it's worth handling ENOSYS then I'm not sure, it would likely be very old kernels as Martin noted in his reply. -Alan From brian.burkhalter at oracle.com Tue Jan 3 19:11:19 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 3 Jan 2017 11:11:19 -0800 Subject: JDK 9 RFR of 8171452: (ch) linux io_util_md: Operation not supported exception after 8168628 In-Reply-To: References: <31751D55-17F9-465B-8657-2A7C39114E1B@oracle.com> <5C68B212-D633-464C-A206-9787D6CC579C@oracle.com> Message-ID: <56FEF2A4-6B4D-4450-AC25-D337EEF2E32E@oracle.com> On Jan 3, 2017, at 12:51 AM, Alan Bateman wrote: > Falling back when the error is ENOSYS or EOPNOTSUPP seems right, I don't see a reasonable to fallback for other errors. As to whether it's worth handling ENOSYS then I'm not sure, it would likely be very old kernels as Martin noted in his reply. Is this a +1 on the patch as-is or should I add ENOSYS? Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From joe.darcy at oracle.com Wed Jan 4 00:27:57 2017 From: joe.darcy at oracle.com (joe darcy) Date: Tue, 3 Jan 2017 16:27:57 -0800 Subject: JDK 9 RFR of JDK-8172200: Mark StressLoopback.java as intermittently failing Message-ID: <6d696e1a-e728-cbbe-a275-9f7b218e6313@oracle.com> Hello, The test java/nio/channels/AsynchronousSocketChannel/StressLoopback.java intermittently fails on Solaris x64 on one of our testing systems. The test should be marked accordingly. Please review the patch below which does this. Thanks, -Joe diff -r d27bab22ff62 test/java/nio/channels/AsynchronousSocketChannel/StressLoopback.java --- a/test/java/nio/channels/AsynchronousSocketChannel/StressLoopback.java Mon Jan 02 22:45:34 2017 +0100 +++ b/test/java/nio/channels/AsynchronousSocketChannel/StressLoopback.java Tue Jan 03 16:26:21 2017 -0800 @@ -26,7 +26,7 @@ * @summary Stress test connections through the loopback interface * @run main StressLoopback * @run main/othervm -Djdk.net.useFastTcpLoopback StressLoopback - * @key randomness + * @key randomness intermittent */ import java.nio.ByteBuffer; From lance.andersen at oracle.com Wed Jan 4 00:57:09 2017 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 3 Jan 2017 19:57:09 -0500 Subject: JDK 9 RFR of JDK-8172200: Mark StressLoopback.java as intermittently failing In-Reply-To: <6d696e1a-e728-cbbe-a275-9f7b218e6313@oracle.com> References: <6d696e1a-e728-cbbe-a275-9f7b218e6313@oracle.com> Message-ID: +1 > On Jan 3, 2017, at 7:27 PM, joe darcy wrote: > > Hello, > > The test > > java/nio/channels/AsynchronousSocketChannel/StressLoopback.java > > intermittently fails on Solaris x64 on one of our testing systems. The test should be marked accordingly. Please review the patch below which does this. > > Thanks, > > -Joe > > diff -r d27bab22ff62 test/java/nio/channels/AsynchronousSocketChannel/StressLoopback.java > --- a/test/java/nio/channels/AsynchronousSocketChannel/StressLoopback.java Mon Jan 02 22:45:34 2017 +0100 > +++ b/test/java/nio/channels/AsynchronousSocketChannel/StressLoopback.java Tue Jan 03 16:26:21 2017 -0800 > @@ -26,7 +26,7 @@ > * @summary Stress test connections through the loopback interface > * @run main StressLoopback > * @run main/othervm -Djdk.net.useFastTcpLoopback StressLoopback > - * @key randomness > + * @key randomness intermittent > */ > > import java.nio.ByteBuffer; > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From brian.burkhalter at oracle.com Wed Jan 4 01:50:02 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 3 Jan 2017 17:50:02 -0800 Subject: JDK 9 RFR of JDK-8172200: Mark StressLoopback.java as intermittently failing In-Reply-To: <6d696e1a-e728-cbbe-a275-9f7b218e6313@oracle.com> References: <6d696e1a-e728-cbbe-a275-9f7b218e6313@oracle.com> Message-ID: Hi Joe, Looks fine. Brian On Jan 3, 2017, at 4:27 PM, joe darcy wrote: > java/nio/channels/AsynchronousSocketChannel/StressLoopback.java > > intermittently fails on Solaris x64 on one of our testing systems. The test should be marked accordingly. Please review the patch below which does this. > > --- a/test/java/nio/channels/AsynchronousSocketChannel/StressLoopback.java Mon Jan 02 22:45:34 2017 +0100 > +++ b/test/java/nio/channels/AsynchronousSocketChannel/StressLoopback.java Tue Jan 03 16:26:21 2017 -0800 > @@ -26,7 +26,7 @@ > * @summary Stress test connections through the loopback interface > * @run main StressLoopback > * @run main/othervm -Djdk.net.useFastTcpLoopback StressLoopback > - * @key randomness > + * @key randomness intermittent > */ From Alan.Bateman at oracle.com Wed Jan 4 07:08:50 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 4 Jan 2017 07:08:50 +0000 Subject: JDK 9 RFR of 8171452: (ch) linux io_util_md: Operation not supported exception after 8168628 In-Reply-To: <56FEF2A4-6B4D-4450-AC25-D337EEF2E32E@oracle.com> References: <31751D55-17F9-465B-8657-2A7C39114E1B@oracle.com> <5C68B212-D633-464C-A206-9787D6CC579C@oracle.com> <56FEF2A4-6B4D-4450-AC25-D337EEF2E32E@oracle.com> Message-ID: <020612fc-086d-1379-a5f0-64e50b14df73@oracle.com> On 03/01/2017 19:11, Brian Burkhalter wrote: > > On Jan 3, 2017, at 12:51 AM, Alan Bateman > wrote: > >> Falling back when the error is ENOSYS or EOPNOTSUPP seems right, I >> don't see a reasonable to fallback for other errors. As to whether >> it's worth handling ENOSYS then I'm not sure, it would likely be very >> old kernels as Martin noted in his reply. > > Is this a +1 on the patch as-is or should I add ENOSYS? > The patch is okay as is, and no objection to also checking for ENOSYS if you want. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jan 4 19:44:58 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 4 Jan 2017 11:44:58 -0800 Subject: JDK 9 RFR of 8171452: (ch) linux io_util_md: Operation not supported exception after 8168628 In-Reply-To: <020612fc-086d-1379-a5f0-64e50b14df73@oracle.com> References: <31751D55-17F9-465B-8657-2A7C39114E1B@oracle.com> <5C68B212-D633-464C-A206-9787D6CC579C@oracle.com> <56FEF2A4-6B4D-4450-AC25-D337EEF2E32E@oracle.com> <020612fc-086d-1379-a5f0-64e50b14df73@oracle.com> Message-ID: <6898A5B3-547E-456E-9E4D-738EF24B2838@oracle.com> On Jan 3, 2017, at 11:08 PM, Alan Bateman wrote: >> Is this a +1 on the patch as-is or should I add ENOSYS? >> > The patch is okay as is, and no objection to also checking for ENOSYS if you want. I changed it to check for ENOSYS as well. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jan 4 19:47:20 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 4 Jan 2017 11:47:20 -0800 Subject: JDK 9 RFR of 8171452: (ch) linux io_util_md: Operation not supported exception after 8168628 In-Reply-To: <6898A5B3-547E-456E-9E4D-738EF24B2838@oracle.com> References: <31751D55-17F9-465B-8657-2A7C39114E1B@oracle.com> <5C68B212-D633-464C-A206-9787D6CC579C@oracle.com> <56FEF2A4-6B4D-4450-AC25-D337EEF2E32E@oracle.com> <020612fc-086d-1379-a5f0-64e50b14df73@oracle.com> <6898A5B3-547E-456E-9E4D-738EF24B2838@oracle.com> Message-ID: On Jan 4, 2017, at 11:44 AM, Brian Burkhalter wrote: > On Jan 3, 2017, at 11:08 PM, Alan Bateman wrote: > >>> Is this a +1 on the patch as-is or should I add ENOSYS? >>> >> The patch is okay as is, and no objection to also checking for ENOSYS if you want. > > I changed it to check for ENOSYS as well. Forgot the link: http://cr.openjdk.java.net/~bpb/8171452/webrev.01/ Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Jan 5 13:03:25 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 5 Jan 2017 13:03:25 +0000 Subject: JDK 9 RFR of 8171452: (ch) linux io_util_md: Operation not supported exception after 8168628 In-Reply-To: References: <31751D55-17F9-465B-8657-2A7C39114E1B@oracle.com> <5C68B212-D633-464C-A206-9787D6CC579C@oracle.com> <56FEF2A4-6B4D-4450-AC25-D337EEF2E32E@oracle.com> <020612fc-086d-1379-a5f0-64e50b14df73@oracle.com> <6898A5B3-547E-456E-9E4D-738EF24B2838@oracle.com> Message-ID: <606ee1d9-bd8b-390d-94b5-28dabd8b7cf8@oracle.com> On 04/01/2017 19:47, Brian Burkhalter wrote: > > On Jan 4, 2017, at 11:44 AM, Brian Burkhalter > > wrote: > >> On Jan 3, 2017, at 11:08 PM, Alan Bateman > > wrote: >> >>>> Is this a +1 on the patch as-is or should I add ENOSYS? >>>> >>> The patch is okay as is, and no objection to also checking for >>> ENOSYS if you want. >> >> I changed it to check for ENOSYS as well. > > Forgot the link: http://cr.openjdk.java.net/~bpb/8171452/webrev.01/ > > Looks good to me. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Jan 5 22:40:28 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 5 Jan 2017 14:40:28 -0800 Subject: JDK 9 RFR of 8171452: (ch) linux io_util_md: Operation not supported exception after 8168628 In-Reply-To: <5C68B212-D633-464C-A206-9787D6CC579C@oracle.com> References: <31751D55-17F9-465B-8657-2A7C39114E1B@oracle.com> <5C68B212-D633-464C-A206-9787D6CC579C@oracle.com> Message-ID: <523132EF-CAB0-4A74-ADD9-1BD8D2A62E35@oracle.com> Here?s an alternative version of the patch which always falls back to ftruncate64() when fallocate64() returns non-zero: http://cr.openjdk.java.net/~bpb/8171452/webrev.01/ Thanks, Brian On Dec 20, 2016, at 12:16 PM, Brian Burkhalter wrote: > As noted below, I wonder whether the code should not just fall back for all unsuccessful fallocate64() calls. The fallback is to the same behavior which was there prior to the fix for JDK-8168628 so this seems safe. We?ll see whether a JDK 9 Reviewer has any comment on this point. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Jan 5 22:51:43 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 5 Jan 2017 14:51:43 -0800 Subject: JDK 9 RFR of 8171452: (ch) linux io_util_md: Operation not supported exception after 8168628 In-Reply-To: <523132EF-CAB0-4A74-ADD9-1BD8D2A62E35@oracle.com> References: <31751D55-17F9-465B-8657-2A7C39114E1B@oracle.com> <5C68B212-D633-464C-A206-9787D6CC579C@oracle.com> <523132EF-CAB0-4A74-ADD9-1BD8D2A62E35@oracle.com> Message-ID: <9537C665-FB33-4FD4-88C2-D05E1700657F@oracle.com> Oops - please ignore this: it was a draft intended for deletion but inadvertently sent. The fix for this has already been pushed. On Jan 5, 2017, at 2:40 PM, Brian Burkhalter wrote: > Here?s an alternative version of the patch which always falls back to ftruncate64() when fallocate64() returns non-zero: > > http://cr.openjdk.java.net/~bpb/8171452/webrev.01/ > > Thanks, > > Brian > > On Dec 20, 2016, at 12:16 PM, Brian Burkhalter wrote: > >> As noted below, I wonder whether the code should not just fall back for all unsuccessful fallocate64() calls. The fallback is to the same behavior which was there prior to the fix for JDK-8168628 so this seems safe. We?ll see whether a JDK 9 Reviewer has any comment on this point. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Jan 6 00:54:43 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 5 Jan 2017 16:54:43 -0800 Subject: JDK 9 RFR of 8152272: Unable to create temporary file using createTempFile method if System.getProperty("file.separator") is used Message-ID: Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8152272 Patch: http://cr.openjdk.java.net/~bpb/8152272/webrev.00/ Normalize the file name component of the temporary file path. This at least makes the behavior of java.io.File.createTempFile() consistent with that of java.nio.file.Files.createTempFile(). Or would it be better simply to continue to throw the IOException in this situation in which case the bug may be resolved as ?Not an Issue? or ?Wont Fix?? Thanks, Brian From jness at proofpoint.com Mon Jan 9 17:16:32 2017 From: jness at proofpoint.com (Jeremiah Ness) Date: Mon, 9 Jan 2017 17:16:32 +0000 Subject: all EventHandlerTasks in EPollPort waiting on queue Message-ID: Using jre8u112 on CentOs7. I have an application which uses AsynchronousChannelGroup.withFixedThreadPool with 5 threads. When the application in under high load creating 1000s of AsynchronousSocketChannels per second, simultaneously closing many AsynchronousSocketChannels, on occasion all of the EventHandlerTasks in EPollPort become stuck waiting for events on the EPollPort.queue. All the threads have the following stack: "CompletionThread-5" #33 daemon prio=5 os_prio=0 tid=0x00007f0a2829f800 nid=0x5c41 waiting on condition [0x00007f0a11ee1000] ?? java.lang.Thread.State: WAITING (parking) ????????at sun.misc.Unsafe.park(Native Method) ????????- parking to wait for??<0x00000006c4fe8868> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) ????????at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) ????????at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) ????????at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403) ????????at sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:262) ????????at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) ????????at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ????????at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) They all appear to be waiting for the queue to be filled (calling BlockingQueue.take) however there are no threads calling the native method epollWait to get more IO events. This condition persists until the application is restarted. By examining the EPollPort class I have the understanding that one of the threads should be polling. It this correct? By examining the EPollPort.EventHandlerTask.poll method I am wondering if there is a code path that would allow all threads to be waiting on the queue. In particular is the following possible within EPollPort.EventHandlerTask.poll: 1. The native method epollWait returns 512 events. 2. Before the fdToChannelLock.readLock is acquired the channel associated with ?? the 512th event is closed. 3. The fixed size EPollPort.queue is filled to size 511. 4. The 512th event is processed, however because it has been closed it is no ?? longer in the fdToChannel map 5. The thread loops around the for(;;) loop and calls epollWait again 6. epollWait returns 2 more events 7. The fixed size queue is filled to its maximum capacity of 512. 8. The finally queue.offer(NEED_TO_POLL) call fails because the queue is full. If this occurs would all the EventHandlerTasks then eventually be stuck as per the stack trace above? Thanks, Jeremiah Ness From Roger.Riggs at Oracle.com Tue Jan 10 18:23:53 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 10 Jan 2017 13:23:53 -0500 Subject: JDK 9 RFR of 8152272: Unable to create temporary file using createTempFile method if System.getProperty("file.separator") is used In-Reply-To: References: Message-ID: <525c9576-8359-00df-976a-7f1cdcd90d57@Oracle.com> Hi Brian, Looks fine. Roger On 1/5/2017 7:54 PM, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8152272 > Patch: http://cr.openjdk.java.net/~bpb/8152272/webrev.00/ > > Normalize the file name component of the temporary file path. This at least makes the behavior of java.io.File.createTempFile() consistent with that of java.nio.file.Files.createTempFile(). Or would it be better simply to continue to throw the IOException in this situation in which case the bug may be resolved as ?Not an Issue? or ?Wont Fix?? > > Thanks, > > Brian From jness at proofpoint.com Wed Jan 11 22:11:58 2017 From: jness at proofpoint.com (Jeremiah Ness) Date: Wed, 11 Jan 2017 22:11:58 +0000 Subject: all EventHandlerTasks in EPollPort waiting on queue In-Reply-To: References: Message-ID: <9BD02A61-33A6-42A9-8B80-FC73CD6312BB@proofpoint.com> (Sorry is this is the wrong place to discuss this matter. If you know of a better place, please let me know.) I suspect there is a bug in EPollPort.java (and KQueuePort.java) and am looking for a more informed opinion. Please reference the source code below from src//solaris/classes/sun/nio/ch/EPollPort.java. On line 244 below the poll() method attempts to insert the NEED_TO_POLL event into the queue. The queue is a fixed size ArrayBlockingQueue of size MAX_EPOLL_EVENTS. On line 244, if the queue is full, then the event silently fails to be inserted. The NEED_TO_POLL event is critical for the operation of the EPollPort as it is the event which signals one of the threads to poll again. I believe this is what is causing all of my application?s completion threads to get stuck. The queue can become full if: - the inner loop (lines 203 -> 235) is processing MAX_EPOLL_EVENTS events (512 events) - the last inner loop iteration gets a null channel on line 223 - in this case we loop around to line 193 instead of returning from the method - we get 2 more events when calling epollWait again on line 194 - these events fill the queue to its maximum size - we return from the method however fail to insert NEED_TO_POLL in the queue Am I perhaps missing something? Thanks for your thoughts. 191 private Event poll() throws IOException { 192 try { 193 for (;;) { 194 int n = epollWait(epfd, address, MAX_EPOLL_EVENTS); 195 /* 196 * 'n' events have been read. Here we map them to their 197 * corresponding channel in batch and queue n-1 so that 198 * they can be handled by other handler threads. The last 199 * event is handled by this thread (and so is not queued). 200 */ 201 fdToChannelLock.readLock().lock(); 202 try { 203 while (n-- > 0) { 204 long eventAddress = getEvent(address, n); 205 int fd = getDescriptor(eventAddress); 206 207 // wakeup 208 if (fd == sp[0]) { 209 if (wakeupCount.decrementAndGet() == 0) { 210 // no more wakeups so drain pipe 211 drain1(sp[0]); 212 } 213 214 // queue special event if there are more events 215 // to handle. 216 if (n > 0) { 217 queue.offer(EXECUTE_TASK_OR_SHUTDOWN); 218 continue; 219 } 220 return EXECUTE_TASK_OR_SHUTDOWN; 221 } 222 223 PollableChannel channel = fdToChannel.get(fd); 224 if (channel != null) { 225 int events = getEvents(eventAddress); 226 Event ev = new Event(channel, events); 227 228 // n-1 events are queued; This thread handles 229 // the last one except for the wakeup 230 if (n > 0) { 231 queue.offer(ev); 232 } else { 233 return ev; 234 } 235 } 236 } 237 } finally { 238 fdToChannelLock.readLock().unlock(); 239 } 240 } 241 } finally { 242 // to ensure that some thread will poll when all events have 243 // been consumed 244 queue.offer(NEED_TO_POLL); 245 } On 1/9/17, 12:16 PM, "nio-dev on behalf of Jeremiah Ness" wrote: >Using jre8u112 on CentOs7. > >I have an application which uses AsynchronousChannelGroup.withFixedThreadPool >with 5 threads. When the application in under high load creating 1000s of >AsynchronousSocketChannels per second, simultaneously closing many >AsynchronousSocketChannels, on occasion all of the EventHandlerTasks in >EPollPort become stuck waiting for events on the EPollPort.queue. All the >threads have the following stack: > >"CompletionThread-5" #33 daemon prio=5 os_prio=0 tid=0x00007f0a2829f800 nid=0x5c41 waiting on condition [0x00007f0a11ee1000] > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x00000006c4fe8868> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403) > at sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:262) > at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > >They all appear to be waiting for the queue to be filled (calling >BlockingQueue.take) however there are no threads calling the native method >epollWait to get more IO events. This condition persists until the application >is restarted. > >By examining the EPollPort class I have the understanding that one of the >threads should be polling. It this correct? > >By examining the EPollPort.EventHandlerTask.poll method I am wondering if there >is a code path that would allow all threads to be waiting on the queue. In >particular is the following possible within EPollPort.EventHandlerTask.poll: > >1. The native method epollWait returns 512 events. >2. Before the fdToChannelLock.readLock is acquired the channel associated with > the 512th event is closed. >3. The fixed size EPollPort.queue is filled to size 511. >4. The 512th event is processed, however because it has been closed it is no > longer in the fdToChannel map >5. The thread loops around the for(;;) loop and calls epollWait again >6. epollWait returns 2 more events >7. The fixed size queue is filled to its maximum capacity of 512. >8. The finally queue.offer(NEED_TO_POLL) call fails because the queue is full. > >If this occurs would all the EventHandlerTasks then eventually be stuck as per >the stack trace above? > >Thanks, >Jeremiah Ness > > > > From Alan.Bateman at oracle.com Thu Jan 12 14:24:32 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 12 Jan 2017 14:24:32 +0000 Subject: all EventHandlerTasks in EPollPort waiting on queue In-Reply-To: <9BD02A61-33A6-42A9-8B80-FC73CD6312BB@proofpoint.com> References: <9BD02A61-33A6-42A9-8B80-FC73CD6312BB@proofpoint.com> Message-ID: <819ea831-b31f-5082-b5d6-5bdb11b3d368@oracle.com> On 11/01/2017 22:11, Jeremiah Ness wrote: > (Sorry is this is the wrong place to discuss this matter. If you know of a > better place, please let me know.) > > I suspect there is a bug in EPollPort.java (and KQueuePort.java) and am looking > for a more informed opinion. > > Please reference the source code below from > src//solaris/classes/sun/nio/ch/EPollPort.java. > > On line 244 below the poll() method attempts to insert the NEED_TO_POLL event > into the queue. The queue is a fixed size ArrayBlockingQueue of size > MAX_EPOLL_EVENTS. On line 244, if the queue is full, then the event silently > fails to be inserted. The NEED_TO_POLL event is critical for the operation of > the EPollPort as it is the event which signals one of the threads to poll > again. > There does appear to be an issue here. Can you create a standalone test case to tickle test so that we can include it in a bug report? That would really help get to a regression test to include with the fix. -Alan. From jness at proofpoint.com Thu Jan 12 16:03:31 2017 From: jness at proofpoint.com (Jeremiah Ness) Date: Thu, 12 Jan 2017 16:03:31 +0000 Subject: all EventHandlerTasks in EPollPort waiting on queue In-Reply-To: <819ea831-b31f-5082-b5d6-5bdb11b3d368@oracle.com> References: <9BD02A61-33A6-42A9-8B80-FC73CD6312BB@proofpoint.com> <819ea831-b31f-5082-b5d6-5bdb11b3d368@oracle.com> Message-ID: <6E823DB7-749C-42C8-905E-3EE4F3FD4D1C@proofpoint.com> On 1/12/17, 9:24 AM, "Alan Bateman" wrote: >On 11/01/2017 22:11, Jeremiah Ness wrote: > >> Please reference the source code below from >> src//solaris/classes/sun/nio/ch/EPollPort.java. >> >> On line 244 below the poll() method attempts to insert the NEED_TO_POLL event >> into the queue. The queue is a fixed size ArrayBlockingQueue of size >> MAX_EPOLL_EVENTS. On line 244, if the queue is full, then the event silently >> fails to be inserted. The NEED_TO_POLL event is critical for the operation of >> the EPollPort as it is the event which signals one of the threads to poll >> again. >> >There does appear to be an issue here. Can you create a standalone test >case to tickle test so that we can include it in a bug report? That >would really help get to a regression test to include with the fix. I do have a test program that can trigger the condition for me. I have attached the source code for PortTest.java. PortTest has successfully demonstrated the issue in the following configurations: - OSX 10.11.6 with java version "1.8.0_112" - CentOs7 with kernel 3.10.0-514.2.2.el7.x86_64 and with openjdk version "1.8.0_111" Note the following caveats: - PortTest depends on the order in which IO events are returned (because we have to arrange to have the last iteration through the loop have a null channel) - PortTest uses reflection to artificially grab the writer Port.fdToChannelLock so that the race condition can be triggered There may be better ways to trigger this race condition. If you have other ideas, I could try them. By default, the PortTest will attempt to close the channel which it believes will be processed last by the inner loop. That corresponds to a channel that is given an id=0. In this case the program will not exit because the ChannelGroup cannot be shutdown. Using jstack I could see the ChannelGroup thread is stuck waiting on the queue instead of polling for more events. This is the output of the program I see when the race is triggered: $ java PortTest ? Jan 12, 2017 10:56:52 AM PortTest$MyCompletionHandler finish INFO: Closed client channel id=0 completed=false Jan 12, 2017 10:56:55 AM PortTest test INFO: TIMED OUT WAITING FOR EVENTS -- completion-thread must be stuck Jan 12, 2017 10:56:55 AM PortTest test INFO: finished waiting for batch of completion handlers Jan 12, 2017 10:56:55 AM PortTest simpleTcpServer INFO: Closing socket (on server) id=-100 Jan 12, 2017 10:56:58 AM PortTest test INFO: timed out waiting for termination of ChannelGroup Jan 12, 2017 10:56:58 AM PortTest test INFO: completedCount=512 failedCount=1 You can run PortTest and tell it to close a different channel instead of the one with id=0. When I do this, the program shuts down and exits normally. Thoughts? Thanks, Jeremiah Ness -------------- next part -------------- A non-text attachment was scrubbed... Name: PortTest.java Type: application/octet-stream Size: 12761 bytes Desc: PortTest.java URL: From brian.burkhalter at oracle.com Fri Jan 13 01:06:30 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 12 Jan 2017 17:06:30 -0800 Subject: all EventHandlerTasks in EPollPort waiting on queue In-Reply-To: <6E823DB7-749C-42C8-905E-3EE4F3FD4D1C@proofpoint.com> References: <9BD02A61-33A6-42A9-8B80-FC73CD6312BB@proofpoint.com> <819ea831-b31f-5082-b5d6-5bdb11b3d368@oracle.com> <6E823DB7-749C-42C8-905E-3EE4F3FD4D1C@proofpoint.com> Message-ID: <60C4AB87-50A1-4158-9D79-3170DB6E9565@oracle.com> On Jan 12, 2017, at 8:03 AM, Jeremiah Ness wrote: >> There does appear to be an issue here. Can you create a standalone test >> case to tickle test so that we can include it in a bug report? That >> would really help get to a regression test to include with the fix. > > I do have a test program that can trigger the condition for me. I have attached the source code for PortTest.java. PortTest has successfully demonstrated the issue in the following configurations: > > - OSX 10.11.6 with java version "1.8.0_112" > - CentOs7 with kernel 3.10.0-514.2.2.el7.x86_64 and with openjdk version "1.8.0_111" I have filed https://bugs.openjdk.java.net/browse/JDK-8172750 to track this problem. Thank you for providing a test case. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Jan 13 17:15:08 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 13 Jan 2017 09:15:08 -0800 Subject: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires repeatedly Message-ID: Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8172547 Patch: http://cr.openjdk.java.net/~bpb/8172547/webrev.00/ The 64-bit signed integer timeout was converted to a Windows native timeval struct without taking account that the members of this struct are 32-bit signed integers, hence timeouts greater than or equal to 2147483648000L milliseconds could end up being converted to a negative value. Thanks, Brian From Roger.Riggs at Oracle.com Fri Jan 13 22:23:35 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 13 Jan 2017 17:23:35 -0500 Subject: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires repeatedly In-Reply-To: References: Message-ID: <631b765c-1031-84c1-890a-de2964cee1e9@Oracle.com> Hi Brian, SelectTimeout: - 35: for consistency use the '_' in both constants - 40: you may want isTimedOut and theException to be volatile since it is written and read by different threads Comment on the existing code: line 77. Using Thread.join(SLEEP_MILLIES) would continue quicker in the case of an exception or for the timeout=0 case. It would knock some time off the clock time for the test. Roger On 1/13/2017 12:15 PM, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8172547 > Patch: http://cr.openjdk.java.net/~bpb/8172547/webrev.00/ > > The 64-bit signed integer timeout was converted to a Windows native timeval struct without taking account that the members of this struct are 32-bit signed integers, hence timeouts greater than or equal to 2147483648000L milliseconds could end up being converted to a negative value. > > Thanks, > > Brian From brian.burkhalter at oracle.com Fri Jan 13 22:57:04 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 13 Jan 2017 14:57:04 -0800 Subject: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires repeatedly In-Reply-To: <631b765c-1031-84c1-890a-de2964cee1e9@Oracle.com> References: <631b765c-1031-84c1-890a-de2964cee1e9@Oracle.com> Message-ID: <12F1CE2B-2CDC-4CC4-8323-A69DF5126ED1@oracle.com> Hi Roger, Thanks for the comments. Here is an update which takes them into account: http://cr.openjdk.java.net/~bpb/8172547/webrev.01/ Brian On Jan 13, 2017, at 2:23 PM, Roger Riggs wrote: > SelectTimeout: > - 35: for consistency use the '_' in both constants > - 40: you may want isTimedOut and theException to be volatile since it is written and read by different threads > > > Comment on the existing code: line 77. > Using Thread.join(SLEEP_MILLIES) would continue quicker in the case of an exception or for the timeout=0 case. > It would knock some time off the clock time for the test. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Mon Jan 16 08:23:22 2017 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 16 Jan 2017 08:23:22 +0000 Subject: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires repeatedly In-Reply-To: <12F1CE2B-2CDC-4CC4-8323-A69DF5126ED1@oracle.com> References: <631b765c-1031-84c1-890a-de2964cee1e9@Oracle.com> <12F1CE2B-2CDC-4CC4-8323-A69DF5126ED1@oracle.com> Message-ID: <88447a5823e044eb8ba473b7aef3b796@derote13de46.global.corp.sap> Hi Brian, the fix and test look fine, +1. Minor remark for the comment in src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c: 82 // signed 64-bit jlong needs to clamped -> Should be 82 // signed 64-bit jlong needs to be clamped Best regards Christoph From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Brian Burkhalter Sent: Freitag, 13. Januar 2017 23:57 To: Roger Riggs Cc: nio-dev at openjdk.java.net Subject: Re: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires repeatedly Hi Roger, Thanks for the comments. Here is an update which takes them into account: http://cr.openjdk.java.net/~bpb/8172547/webrev.01/ Brian On Jan 13, 2017, at 2:23 PM, Roger Riggs > wrote: SelectTimeout: - 35: for consistency use the '_' in both constants - 40: you may want isTimedOut and theException to be volatile since it is written and read by different threads Comment on the existing code: line 77. Using Thread.join(SLEEP_MILLIES) would continue quicker in the case of an exception or for the timeout=0 case. It would knock some time off the clock time for the test. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon Jan 16 20:12:08 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 16 Jan 2017 20:12:08 +0000 Subject: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires repeatedly In-Reply-To: <12F1CE2B-2CDC-4CC4-8323-A69DF5126ED1@oracle.com> References: <631b765c-1031-84c1-890a-de2964cee1e9@Oracle.com> <12F1CE2B-2CDC-4CC4-8323-A69DF5126ED1@oracle.com> Message-ID: On 13/01/2017 22:57, Brian Burkhalter wrote: > Hi Roger, > > Thanks for the comments. Here is an update which takes them into account: > > http://cr.openjdk.java.net/~bpb/8172547/webrev.01/ > > > Aside from the typo in the comment that Christophpointed then then this looks good to me. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Jan 17 20:11:34 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 17 Jan 2017 12:11:34 -0800 Subject: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires repeatedly In-Reply-To: References: <631b765c-1031-84c1-890a-de2964cee1e9@Oracle.com> <12F1CE2B-2CDC-4CC4-8323-A69DF5126ED1@oracle.com> Message-ID: <39BD9A74-86D7-4AC9-A4B5-DA0708C6353B@oracle.com> On Jan 16, 2017, at 12:12 PM, Alan Bateman wrote: > On 13/01/2017 22:57, Brian Burkhalter wrote: >> Hi Roger, >> >> Thanks for the comments. Here is an update which takes them into account: >> >> http://cr.openjdk.java.net/~bpb/8172547/webrev.01/ >> >> > Aside from the typo in the comment that Christoph pointed then then this looks good to me. Here is an updated patch with the typo in the implementation fixed and line 77 of the test changed to t.join(SLEEP_MILLIS). http://cr.openjdk.java.net/~bpb/8172547/webrev.02/ Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Roger.Riggs at Oracle.com Tue Jan 17 20:16:48 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 17 Jan 2017 15:16:48 -0500 Subject: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires repeatedly In-Reply-To: <39BD9A74-86D7-4AC9-A4B5-DA0708C6353B@oracle.com> References: <631b765c-1031-84c1-890a-de2964cee1e9@Oracle.com> <12F1CE2B-2CDC-4CC4-8323-A69DF5126ED1@oracle.com> <39BD9A74-86D7-4AC9-A4B5-DA0708C6353B@oracle.com> Message-ID: <135dd36c-b51b-9f9b-8f7c-67825cf0e405@Oracle.com> Looks good Brian. +1 Roger On 1/17/2017 3:11 PM, Brian Burkhalter wrote: > > On Jan 16, 2017, at 12:12 PM, Alan Bateman > wrote: > >> On 13/01/2017 22:57, Brian Burkhalter wrote: >>> Hi Roger, >>> >>> Thanks for the comments. Here is an update which takes them into >>> account: >>> >>> http://cr.openjdk.java.net/~bpb/8172547/webrev.01/ >>> >>> >>> >> Aside from the typo in the comment that Christophpointed then then >> this looks good to me. > > Here is an updated patch with the typo in the implementation fixed and > line 77 of the test changed to t.join(SLEEP_MILLIS). > > http://cr.openjdk.java.net/~bpb/8172547/webrev.02/ > > > Thanks, > > Brian From xueming.shen at oracle.com Tue Jan 17 22:39:13 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 17 Jan 2017 14:39:13 -0800 Subject: RFR JDK-8172921: Zip filesystem performance improvement and code cleanup Message-ID: <587E9D11.2000901@oracle.com> Hi, Please review the following changes for zipfs implementation. issue: https://bugs.openjdk.java.net/browse/JDK-8172921 webrev: http://cr.openjdk.java.net/~sherman/8172921/webrev/ javac has moved to use zipfs (instead of ZipFile) to access jar files in jdk9. Here are some changes to improve the performance of access time and footprint (reduce the unnecessary object allocation ...) The improvement has been measured by the jmh benchmark test as http://cr.openjdk.java.net/~sherman/8172921/MyBenchmark.java with the benchmark sores (before-OLD/after-NEW) at: http://cr.openjdk.java.net/~sherman/8172921/scores The main change is to change the internal directory path representation from the zip specific format (directory name ends with "/", "/dir/" for directory ".dir" for example) to the "normalized" form with the tailing "/", which reduces the back and forth conversion between the normal "unix style" path and the "zip style" path when doing path creation, path lookup and entry access, which also simplified the entry lookup logic. We are seeing pretty good performance improvement. There is another change, which involves the API change, for the "non-existing" path look up (which shows pretty bad numbers as ZFS_ExistsNG"), is not included in this patch. I hope we can address that as well in jdk9, but probably in another patch. Thanks, Sherman From claes.redestad at oracle.com Tue Jan 17 23:24:31 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 18 Jan 2017 00:24:31 +0100 Subject: RFR JDK-8172921: Zip filesystem performance improvement and code cleanup In-Reply-To: <587E9D11.2000901@oracle.com> References: <587E9D11.2000901@oracle.com> Message-ID: <5cc580f1-cd0c-94fb-d671-7001847a7b30@oracle.com> Hi Sherman, thanks for doing this! Looks good - I stumbled on the ZipPath::getResolved changes at first, but they seem sound and I can see how the previous test was likely to unnecessarily create a new byte[] on most regular files. Nits: Code formatting seems unintentionally messed up in places, especially ZipCoder.java. ZipCoder$UTF8 has a chunk of commented out code that should likely be removed. /Claes On 2017-01-17 23:39, Xueming Shen wrote: > Hi, > > Please review the following changes for zipfs implementation. > > issue: https://bugs.openjdk.java.net/browse/JDK-8172921 > webrev: http://cr.openjdk.java.net/~sherman/8172921/webrev/ > > > javac has moved to use zipfs (instead of ZipFile) to access jar files in > jdk9. Here are > some changes to improve the performance of access time and footprint > (reduce the > unnecessary object allocation ...) The improvement has been measured by > the jmh > benchmark test as > > http://cr.openjdk.java.net/~sherman/8172921/MyBenchmark.java > > with the benchmark sores (before-OLD/after-NEW) at: > > http://cr.openjdk.java.net/~sherman/8172921/scores > > > The main change is to change the internal directory path representation > from the > zip specific format (directory name ends with "/", "/dir/" for directory > ".dir" for > example) to the "normalized" form with the tailing "/", which reduces > the back and > forth conversion between the normal "unix style" path and the "zip > style" path when > doing path creation, path lookup and entry access, which also simplified > the entry > lookup logic. > > We are seeing pretty good performance improvement. > > There is another change, which involves the API change, for the > "non-existing" path > look up (which shows pretty bad numbers as ZFS_ExistsNG"), is not > included in this > patch. I hope we can address that as well in jdk9, but probably in > another patch. > > Thanks, > Sherman > > From xueming.shen at oracle.com Tue Jan 17 23:56:12 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 17 Jan 2017 15:56:12 -0800 Subject: RFR JDK-8172921: Zip filesystem performance improvement and code cleanup In-Reply-To: <5cc580f1-cd0c-94fb-d671-7001847a7b30@oracle.com> References: <587E9D11.2000901@oracle.com> <5cc580f1-cd0c-94fb-d671-7001847a7b30@oracle.com> Message-ID: <587EAF1C.6090002@oracle.com> On 1/17/17, 3:24 PM, Claes Redestad wrote: > > Looks good - I stumbled on the ZipPath::getResolved changes at first, > but they seem sound and I can see how the previous test was likely to > unnecessarily create a new byte[] on most regular files. It's a little hacky to only go into resolve0 for "." or ".." with a tailing "/" to avoid a single ".", such in Foo.class, to trigger the expensive resolve0(). > > Nits: > Code formatting seems unintentionally messed up in places, especially > ZipCoder.java. ran the normalizer on the source code. > > ZipCoder$UTF8 has a chunk of commented out code that should likely be > removed. > removed thanks, Sherman From christoph.langer at sap.com Wed Jan 18 07:57:36 2017 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 18 Jan 2017 07:57:36 +0000 Subject: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires repeatedly In-Reply-To: <135dd36c-b51b-9f9b-8f7c-67825cf0e405@Oracle.com> References: <631b765c-1031-84c1-890a-de2964cee1e9@Oracle.com> <12F1CE2B-2CDC-4CC4-8323-A69DF5126ED1@oracle.com> <39BD9A74-86D7-4AC9-A4B5-DA0708C6353B@oracle.com> <135dd36c-b51b-9f9b-8f7c-67825cf0e405@Oracle.com> Message-ID: <638afd0d79364f9794c3a08f0bc804a3@derote13de46.global.corp.sap> Yep, +1. > -----Original Message----- > From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Roger > Riggs > Sent: Dienstag, 17. Januar 2017 21:17 > To: nio-dev at openjdk.java.net > Subject: Re: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires > repeatedly > > Looks good Brian. > > +1 > > Roger > > On 1/17/2017 3:11 PM, Brian Burkhalter wrote: > > > > On Jan 16, 2017, at 12:12 PM, Alan Bateman > > wrote: > > > >> On 13/01/2017 22:57, Brian Burkhalter wrote: > >>> Hi Roger, > >>> > >>> Thanks for the comments. Here is an update which takes them into > >>> account: > >>> > >>> http://cr.openjdk.java.net/~bpb/8172547/webrev.01/ > >>> > >>> > >>> > >> Aside from the typo in the comment that Christophpointed then then > >> this looks good to me. > > > > Here is an updated patch with the typo in the implementation fixed and > > line 77 of the test changed to t.join(SLEEP_MILLIS). > > > > http://cr.openjdk.java.net/~bpb/8172547/webrev.02/ > > > > > > Thanks, > > > > Brian From xueming.shen at oracle.com Wed Jan 18 18:43:10 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 18 Jan 2017 10:43:10 -0800 Subject: RFR JDK-8172921: Zip filesystem performance improvement and code cleanup In-Reply-To: References: <587E9D11.2000901@oracle.com> Message-ID: <587FB73E.6060302@oracle.com> Hi Andrej, Thanks for the review. I was debating with myself if it's really a good idea to "silently" go with the default charset in case the specified charset can not be obtained. Given the charset name is passed via the env map, it might be better to simply let the runtime exception get thrown to communicate to the caller that the encoding property is wrong. The webrev has been updated accordingly. Sherman On 01/17/2017 11:56 PM, Andrej Golovnin wrote: > Hi Sherman, > > src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipCoder.java > > 72 public static ZipCoder get(String csn) { > 73 try { > 74 Charset cs = Charset.forName(csn); > 75 if (cs.name().equals("UTF-8")) { > 76 return utf8; > 77 } > 78 return new ZipCoder(cs); > 79 } catch (Throwable t) { > 80 t.printStackTrace(); > 81 } > 82 return new ZipCoder(Charset.defaultCharset()); > 83 } > > Wouldn't it be better to use System.Logger instead of printStackTrace > in the line 80? > > Best regards, > Andrej Golovnin > > On Tue, Jan 17, 2017 at 11:39 PM, Xueming Shen wrote: >> Hi, >> >> Please review the following changes for zipfs implementation. >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8172921 >> webrev: http://cr.openjdk.java.net/~sherman/8172921/webrev/ >> >> >> javac has moved to use zipfs (instead of ZipFile) to access jar files in >> jdk9. Here are >> some changes to improve the performance of access time and footprint (reduce >> the >> unnecessary object allocation ...) The improvement has been measured by the >> jmh >> benchmark test as >> >> http://cr.openjdk.java.net/~sherman/8172921/MyBenchmark.java >> >> with the benchmark sores (before-OLD/after-NEW) at: >> >> http://cr.openjdk.java.net/~sherman/8172921/scores >> >> >> The main change is to change the internal directory path representation from >> the >> zip specific format (directory name ends with "/", "/dir/" for directory >> ".dir" for >> example) to the "normalized" form with the tailing "/", which reduces the >> back and >> forth conversion between the normal "unix style" path and the "zip style" >> path when >> doing path creation, path lookup and entry access, which also simplified the >> entry >> lookup logic. >> >> We are seeing pretty good performance improvement. >> >> There is another change, which involves the API change, for the >> "non-existing" path >> look up (which shows pretty bad numbers as ZFS_ExistsNG"), is not included >> in this >> patch. I hope we can address that as well in jdk9, but probably in another >> patch. >> >> Thanks, >> Sherman >> >> From brian.burkhalter at oracle.com Wed Jan 18 20:08:59 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 18 Jan 2017 12:08:59 -0800 Subject: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires repeatedly In-Reply-To: <638afd0d79364f9794c3a08f0bc804a3@derote13de46.global.corp.sap> References: <631b765c-1031-84c1-890a-de2964cee1e9@Oracle.com> <12F1CE2B-2CDC-4CC4-8323-A69DF5126ED1@oracle.com> <39BD9A74-86D7-4AC9-A4B5-DA0708C6353B@oracle.com> <135dd36c-b51b-9f9b-8f7c-67825cf0e405@Oracle.com> <638afd0d79364f9794c3a08f0bc804a3@derote13de46.global.corp.sap> Message-ID: <7BCD972B-1A9A-4B4C-A154-D2367E0D0E0A@oracle.com> I observed some failures with isTimedOut in the test being set to ?true? despite the select() thread having been interrupted. This is fixed by the change to the test included below [1]. With this change, the test predictably fails on Windows without the implementation change, but passes with it. the complete patch is at [2]. Thanks, Brian [1] Delta: webrev.02 -> webrev.03 --- a/test/java/nio/channels/Selector/SelectTimeout.java +++ b/test/java/nio/channels/Selector/SelectTimeout.java @@ -67,7 +67,9 @@ Thread t = new Thread(() -> { try { selector.select(timeout); - isTimedOut = true; + if (!Thread.currentThread().isInterrupted()) { + isTimedOut = true; + } } catch (IOException ioe) { theException = ioe; } [2] http://cr.openjdk.java.net/~bpb/8172547/webrev.03/ On Jan 17, 2017, at 11:57 PM, Langer, Christoph wrote: > Yep, +1. > >> -----Original Message----- >> From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Roger >> Riggs >> Sent: Dienstag, 17. Januar 2017 21:17 >> To: nio-dev at openjdk.java.net >> Subject: Re: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires >> repeatedly >> >> Looks good Brian. >> >> +1 >> >> Roger >> >> On 1/17/2017 3:11 PM, Brian Burkhalter wrote: >>> >>> On Jan 16, 2017, at 12:12 PM, Alan Bateman >> > wrote: >>> >>>> Aside from the typo in the comment that Christophpointed then then >>>> this looks good to me. From andrej.golovnin at gmail.com Wed Jan 18 21:16:25 2017 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Wed, 18 Jan 2017 22:16:25 +0100 Subject: RFR JDK-8172921: Zip filesystem performance improvement and code cleanup In-Reply-To: <587FB73E.6060302@oracle.com> References: <587E9D11.2000901@oracle.com> <587FB73E.6060302@oracle.com> Message-ID: <862DD833-1BD4-4305-AF9E-B4E98B2D0234@gmail.com> Hi Sherman, thanks for the update. It looks much better now. One more thing: src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipCoder.java 70 private static ZipCoder utf8 = new UTF8(); I think this field can be final as it never changes. Best regards, Andrej Golovnin > On 18 Jan 2017, at 19:43, Xueming Shen wrote: > > Hi Andrej, > > Thanks for the review. > > I was debating with myself if it's really a good idea to "silently" go with the > default charset in case the specified charset can not be obtained. Given the > charset name is passed via the env map, it might be better to simply let the > runtime exception get thrown to communicate to the caller that the encoding > property is wrong. > > The webrev has been updated accordingly. > > Sherman > > On 01/17/2017 11:56 PM, Andrej Golovnin wrote: >> Hi Sherman, >> >> src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipCoder.java >> >> 72 public static ZipCoder get(String csn) { >> 73 try { >> 74 Charset cs = Charset.forName(csn); >> 75 if (cs.name().equals("UTF-8")) { >> 76 return utf8; >> 77 } >> 78 return new ZipCoder(cs); >> 79 } catch (Throwable t) { >> 80 t.printStackTrace(); >> 81 } >> 82 return new ZipCoder(Charset.defaultCharset()); >> 83 } >> >> Wouldn't it be better to use System.Logger instead of printStackTrace >> in the line 80? >> >> Best regards, >> Andrej Golovnin >> >> On Tue, Jan 17, 2017 at 11:39 PM, Xueming Shen wrote: >>> Hi, >>> >>> Please review the following changes for zipfs implementation. >>> >>> issue: https://bugs.openjdk.java.net/browse/JDK-8172921 >>> webrev: http://cr.openjdk.java.net/~sherman/8172921/webrev/ >>> >>> >>> javac has moved to use zipfs (instead of ZipFile) to access jar files in >>> jdk9. Here are >>> some changes to improve the performance of access time and footprint (reduce >>> the >>> unnecessary object allocation ...) The improvement has been measured by the >>> jmh >>> benchmark test as >>> >>> http://cr.openjdk.java.net/~sherman/8172921/MyBenchmark.java >>> >>> with the benchmark sores (before-OLD/after-NEW) at: >>> >>> http://cr.openjdk.java.net/~sherman/8172921/scores >>> >>> >>> The main change is to change the internal directory path representation from >>> the >>> zip specific format (directory name ends with "/", "/dir/" for directory >>> ".dir" for >>> example) to the "normalized" form with the tailing "/", which reduces the >>> back and >>> forth conversion between the normal "unix style" path and the "zip style" >>> path when >>> doing path creation, path lookup and entry access, which also simplified the >>> entry >>> lookup logic. >>> >>> We are seeing pretty good performance improvement. >>> >>> There is another change, which involves the API change, for the >>> "non-existing" path >>> look up (which shows pretty bad numbers as ZFS_ExistsNG"), is not included >>> in this >>> patch. I hope we can address that as well in jdk9, but probably in another >>> patch. >>> >>> Thanks, >>> Sherman >>> >>> > From brian.burkhalter at oracle.com Wed Jan 18 22:20:01 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 18 Jan 2017 14:20:01 -0800 Subject: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires repeatedly In-Reply-To: <7BCD972B-1A9A-4B4C-A154-D2367E0D0E0A@oracle.com> References: <631b765c-1031-84c1-890a-de2964cee1e9@Oracle.com> <12F1CE2B-2CDC-4CC4-8323-A69DF5126ED1@oracle.com> <39BD9A74-86D7-4AC9-A4B5-DA0708C6353B@oracle.com> <135dd36c-b51b-9f9b-8f7c-67825cf0e405@Oracle.com> <638afd0d79364f9794c3a08f0bc804a3@derote13de46.global.corp.sap> <7BCD972B-1A9A-4B4C-A154-D2367E0D0E0A@oracle.com> Message-ID: <998F5C62-A7D2-4813-98EF-CC3B3FFAE4B0@oracle.com> All right, let?s forget webrev.03. Here is what I hope to be the final version: http://cr.openjdk.java.net/~bpb/8172547/webrev.04/ Changes are only to the test, not the implementation code. Thanks, Brian On Jan 18, 2017, at 12:08 PM, Brian Burkhalter wrote: > I observed some failures with isTimedOut in the test being set to ?true? [?] > > [2] http://cr.openjdk.java.net/~bpb/8172547/webrev.03/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From xueming.shen at oracle.com Thu Jan 19 03:18:01 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 18 Jan 2017 19:18:01 -0800 Subject: RFR JDK-8172921: Zip filesystem performance improvement and code cleanup In-Reply-To: <862DD833-1BD4-4305-AF9E-B4E98B2D0234@gmail.com> References: <587E9D11.2000901@oracle.com> <587FB73E.6060302@oracle.com> <862DD833-1BD4-4305-AF9E-B4E98B2D0234@gmail.com> Message-ID: <58802FE9.1040700@oracle.com> Andrej, I have pushed in the change before this suggestion came in. I will put this on my todo list when touch that code again next time. Thanks, Sherman On 1/18/17, 1:16 PM, Andrej Golovnin wrote: > Hi Sherman, > > thanks for the update. It looks much better now. > One more thing: > > src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipCoder.java > > 70 private static ZipCoder utf8 = new UTF8(); > > I think this field can be final as it never changes. > > Best regards, > Andrej Golovnin > >> On 18 Jan 2017, at 19:43, Xueming Shen wrote: >> >> Hi Andrej, >> >> Thanks for the review. >> >> I was debating with myself if it's really a good idea to "silently" go with the >> default charset in case the specified charset can not be obtained. Given the >> charset name is passed via the env map, it might be better to simply let the >> runtime exception get thrown to communicate to the caller that the encoding >> property is wrong. >> >> The webrev has been updated accordingly. >> >> Sherman >> >> On 01/17/2017 11:56 PM, Andrej Golovnin wrote: >>> Hi Sherman, >>> >>> src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipCoder.java >>> >>> 72 public static ZipCoder get(String csn) { >>> 73 try { >>> 74 Charset cs = Charset.forName(csn); >>> 75 if (cs.name().equals("UTF-8")) { >>> 76 return utf8; >>> 77 } >>> 78 return new ZipCoder(cs); >>> 79 } catch (Throwable t) { >>> 80 t.printStackTrace(); >>> 81 } >>> 82 return new ZipCoder(Charset.defaultCharset()); >>> 83 } >>> >>> Wouldn't it be better to use System.Logger instead of printStackTrace >>> in the line 80? >>> >>> Best regards, >>> Andrej Golovnin >>> >>> On Tue, Jan 17, 2017 at 11:39 PM, Xueming Shen wrote: >>>> Hi, >>>> >>>> Please review the following changes for zipfs implementation. >>>> >>>> issue: https://bugs.openjdk.java.net/browse/JDK-8172921 >>>> webrev: http://cr.openjdk.java.net/~sherman/8172921/webrev/ >>>> >>>> >>>> javac has moved to use zipfs (instead of ZipFile) to access jar files in >>>> jdk9. Here are >>>> some changes to improve the performance of access time and footprint (reduce >>>> the >>>> unnecessary object allocation ...) The improvement has been measured by the >>>> jmh >>>> benchmark test as >>>> >>>> http://cr.openjdk.java.net/~sherman/8172921/MyBenchmark.java >>>> >>>> with the benchmark sores (before-OLD/after-NEW) at: >>>> >>>> http://cr.openjdk.java.net/~sherman/8172921/scores >>>> >>>> >>>> The main change is to change the internal directory path representation from >>>> the >>>> zip specific format (directory name ends with "/", "/dir/" for directory >>>> ".dir" for >>>> example) to the "normalized" form with the tailing "/", which reduces the >>>> back and >>>> forth conversion between the normal "unix style" path and the "zip style" >>>> path when >>>> doing path creation, path lookup and entry access, which also simplified the >>>> entry >>>> lookup logic. >>>> >>>> We are seeing pretty good performance improvement. >>>> >>>> There is another change, which involves the API change, for the >>>> "non-existing" path >>>> look up (which shows pretty bad numbers as ZFS_ExistsNG"), is not included >>>> in this >>>> patch. I hope we can address that as well in jdk9, but probably in another >>>> patch. >>>> >>>> Thanks, >>>> Sherman >>>> >>>> From Roger.Riggs at Oracle.com Thu Jan 19 20:09:27 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 19 Jan 2017 15:09:27 -0500 Subject: JDK 9 RFR of 8172547: (se) Selector.select(Long.MAX_VALUE) fires repeatedly In-Reply-To: <998F5C62-A7D2-4813-98EF-CC3B3FFAE4B0@oracle.com> References: <631b765c-1031-84c1-890a-de2964cee1e9@Oracle.com> <12F1CE2B-2CDC-4CC4-8323-A69DF5126ED1@oracle.com> <39BD9A74-86D7-4AC9-A4B5-DA0708C6353B@oracle.com> <135dd36c-b51b-9f9b-8f7c-67825cf0e405@Oracle.com> <638afd0d79364f9794c3a08f0bc804a3@derote13de46.global.corp.sap> <7BCD972B-1A9A-4B4C-A154-D2367E0D0E0A@oracle.com> <998F5C62-A7D2-4813-98EF-CC3B3FFAE4B0@oracle.com> Message-ID: <20e7b49c-a0c8-fde9-a7aa-8768854d8cee@Oracle.com> Hi Brian, Looks fine. Thanks, Roger On 1/18/2017 5:20 PM, Brian Burkhalter wrote: > All right, let?s forget webrev.03. Here is what I hope to be the final > version: > > http://cr.openjdk.java.net/~bpb/8172547/webrev.04/ > > > Changes are only to the test, not the implementation code. > > Thanks, > > Brian > > On Jan 18, 2017, at 12:08 PM, Brian Burkhalter > > wrote: > >> I observed some failures with isTimedOut in the test being set to >> ?true? [?] >> >> [2]http://cr.openjdk.java.net/~bpb/8172547/webrev.03/ >> > From xueming.shen at oracle.com Thu Jan 19 21:39:38 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 19 Jan 2017 13:39:38 -0800 Subject: JDK-8173072: zipfs fails to handle incorrect info-zip "extended timestamp extra field" Message-ID: <5881321A.9070105@oracle.com> Hi Please review the change for issue: https://bugs.openjdk.java.net/browse/JDK-8173072 webrev: http://cr.openjdk.java.net/~sherman/8173072/webrev As described in the issue, the root cause is that the offending zip/jar file has a "extended timestamp extra field" that does not strictly follow the spec, AND the zipfs implementation does not do a good job to do sanity check on the data size like what the ZipEntry code does when check the timestamps. The change here is to check both the flag and the length, stop reading a/ctime if there is no more data, even the flags field says there are more data. I have run the test manually to verify the change, but decided not to check in a binary zip/jar file (which is not encouraged for jdk repo) for the regression test. Thanks, Sherman -------------- next part -------------- An HTML attachment was scrubbed... URL: From claes.redestad at oracle.com Thu Jan 19 23:04:28 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 20 Jan 2017 00:04:28 +0100 Subject: JDK-8173072: zipfs fails to handle incorrect info-zip "extended timestamp extra field" In-Reply-To: <5881321A.9070105@oracle.com> References: <5881321A.9070105@oracle.com> Message-ID: Looks good to me. You could simplify a bit and write: int end = locPos + locSZ - 4; and use direct comparisons instead of adding 4 in each test. Thanks! /Claes On 2017-01-19 22:39, Xueming Shen wrote: > Hi > > Please review the change for > > issue: https://bugs.openjdk.java.net/browse/JDK-8173072 > webrev: http://cr.openjdk.java.net/~sherman/8173072/webrev > > > As described in the issue, the root cause is that the offending zip/jar > file > has a "extended timestamp extra field" that does not strictly follow the > spec, AND the zipfs implementation does not do a good job to do sanity > check on the data size like what the ZipEntry code does when check the > timestamps. The change here is to check both the flag and the length, > stop reading a/ctime if there is no more data, even the flags field says > there are more data. > > I have run the test manually to verify the change, but decided not to check > in a binary zip/jar file (which is not encouraged for jdk repo) for the > regression test. > > Thanks, > Sherman From xueming.shen at oracle.com Thu Jan 19 23:35:28 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 19 Jan 2017 15:35:28 -0800 Subject: JDK-8173072: zipfs fails to handle incorrect info-zip "extended timestamp extra field" In-Reply-To: References: <5881321A.9070105@oracle.com> Message-ID: <58814D40.8030804@oracle.com> Thanks Claes! webrev has been updated accordingly http://cr.openjdk.java.net/~sherman/8173072/webrev On 01/19/2017 03:04 PM, Claes Redestad wrote: > Looks good to me. > > You could simplify a bit and write: int end = locPos + locSZ - 4; > and use direct comparisons instead of adding 4 in each test. > > Thanks! > > /Claes > > On 2017-01-19 22:39, Xueming Shen wrote: >> Hi >> >> Please review the change for >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8173072 >> webrev: http://cr.openjdk.java.net/~sherman/8173072/webrev >> >> >> As described in the issue, the root cause is that the offending zip/jar >> file >> has a "extended timestamp extra field" that does not strictly follow the >> spec, AND the zipfs implementation does not do a good job to do sanity >> check on the data size like what the ZipEntry code does when check the >> timestamps. The change here is to check both the flag and the length, >> stop reading a/ctime if there is no more data, even the flags field says >> there are more data. >> >> I have run the test manually to verify the change, but decided not to check >> in a binary zip/jar file (which is not encouraged for jdk repo) for the >> regression test. >> >> Thanks, >> Sherman From claes.redestad at oracle.com Fri Jan 20 00:11:07 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 20 Jan 2017 01:11:07 +0100 Subject: JDK-8173072: zipfs fails to handle incorrect info-zip "extended timestamp extra field" In-Reply-To: <58814D40.8030804@oracle.com> References: <5881321A.9070105@oracle.com> <58814D40.8030804@oracle.com> Message-ID: On 2017-01-20 00:35, Xueming Shen wrote: > Thanks Claes! > > webrev has been updated accordingly > > http://cr.openjdk.java.net/~sherman/8173072/webrev +1 /Claes > > On 01/19/2017 03:04 PM, Claes Redestad wrote: >> Looks good to me. >> >> You could simplify a bit and write: int end = locPos + locSZ - 4; >> and use direct comparisons instead of adding 4 in each test. >> >> Thanks! >> >> /Claes >> >> On 2017-01-19 22:39, Xueming Shen wrote: >>> Hi >>> >>> Please review the change for >>> >>> issue: https://bugs.openjdk.java.net/browse/JDK-8173072 >>> webrev: http://cr.openjdk.java.net/~sherman/8173072/webrev >>> >>> >>> As described in the issue, the root cause is that the offending zip/jar >>> file >>> has a "extended timestamp extra field" that does not strictly follow the >>> spec, AND the zipfs implementation does not do a good job to do sanity >>> check on the data size like what the ZipEntry code does when check the >>> timestamps. The change here is to check both the flag and the length, >>> stop reading a/ctime if there is no more data, even the flags field says >>> there are more data. >>> >>> I have run the test manually to verify the change, but decided not to >>> check >>> in a binary zip/jar file (which is not encouraged for jdk repo) for the >>> regression test. >>> >>> Thanks, >>> Sherman > From Alan.Bateman at oracle.com Fri Jan 20 07:44:21 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 20 Jan 2017 07:44:21 +0000 Subject: JDK-8173072: zipfs fails to handle incorrect info-zip "extended timestamp extra field" In-Reply-To: <58814D40.8030804@oracle.com> References: <5881321A.9070105@oracle.com> <58814D40.8030804@oracle.com> Message-ID: On 19/01/2017 23:35, Xueming Shen wrote: > Thanks Claes! > > webrev has been updated accordingly > > http://cr.openjdk.java.net/~sherman/8173072/webrev > The changes looks okay although I'd to understand more as to why tools might be generating the extra fields in this way. Also is the "zipinfo-time" property needed now? -Alan From xueming.shen at oracle.com Fri Jan 20 08:28:24 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 20 Jan 2017 00:28:24 -0800 Subject: JDK-8173072: zipfs fails to handle incorrect info-zip "extended timestamp extra field" In-Reply-To: References: <5881321A.9070105@oracle.com> <58814D40.8030804@oracle.com> Message-ID: <5881CA28.8010802@oracle.com> On 1/19/17, 11:44 PM, Alan Bateman wrote: > > > On 19/01/2017 23:35, Xueming Shen wrote: >> Thanks Claes! >> >> webrev has been updated accordingly >> >> http://cr.openjdk.java.net/~sherman/8173072/webrev >> > The changes looks okay although I'd to understand more as to why tools > might be generating the extra fields in this way. I'm reading info-zip ZIP file spec again. it turned out this might be a legitimate extra fields. The spec suggests the "flags" in BOTH headers indicate what are in the "local" extra field. So The flags in central header also indicates the local entry, not the cen entry at all, confirmed with info zip tool. ----------------------------- The lower three bits of Flags in both headers indicate which time- stamps are present in the LOCAL extra field: bit 0 if set, modification time is present bit 1 if set, access time is present bit 2 if set, creation time is present bits 3-7 reserved for additional timestamps; not set ------------------------------ It means we need to fix the ZipOutputStream to output the correct cen entry flags, if there are more extra timestamps. (jar does not create them) > > Also is the "zipinfo-time" property needed now? Yes, that property basically serves the purpose of helping performance. If you are interested at mtime ony and care about performance. The flag will help not go up to read the loc entry, even there are a/ctime entries there. -Sherman From Alan.Bateman at oracle.com Fri Jan 20 08:40:11 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 20 Jan 2017 08:40:11 +0000 Subject: JDK-8173072: zipfs fails to handle incorrect info-zip "extended timestamp extra field" In-Reply-To: <58814D40.8030804@oracle.com> References: <5881321A.9070105@oracle.com> <58814D40.8030804@oracle.com> Message-ID: <3b0ef252-d44c-dd92-2ce4-a12c8b174bf3@oracle.com> BTW: Do we need to create some test infrastructure to allow tests create specially crafted zip files? With changes like this then the implementation tolerates truncated extended timestamps but we don't have anything in the tests to be confident that such changes will keep working. -Alan From xueming.shen at oracle.com Fri Jan 20 08:46:49 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 20 Jan 2017 00:46:49 -0800 Subject: JDK-8173072: zipfs fails to handle incorrect info-zip "extended timestamp extra field" In-Reply-To: <3b0ef252-d44c-dd92-2ce4-a12c8b174bf3@oracle.com> References: <5881321A.9070105@oracle.com> <58814D40.8030804@oracle.com> <3b0ef252-d44c-dd92-2ce4-a12c8b174bf3@oracle.com> Message-ID: <5881CE79.70809@oracle.com> Can put in something together with the fix. Probably a variant of ZOS that remove those sanity check. -Sherman On 1/20/17, 12:40 AM, Alan Bateman wrote: > BTW: Do we need to create some test infrastructure to allow tests > create specially crafted zip files? With changes like this then the > implementation tolerates truncated extended timestamps but we don't > have anything in the tests to be confident that such changes will keep > working. > > -Alan > From xueming.shen at oracle.com Fri Jan 20 09:02:12 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 20 Jan 2017 01:02:12 -0800 Subject: JDK-8173072: zipfs fails to handle incorrect info-zip "extended timestamp extra field" In-Reply-To: <5881CA28.8010802@oracle.com> References: <5881321A.9070105@oracle.com> <58814D40.8030804@oracle.com> <5881CA28.8010802@oracle.com> Message-ID: <5881D214.8060701@oracle.com> On 1/20/17, 12:28 AM, Xueming Shen wrote: > > ----------------------------- > > The lower three bits of Flags in both headers indicate which > time- > stamps are present in the LOCAL extra field: > > bit 0 if set, modification time is present > bit 1 if set, access time is present > bit 2 if set, creation time is present > bits 3-7 reserved for additional timestamps; > not set > > ------------------------------ > > It means we need to fix the ZipOutputStream to output the correct cen > entry flags, if there > are more extra timestamps. (jar does not create them) > False alarm, It has been years since I wrote this part of the code :-) Just doubt checked the code, it appears both ZipOutputStream and zipfs.Entry.writeCEN() are implemented correctly to output the correct flags that indicate the timestamps in loc only. ZFS.Entry.writeCEN if (elenEXTT != 0) { writeShort(os, EXTID_EXTT); writeShort(os, elenEXTT - 4); if (ctime == -1) os.write(0x3); // mtime and atime else os.write(0x7); // mtime, atime and ctime writeInt(os, javaToUnixTime(mtime)); } ZOS.writeCEN { writeShort(EXTID_EXTT); if (e.mtime != null) { writeShort(5); // flag + mtime writeByte(flagEXTT); writeInt(umtime); } else { writeShort(1); // flag only writeByte(flagEXTT); } So the better fix for this one should simply not try to read a/ctime at all in cen reading code. -Sherman From xueming.shen at oracle.com Fri Jan 20 09:03:49 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 20 Jan 2017 01:03:49 -0800 Subject: JDK-8173072: zipfs fails to handle incorrect info-zip "extended timestamp extra field" Message-ID: <5881D275.1040404@oracle.com> On 1/20/17, 12:28 AM, Xueming Shen wrote: > > ----------------------------- > > The lower three bits of Flags in both headers indicate which > time- > stamps are present in the LOCAL extra field: > > bit 0 if set, modification time is present > bit 1 if set, access time is present > bit 2 if set, creation time is present > bits 3-7 reserved for additional timestamps; > not set > > ------------------------------ > > It means we need to fix the ZipOutputStream to output the correct cen > entry flags, if there > are more extra timestamps. (jar does not create them) > False alarm, It has been years since I wrote this part of the code Just doubt checked the code, it appears both ZipOutputStream and zipfs.Entry.writeCEN() are implemented correctly to output the correct flags that indicate the timestamps in loc only. ZFS.Entry.writeCEN if (elenEXTT != 0) { writeShort(os, EXTID_EXTT); writeShort(os, elenEXTT - 4); if (ctime == -1) os.write(0x3); // mtime and atime else os.write(0x7); // mtime, atime and ctime writeInt(os, javaToUnixTime(mtime)); } ZOS.writeCEN { writeShort(EXTID_EXTT); if (e.mtime != null) { writeShort(5); // flag + mtime writeByte(flagEXTT); writeInt(umtime); } else { writeShort(1); // flag only writeByte(flagEXTT); } So the better fix for this one should simply not try to read a/ctime at all in cen reading code. -Sherman From christoph.langer at sap.com Mon Jan 23 10:54:27 2017 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 23 Jan 2017 10:54:27 +0000 Subject: RFR (XXS) 8173197: Change for 8172547 breaks builds with VS2010 Message-ID: Hi, can I please get a review for the following trivial fix: --- a/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c Mon Jan 23 10:35:30 2017 +0100 +++ b/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c Mon Jan 23 11:47:03 2017 +0100 @@ -75,8 +75,8 @@ } else if (timeout < 0) { tv = NULL; } else { + jlong sec = timeout / 1000; tv = &timevalue; - jlong sec = timeout / 1000; // // struct timeval members are signed 32-bit integers so the // signed 64-bit jlong needs to be clamped The fix for 8172547 [1] introduces code that violates C98 standard and causes a compile error with Mircosoft VS 2010 because the variable declaration is not done at the beginning of its scope. I oversaw that myself when reviewing the change. Thanks Christoph [1]: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/abc51aa40c7e -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Mon Jan 23 10:57:16 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 23 Jan 2017 10:57:16 +0000 Subject: RFR (XXS) 8173197: Change for 8172547 breaks builds with VS2010 In-Reply-To: References: Message-ID: Looks good Christoph, thanks for fixing on this. -Chris. On 23/01/17 10:54, Langer, Christoph wrote: > Hi, > > > > can I please get a review for the following trivial fix: > > > > --- a/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c > Mon Jan 23 10:35:30 2017 +0100 > > +++ b/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c > Mon Jan 23 11:47:03 2017 +0100 > > @@ -75,8 +75,8 @@ > > } else if (timeout < 0) { > > tv = NULL; > > } else { > > + jlong sec = timeout / 1000; > > tv = &timevalue; > > - jlong sec = timeout / 1000; > > // > > // struct timeval members are signed 32-bit integers so the > > // signed 64-bit jlong needs to be clamped > > > > The fix for 8172547 [1] introduces code that violates C98 standard and > causes a compile error with Mircosoft VS 2010 because the variable > declaration is not done at the beginning of its scope. I oversaw that > myself when reviewing the change. > > > > Thanks > > Christoph > > > > [1]: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/abc51aa40c7e > > > From Alan.Bateman at oracle.com Mon Jan 23 10:59:27 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 23 Jan 2017 10:59:27 +0000 Subject: RFR (XXS) 8173197: Change for 8172547 breaks builds with VS2010 In-Reply-To: References: Message-ID: <370c7938-a73f-cf1e-34dd-61673c015130@oracle.com> On 23/01/2017 10:54, Langer, Christoph wrote: > Hi, > > can I please get a review for the following trivial fix: > > --- a/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c Mon > Jan 23 10:35:30 2017 +0100 > > +++ b/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c Mon > Jan 23 11:47:03 2017 +0100 > > @@ -75,8 +75,8 @@ > > } else if (timeout < 0) { > > tv = NULL; > > } else { > > + jlong sec = timeout / 1000; > > tv = &timevalue; > > - jlong sec = timeout / 1000; > > // > > // struct timeval members are signed 32-bit integers so the > > // signed 64-bit jlong needs to be clamped > > The fix for 8172547 [1] introduces code that violates C98 standard and > causes a compile error with Mircosoft VS 2010 because the variable > declaration is not done at the beginning of its scope. I oversaw that > myself when reviewing the change. > > This looks okay although the oldest version of VisualStudio listed in the build README is VS2013. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Mon Jan 23 11:02:44 2017 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 23 Jan 2017 11:02:44 +0000 Subject: RFR (XXS) 8173197: Change for 8172547 breaks builds with VS2010 In-Reply-To: <370c7938-a73f-cf1e-34dd-61673c015130@oracle.com> References: <370c7938-a73f-cf1e-34dd-61673c015130@oracle.com> Message-ID: <462dc629e655435e938c520d23585420@derote13de46.global.corp.sap> Thanks, Alan, for the quick response. We know that - however we have some builds which still run with VS2010 and as long as it is so simple to keep compatibility, I think we should try to maintain it. Best regards Christoph From: Alan Bateman [mailto:Alan.Bateman at oracle.com] Sent: Montag, 23. Januar 2017 11:59 To: Langer, Christoph ; nio-dev at openjdk.java.net Subject: Re: RFR (XXS) 8173197: Change for 8172547 breaks builds with VS2010 On 23/01/2017 10:54, Langer, Christoph wrote: Hi, can I please get a review for the following trivial fix: --- a/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c Mon Jan 23 10:35:30 2017 +0100 +++ b/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c Mon Jan 23 11:47:03 2017 +0100 @@ -75,8 +75,8 @@ } else if (timeout < 0) { tv = NULL; } else { + jlong sec = timeout / 1000; tv = &timevalue; - jlong sec = timeout / 1000; // // struct timeval members are signed 32-bit integers so the // signed 64-bit jlong needs to be clamped The fix for 8172547 [1] introduces code that violates C98 standard and causes a compile error with Mircosoft VS 2010 because the variable declaration is not done at the beginning of its scope. I oversaw that myself when reviewing the change. This looks okay although the oldest version of VisualStudio listed in the build README is VS2013. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Mon Jan 23 13:11:30 2017 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 23 Jan 2017 13:11:30 +0000 Subject: RFR (XXS) 8173197: Change for 8172547 breaks builds with VS2010 In-Reply-To: References: Message-ID: <36418a5f55864593a09cac2ff5087dc4@derote13de46.global.corp.sap> Thanks, Chris. It's checked in: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/fb36a29be4a3 > -----Original Message----- > From: Chris Hegarty [mailto:chris.hegarty at oracle.com] > Sent: Montag, 23. Januar 2017 11:57 > To: Langer, Christoph ; nio-dev at openjdk.java.net > Subject: Re: RFR (XXS) 8173197: Change for 8172547 breaks builds with VS2010 > > Looks good Christoph, thanks for fixing on this. > > -Chris. > > > On 23/01/17 10:54, Langer, Christoph wrote: > > Hi, > > > > > > > > can I please get a review for the following trivial fix: > > > > > > > > --- a/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c > > Mon Jan 23 10:35:30 2017 +0100 > > > > +++ b/src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c > > Mon Jan 23 11:47:03 2017 +0100 > > > > @@ -75,8 +75,8 @@ > > > > } else if (timeout < 0) { > > > > tv = NULL; > > > > } else { > > > > + jlong sec = timeout / 1000; > > > > tv = &timevalue; > > > > - jlong sec = timeout / 1000; > > > > // > > > > // struct timeval members are signed 32-bit integers so the > > > > // signed 64-bit jlong needs to be clamped > > > > > > > > The fix for 8172547 [1] introduces code that violates C98 standard and > > causes a compile error with Mircosoft VS 2010 because the variable > > declaration is not done at the beginning of its scope. I oversaw that > > myself when reviewing the change. > > > > > > > > Thanks > > > > Christoph > > > > > > > > [1]: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/abc51aa40c7e > > > > > > From brian.burkhalter at oracle.com Thu Jan 26 20:37:44 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 26 Jan 2017 12:37:44 -0800 Subject: JDK 9 RFR of 8152085: (ch) RandomAccessFile.getChannel().lock() is uninterruptible Message-ID: Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8152085 Patch: http://cr.openjdk.java.net/~bpb/8152085/webrev.00/ On Windows, when a file is locked by one process via a FileChannel, and a second process attempts a blocking lock on the same file via a FileChannel, then the thread blocking on the lock call cannot be interrupted. This patch fixes the problem by replacing the blocking lock call with polling a non-blocking lock call. I am not at all sure that this is a good solution, so should someone else have a better idea about it that they would like to share I would be happy to read it. Thanks, Brian From Alan.Bateman at oracle.com Thu Jan 26 21:50:58 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 26 Jan 2017 21:50:58 +0000 Subject: JDK 9 RFR of 8152085: (ch) RandomAccessFile.getChannel().lock() is uninterruptible In-Reply-To: References: Message-ID: On 26/01/2017 20:37, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8152085 > Patch: http://cr.openjdk.java.net/~bpb/8152085/webrev.00/ > > On Windows, when a file is locked by one process via a FileChannel, and a second process attempts a blocking lock on the same file via a FileChannel, then the thread blocking on the lock call cannot be interrupted. This patch fixes the problem by replacing the blocking lock call with polling a non-blocking lock call. I am not at all sure that this is a good solution, so should someone else have a better idea about it that they would like to share I would be happy to read it. > > Yeah, this is gross. This issue dates back to JDK 1.4 and I think it needs further thought to see if are other APIs that we could use. -Alan From brian.burkhalter at oracle.com Fri Jan 27 01:12:44 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 26 Jan 2017 17:12:44 -0800 Subject: JDK 9 RFR of 8152085: (ch) RandomAccessFile.getChannel().lock() is uninterruptible In-Reply-To: References: Message-ID: On Jan 26, 2017, at 1:50 PM, Alan Bateman wrote: >> I am not at all sure that this is a good solution, so should someone else have a better idea about it that they would like to share I would be happy to read it. >> >> > Yeah, this is gross. This issue dates back to JDK 1.4 and I think it needs further thought to see if are other APIs that we could use. Well as I implied I don?t think it?s a great solution either so I don?t disagree. I have looked over a lot of documentation of Windows functions and did not see any alternative there. I?ll continue to investigate. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: