From xueming.shen at oracle.com Thu Apr 11 13:47:01 2013 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 11 Apr 2013 13:47:01 -0700 Subject: Codereview Request: 8011647: Add java.time.Instant methods to java.nio.file.attribute.FileTime Message-ID: <51672145.3080504@oracle.com> Hi As part of the JSR310 Date/Time project, following methods are proposed to be added into java.nio.file.attribute.FileTime to interoperate with the new JSR310 time class Instant. public static FileTime from(Instant instant); public Instant toInstant(); http://cr.openjdk.java.net/~sherman/8011647/webrev Here are some notes regarding the changes. (1) With its design, the FileTime can store the time point on the time-line further in the future and further in the past than the Instant. To be consistent with the spec and implementation of existing to(TimeUnit)/ toMillis(), the proposed toInstant() saturates to the Instant.MIN/MAX when converting from a FileTime object that is beyond the supported time range of Instant (instead of throwing a "conversion failure" exception, such as a DateTimeException). Time points beyond the Instant range are not interesting in practical cases. (2) The internal supporting class DaysAndNanos is replaced by using the threeten Instant class directly. As the direct consequence, the hashCode() for those time points beyond the Instant.MIN/MAX range will be "compromised/saturated" into the hashCode value of Instant.MIN/MAX. As mentioned in (1), since those time points are not the interest of the FileTime class in real world use scenario, so we believe this is an acceptable compromise. The hashCode/equal contract still holds, though the usages of those "extra time points" may trigger worse performance. (3) compareTo() still provides correct ordering for all supported "time points", including those beyond Instant.MIN/MAX range. (4) toString() has been rewritten to use the threeten LocalDateTime class together with the "home-made" formatting code for better performance. Differences of the spec/implementation of FileTime and JSR310 date/time class listed below prevents us from using the new JSR310 formatter to format the FileTime directly. a) FileTime uses "old" ISO8601 version, in which it dis-allows the '0000' for proleptic year 0, while the JSR310 formats "0000" (which is supported in new version of 8601), so the format for all BCE years are different. b) LocalTime formats the "ss" part optionally. It appears to takes the "it is acceptable to omit lower order time elements..." alternative, while the "ss" part is mandated in FileTime toString(), even the second part is 0. c) FileTime formats the "fraction of second" part as "decimal fraction" while LocalTime uses SSS, SSSSSS or SSSSSSSSS pattern. So the FT strips any tailing zero, but LocalDate does not if it fits into the 3-dit unit. Two addition tests have been used to provide more confidence for the "compatibility" and performance. http://cr.openjdk.java.net/~sherman/8011647/TestFileTime.java http://cr.openjdk.java.net/~sherman/8011647/TestFileTimeBM.java It is also suggested that it might be convenient for FileTime to implement java.time.TemporalAccessor, so the FileTime can be directly formatted by the JSR310 formatter. This is not included in this proposal. I may consider to add this in a later round of update. Thanks, -Sherman From Alan.Bateman at oracle.com Fri Apr 12 05:55:36 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 12 Apr 2013 13:55:36 +0100 Subject: Codereview Request: 8011647: Add java.time.Instant methods to java.nio.file.attribute.FileTime In-Reply-To: <51672145.3080504@oracle.com> References: <51672145.3080504@oracle.com> Message-ID: <51680448.6020303@oracle.com> On 11/04/2013 21:47, Xueming Shen wrote: > Hi > > As part of the JSR310 Date/Time project, following methods are proposed > to be added into java.nio.file.attribute.FileTime to interoperate with > the > new JSR310 time class Instant. > > public static FileTime from(Instant instant); > public Instant toInstant(); > > http://cr.openjdk.java.net/~sherman/8011647/webrev Sherman and I chatted about adding from(Instant)/toInstant() a few times over the last week so most of my comments are already included. Overall I think it has worked out well, just unfortunate that there is a bit of extra complexity because FileTime supports a wider time-line and we also went with the XML schema deviation from (older?) ISO 8601 for year 0000. In practical terms these aren't of course interesting for file times. In terms of API then from(Instant)/toInstant() are simple and I don't think we have an urgent need for FileTimes to be directly formatted. On toString then FileTime doesn't require that trailing zeros be stripped from the fraction of a second. If it is better to use 3-digit units that it is okay. Also if this is something that we got wrong in FileTime then we could change the spec without any compatibility impact. Values outside of MIN/MAX aren't too interesting so hashCode using Instant hashCode should be fine. I had to read compareTo a few times to convince myself that it correctly orders FileTimes where one is created from an Instant and the other from a value outside of MIN/MAX. It might be useful to expand the comment to explain this further. Thanks for expanding the existing test. A minor point at line 117 is that you don't need "Random r" as there is rand already. The pre-existing tests loop over TimeUnit.values() rather than EnumSet.allOf(TimeUnit.class). In eqTime then it prints to System.out where the rest of the test prints to System.err before throwing the exception. That's all I have. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130412/74017318/attachment.html From xueming.shen at oracle.com Fri Apr 12 10:28:45 2013 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 12 Apr 2013 10:28:45 -0700 Subject: Codereview Request: 8011647: Add java.time.Instant methods to java.nio.file.attribute.FileTime In-Reply-To: <51680448.6020303@oracle.com> References: <51672145.3080504@oracle.com> <51680448.6020303@oracle.com> Message-ID: <5168444D.10003@oracle.com> On 04/12/2013 05:55 AM, Alan Bateman wrote: > Sherman and I chatted about adding from(Instant)/toInstant() a few times over the last week so most of my comments are already included. > > Overall I think it has worked out well, just unfortunate that there is a bit of extra complexity because FileTime supports a wider time-line and we also went with the XML schema deviation from (older?) ISO 8601 for year 0000. In practical terms these aren't of course interesting for file times. > > In terms of API then from(Instant)/toInstant() are simple and I don't think we have an urgent need for FileTimes to be directly formatted. > > On toString then FileTime doesn't require that trailing zeros be stripped from the fraction of a second. If it is better to use 3-digit units that it is okay. Also if this is something that we got wrong in FileTime then we could change the spec without any compatibility impact. I'm not sure if the 3-digit unit fitting is "better" or not. But xml spec cited in the toString() API specifies that the "trailing zeros are optional" in its 3.2.3.1 "lexical representatin" section and "trailing zeros are prohibited..." in 3.2.3.2 "canonical representation" section. So I think trimming trailing zeros in FileTime.toString() is just fine. http://www.w3.org/TR/xmlschema-2/#deviantformats > > Values outside of MIN/MAX aren't too interesting so hashCode using Instant hashCode should be fine. I had to read compareTo a few times to convince myself that it correctly orders FileTimes where one is created from an Instant and the other from a value outside of MIN/MAX. It might be useful to expand the comment to explain this further. Added some wording. > > Thanks for expanding the existing test. A minor point at line 117 is that you don't need "Random r" as there is rand already. The pre-existing tests loop over TimeUnit.values() rather than EnumSet.allOf(TimeUnit.class). In eqTime then it prints to System.out where the rest of the test prints to System.err before throwing the exception. > Updated. http://cr.openjdk.java.net/~sherman/8011647/webrev/ Thanks! -Sherman -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130412/16fce602/attachment.html From Alan.Bateman at oracle.com Fri Apr 12 11:10:44 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 12 Apr 2013 19:10:44 +0100 Subject: Codereview Request: 8011647: Add java.time.Instant methods to java.nio.file.attribute.FileTime In-Reply-To: <5168444D.10003@oracle.com> References: <51672145.3080504@oracle.com> <51680448.6020303@oracle.com> <5168444D.10003@oracle.com> Message-ID: <51684E24.2090405@oracle.com> On 12/04/2013 18:28, Xueming Shen wrote: > > I'm not sure if the 3-digit unit fitting is "better" or not. But xml > spec cited in the > toString() API specifies that the "trailing zeros are optional" in its > 3.2.3.1 "lexical > representatin" section and "trailing zeros are prohibited..." in > 3.2.3.2 "canonical > representation" section. So I think trimming trailing zeros in > FileTime.toString() > is just fine. The deviation and reference to the XML scheme spec was intended to be deal with the issues of year 0000 and > 9999. I'm happy with dropping the trailing zeros, I just wasn't sure if you suggesting that it would have been better to use 3-digit unit that allow trailing zeros. > : > Updated. > > http://cr.openjdk.java.net/~sherman/8011647/webrev/ > Looks fine. The eqTime method is still printing to System.out in the test but it's not a big deal. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130412/829a8933/attachment.html From xueming.shen at oracle.com Fri Apr 12 11:16:27 2013 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 12 Apr 2013 11:16:27 -0700 Subject: Codereview Request: 8011647: Add java.time.Instant methods to java.nio.file.attribute.FileTime In-Reply-To: <51684E24.2090405@oracle.com> References: <51672145.3080504@oracle.com> <51680448.6020303@oracle.com> <5168444D.10003@oracle.com> <51684E24.2090405@oracle.com> Message-ID: <51684F7B.7000308@oracle.com> On 04/12/2013 11:10 AM, Alan Bateman wrote: > On 12/04/2013 18:28, Xueming Shen wrote: >> >> I'm not sure if the 3-digit unit fitting is "better" or not. But xml spec cited in the >> toString() API specifies that the "trailing zeros are optional" in its 3.2.3.1 "lexical >> representatin" section and "trailing zeros are prohibited..." in 3.2.3.2 "canonical >> representation" section. So I think trimming trailing zeros in FileTime.toString() >> is just fine. > The deviation and reference to the XML scheme spec was intended to be deal with the issues of year 0000 and > 9999. I'm happy with dropping the trailing zeros, I just wasn't sure if you suggesting that it would have been better to use 3-digit unit that allow trailing zeros. > > >> : >> Updated. >> >> http://cr.openjdk.java.net/~sherman/8011647/webrev/ >> > Looks fine. The eqTime method is still printing to System.out in the test but it's not a big deal. > http://cr.openjdk.java.net/~sherman/8011647/webrev/ Missed this one. Updated. I think we are good to go. Thanks! -Sherman -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130412/3e20e200/attachment-0001.html From Alan.Bateman at oracle.com Mon Apr 15 06:05:45 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 15 Apr 2013 14:05:45 +0100 Subject: 8011536: (fs) BasicFileAttributes.creationTime() should return birth time (mac) Message-ID: <516BFB29.3060801@oracle.com> One of the things left over from the Mac port is to return the st_birthtime as the creationFile attribute. This can be exposed in the stat structure on 64-bit builds. As part of this, I've cleaned-up the handling of optional sys calls as we were missing a way to indicate if futimes (or futimesat) is supported or not. The webrev with the proposed changes is here: http://cr.openjdk.java.net/~alanb/8011536/webrev/ -Alan. From Alan.Bateman at oracle.com Wed Apr 17 05:05:07 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 Apr 2013 13:05:07 +0100 Subject: 8012019: (fs) Thread.interrupt triggers hang in FileChannelImpl.pread (win) Message-ID: <516E8FF3.1050903@oracle.com> The FileChannel read/write methods that take a position are currently synchronized on Windows due ReadFile/WriteFile changing the global file position. This is something that will need to be looked again because it is a long standing (since jdk1.4) scalability issue. In the mean-time, it turns out that there is potential for deadlock on Windows when a position sensitive method is interrupted at around the time that a position insensitive method is executing. It's been there for some time but only reported recently. The problem is that a position sensitive method has the positionLock and if interrupted will close the channel causing it to wait until all threads have finished their I/O operations. With position insensitive methods then the positionLock is acquired as part of the I/O operation, hence the deadlock. The fix is very simple and just changes the two FileChannel methods to acquire positionLock (if needed) before doing the I/O operations. This means dropping the lock parameter from the IOUtil read/write methods so there are more files changed that might be obvious. The webrev with the changes is here: http://cr.openjdk.java.net/~alanb/8012019/webrev/ -Alan. From chris.hegarty at oracle.com Wed Apr 17 07:39:38 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 17 Apr 2013 15:39:38 +0100 Subject: 8012019: (fs) Thread.interrupt triggers hang in FileChannelImpl.pread (win) In-Reply-To: <516E8FF3.1050903@oracle.com> References: <516E8FF3.1050903@oracle.com> Message-ID: <516EB42A.70408@oracle.com> Nice fix Alan, and it really cleans up the commonly unused and confusing lock argument. -Chris. On 17/04/2013 13:05, Alan Bateman wrote: > > The FileChannel read/write methods that take a position are currently > synchronized on Windows due ReadFile/WriteFile changing the global file > position. This is something that will need to be looked again because it > is a long standing (since jdk1.4) scalability issue. > > In the mean-time, it turns out that there is potential for deadlock on > Windows when a position sensitive method is interrupted at around the > time that a position insensitive method is executing. It's been there > for some time but only reported recently. The problem is that a position > sensitive method has the positionLock and if interrupted will close the > channel causing it to wait until all threads have finished their I/O > operations. With position insensitive methods then the positionLock is > acquired as part of the I/O operation, hence the deadlock. > > The fix is very simple and just changes the two FileChannel methods to > acquire positionLock (if needed) before doing the I/O operations. This > means dropping the lock parameter from the IOUtil read/write methods so > there are more files changed that might be obvious. The webrev with the > changes is here: > > http://cr.openjdk.java.net/~alanb/8012019/webrev/ > > -Alan. From chris.hegarty at oracle.com Wed Apr 17 13:00:39 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 17 Apr 2013 21:00:39 +0100 Subject: 8011536: (fs) BasicFileAttributes.creationTime() should return birth time (mac) In-Reply-To: <516BFB29.3060801@oracle.com> References: <516BFB29.3060801@oracle.com> Message-ID: <516EFF67.4000202@oracle.com> Looks fine to me Alan. Trivially, the change to the example in the javadoc for BasicFileAttributeView. Should this be setTimes(null, time, *null*) ?? -Chris. On 15/04/2013 14:05, Alan Bateman wrote: > > One of the things left over from the Mac port is to return the > st_birthtime as the creationFile attribute. This can be exposed in the > stat structure on 64-bit builds. As part of this, I've cleaned-up the > handling of optional sys calls as we were missing a way to indicate if > futimes (or futimesat) is supported or not. > > The webrev with the proposed changes is here: > > http://cr.openjdk.java.net/~alanb/8011536/webrev/ > > -Alan. From Alan.Bateman at oracle.com Thu Apr 18 01:39:17 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 Apr 2013 09:39:17 +0100 Subject: 8011536: (fs) BasicFileAttributes.creationTime() should return birth time (mac) In-Reply-To: <516EFF67.4000202@oracle.com> References: <516BFB29.3060801@oracle.com> <516EFF67.4000202@oracle.com> Message-ID: <516FB135.5060602@oracle.com> On 17/04/2013 21:00, Chris Hegarty wrote: > Looks fine to me Alan. > > Trivially, the change to the example in the javadoc for > BasicFileAttributeView. Should this be setTimes(null, time, *null*) ?? Thanks Chris, I noticed that too after I generated the webrev. I'll fix it before I push. -Alan. From Alan.Bateman at oracle.com Tue Apr 23 02:12:47 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 23 Apr 2013 10:12:47 +0100 Subject: 8012930: (fs) Eliminate recursion from FileTreeWalker Message-ID: <5176508F.5080500@oracle.com> This patch eliminates the recursion from the FileTreeWalker utility class used by Files.walkFileTree, replacing it with its own walking stack. While stack overflow hasn't been a problem, it can be mildly annoying to see a lot of walk methods in the stack trace when an exception occurs when deep in the file system. The other motivation of course is that the events make it more generally useful, particularly with the proposed API additions that flatten the nodes to a stream. With the change then FileTreeWalker will simply provide a sequence of events corresponding to the files in the tree; the dispatching to FileVisitor methods is moved to Files.walkFileTree. The other thing in this patch is splitting out of the tests from the walk_file_tree.sh test. This shell test was originally intended to test walkFileTree against the native "find" program but it ended up running several other tests. It also turns out that SKIP_SUBTREE wasn't being tested so I've added a simple test for that. The webrev with the changes is here: http://cr.openjdk.java.net/~alanb/8012930/webrev Thanks, -Alan. From chris.hegarty at oracle.com Tue Apr 23 06:35:26 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 23 Apr 2013 14:35:26 +0100 Subject: 8012930: (fs) Eliminate recursion from FileTreeWalker In-Reply-To: <5176508F.5080500@oracle.com> References: <5176508F.5080500@oracle.com> Message-ID: <51768E1E.5030004@oracle.com> Wow Alan, very nice. The webrev is a little difficult to see what has changed in FileTreeWalker because some parts have moved, but nothing jumps out at me. The additional test, and the moving of other tests out from under a script is welcome. -Chris. On 04/23/2013 10:12 AM, Alan Bateman wrote: > > This patch eliminates the recursion from the FileTreeWalker utility > class used by Files.walkFileTree, replacing it with its own walking > stack. While stack overflow hasn't been a problem, it can be mildly > annoying to see a lot of walk methods in the stack trace when an > exception occurs when deep in the file system. The other motivation of > course is that the events make it more generally useful, particularly > with the proposed API additions that flatten the nodes to a stream. With > the change then FileTreeWalker will simply provide a sequence of events > corresponding to the files in the tree; the dispatching to FileVisitor > methods is moved to Files.walkFileTree. > > The other thing in this patch is splitting out of the tests from the > walk_file_tree.sh test. This shell test was originally intended to test > walkFileTree against the native "find" program but it ended up running > several other tests. It also turns out that SKIP_SUBTREE wasn't being > tested so I've added a simple test for that. > > The webrev with the changes is here: > > http://cr.openjdk.java.net/~alanb/8012930/webrev > > Thanks, > > -Alan. From Alan.Bateman at oracle.com Tue Apr 23 07:09:10 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 23 Apr 2013 15:09:10 +0100 Subject: 8012930: (fs) Eliminate recursion from FileTreeWalker In-Reply-To: <51768E1E.5030004@oracle.com> References: <5176508F.5080500@oracle.com> <51768E1E.5030004@oracle.com> Message-ID: <51769606.2060309@oracle.com> On 23/04/2013 14:35, Chris Hegarty wrote: > : > > The webrev is a little difficult to see what has changed in > FileTreeWalker because some parts have moved, but nothing jumps out at > me. Thanks for going through it. Overall the changes are relatively simple, it looks more than it is because code has moved around. -Alan. From Alan.Bateman at oracle.com Tue Apr 30 12:18:51 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 30 Apr 2013 20:18:51 +0100 Subject: 8013647: JPRT unable to clean-up after tests that leave file trees with loops Message-ID: <5180191B.1080801@oracle.com> As part of 8012930 [1], the tests for Files.walkFileTree were split out into individual tests so they aren't run by a shell tests. A side effect of this is that the tests leave a file tree that is problematic to clean-up because it contains loops (sym links to ancestor directories). The following webrev "fixes" these four tests so that the clean up after themselves: http://cr.openjdk.java.net/~alanb/8013647/webrev/ Tim and Kumar are familiar with the issue and may be able to review. -Alan [1] http://hg.openjdk.java.net/jdk8/tl/jdk/rev/8b07b318f713 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130430/d057eabd/attachment.html From chris.hegarty at oracle.com Tue Apr 30 12:32:39 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 30 Apr 2013 20:32:39 +0100 Subject: 8013647: JPRT unable to clean-up after tests that leave file trees with loops In-Reply-To: <5180191B.1080801@oracle.com> References: <5180191B.1080801@oracle.com> Message-ID: <51801C57.9010006@oracle.com> Using TestUtil.removeAll to do the clean-up should take care of the issue. Looks fine to me. -Chris. On 04/30/2013 08:18 PM, Alan Bateman wrote: > > As part of 8012930 [1], the tests for Files.walkFileTree were split out > into individual tests so they aren't run by a shell tests. A side effect > of this is that the tests leave a file tree that is problematic to > clean-up because it contains loops (sym links to ancestor directories). > > The following webrev "fixes" these four tests so that the clean up after > themselves: > > http://cr.openjdk.java.net/~alanb/8013647/webrev/ > > Tim and Kumar are familiar with the issue and may be able to review. > > -Alan > > [1] http://hg.openjdk.java.net/jdk8/tl/jdk/rev/8b07b318f713 From tim.bell at oracle.com Tue Apr 30 13:12:08 2013 From: tim.bell at oracle.com (Tim Bell) Date: Tue, 30 Apr 2013 13:12:08 -0700 Subject: 8013647: JPRT unable to clean-up after tests that leave file trees with loops In-Reply-To: <51801C57.9010006@oracle.com> References: <5180191B.1080801@oracle.com> <51801C57.9010006@oracle.com> Message-ID: <51802598.4010300@oracle.com> Alan- Looks good to me. Thanks- Tim On 04/30/13 12:32 PM, Chris Hegarty wrote: > Using TestUtil.removeAll to do the clean-up should take care of the > issue. Looks fine to me. > > -Chris. > > On 04/30/2013 08:18 PM, Alan Bateman wrote: >> >> As part of 8012930 [1], the tests for Files.walkFileTree were split out >> into individual tests so they aren't run by a shell tests. A side effect >> of this is that the tests leave a file tree that is problematic to >> clean-up because it contains loops (sym links to ancestor directories). >> >> The following webrev "fixes" these four tests so that the clean up after >> themselves: >> >> http://cr.openjdk.java.net/~alanb/8013647/webrev/ >> >> Tim and Kumar are familiar with the issue and may be able to review. >> >> -Alan >> >> [1] http://hg.openjdk.java.net/jdk8/tl/jdk/rev/8b07b318f713 From kumar.x.srinivasan at oracle.com Tue Apr 30 13:18:22 2013 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Tue, 30 Apr 2013 13:18:22 -0700 Subject: 8013647: JPRT unable to clean-up after tests that leave file trees with loops In-Reply-To: <5180191B.1080801@oracle.com> References: <5180191B.1080801@oracle.com> Message-ID: <5180270E.9030107@oracle.com> Looks good. Kumar > > As part of 8012930 [1], the tests for Files.walkFileTree were split > out into individual tests so they aren't run by a shell tests. A side > effect of this is that the tests leave a file tree that is problematic > to clean-up because it contains loops (sym links to ancestor directories). > > The following webrev "fixes" these four tests so that the clean up > after themselves: > > http://cr.openjdk.java.net/~alanb/8013647/webrev/ > > Tim and Kumar are familiar with the issue and may be able to review. > > -Alan > > [1] http://hg.openjdk.java.net/jdk8/tl/jdk/rev/8b07b318f713 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130430/aa23bde6/attachment.html