From rob.mckenna at oracle.com Wed Oct 2 12:14:51 2013 From: rob.mckenna at oracle.com (Rob McKenna) Date: Wed, 02 Oct 2013 20:14:51 +0100 Subject: Request for review / feedback: SocketChannel.poll() Message-ID: <524C70AB.9080108@oracle.com> Hi folks, After adding Net.poll() in 7184932, Alan suggested I implement a poll method in SocketChannel to allow for the temporary use of non-blocking SocketChannels in a blocking fashion. So this is my first public effort and is likely quite rough around the edges. Apologies for the rushed approach, but I'd really like to gather feedback as soon as possible as we're running quite late for 8. (its been a busy couple of months unfortunately) http://cr.openjdk.java.net/~robm/sc.poll/webrev.01/ One thing to note is that in the implementation of poll we have 3 calls to Net.poll() within 3 conditionals. The reason for this is to allow for simultaneous calls to sc.poll(OP_READ) and sc.poll(OP_WRITE). Another part in particular that I'd like to highligh is the awkward timeout. The spec does say that this method will block if (e.g.) a read operation is performed in another thread and we poll for OP_READ in our thread. That is to say, the timeout only comes into play once we have the appropriate locks. I've also followed the poll system calls semantics for the timeout parameter. (<0: block indefinitely, 0: return immediately, >0: block for specified interval) The test is still a work in progess, but any feedback at all would be gratefully received. -Rob From david.lloyd at redhat.com Wed Oct 2 14:03:38 2013 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 02 Oct 2013 16:03:38 -0500 Subject: Request for review / feedback: SocketChannel.poll() In-Reply-To: <524C70AB.9080108@oracle.com> References: <524C70AB.9080108@oracle.com> Message-ID: <524C8A2A.8000006@redhat.com> On 10/02/2013 02:14 PM, Rob McKenna wrote: > Hi folks, > > After adding Net.poll() in 7184932, Alan suggested I implement a poll > method in SocketChannel to allow for the temporary use of non-blocking > SocketChannels in a blocking fashion. So this is my first public effort > and is likely quite rough around the edges. > > Apologies for the rushed approach, but I'd really like to gather > feedback as soon as possible as we're running quite late for 8. (its > been a busy couple of months unfortunately) > > http://cr.openjdk.java.net/~robm/sc.poll/webrev.01/ > > One thing to note is that in the implementation of poll we have 3 calls > to Net.poll() within 3 conditionals. The reason for this is to allow for > simultaneous calls to sc.poll(OP_READ) and sc.poll(OP_WRITE). > > Another part in particular that I'd like to highligh is the awkward > timeout. The spec does say that this method will block if (e.g.) a read > operation is performed in another thread and we poll for OP_READ in our > thread. That is to say, the timeout only comes into play once we have > the appropriate locks. I've also followed the poll system calls > semantics for the timeout parameter. (<0: block indefinitely, 0: return > immediately, >0: block for specified interval) > > The test is still a work in progess, but any feedback at all would be > gratefully received. I am not an official reviewer. There are a couple issues with the JavaDoc: 1. "...{@code OP_CONNECT} then the poll may not proceed concurrently with either a read or write operation" - no trailing period 2. @param/@return/@throws should probably be consistent with capitalization (@throws phrases start with caps but others do not; probably no phrases should, though maybe this is some weird convention you guys have) A couple semantic issues: 1. Thread interruption should stop the poll, but absolutely should not *not* close the channel - that was a bad idea at its inception and it is still a bad idea now, and people who do not want this behavior (== sane people) will still be stuck with using temporary selectors. It is up to the application to decide what thread interruption means. InterruptedIOException is more appropriate, and the user can elect to close the channel if they feel it is appropriate to do so. Inconsistency is worthless if the existing behavior does not make sense. 2. Not a fan of the triple-behavior-overload of the timeout parameter. -- - DML From david.lloyd at redhat.com Wed Oct 2 14:06:11 2013 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 02 Oct 2013 16:06:11 -0500 Subject: Request for review / feedback: SocketChannel.poll() In-Reply-To: <524C8A2A.8000006@redhat.com> References: <524C70AB.9080108@oracle.com> <524C8A2A.8000006@redhat.com> Message-ID: <524C8AC3.5040800@redhat.com> On 10/02/2013 04:03 PM, David M. Lloyd wrote: > On 10/02/2013 02:14 PM, Rob McKenna wrote: >> Hi folks, >> >> After adding Net.poll() in 7184932, Alan suggested I implement a poll >> method in SocketChannel to allow for the temporary use of non-blocking >> SocketChannels in a blocking fashion. So this is my first public effort >> and is likely quite rough around the edges. >> >> Apologies for the rushed approach, but I'd really like to gather >> feedback as soon as possible as we're running quite late for 8. (its >> been a busy couple of months unfortunately) >> >> http://cr.openjdk.java.net/~robm/sc.poll/webrev.01/ >> >> One thing to note is that in the implementation of poll we have 3 calls >> to Net.poll() within 3 conditionals. The reason for this is to allow for >> simultaneous calls to sc.poll(OP_READ) and sc.poll(OP_WRITE). >> >> Another part in particular that I'd like to highligh is the awkward >> timeout. The spec does say that this method will block if (e.g.) a read >> operation is performed in another thread and we poll for OP_READ in our >> thread. That is to say, the timeout only comes into play once we have >> the appropriate locks. I've also followed the poll system calls >> semantics for the timeout parameter. (<0: block indefinitely, 0: return >> immediately, >0: block for specified interval) >> >> The test is still a work in progess, but any feedback at all would be >> gratefully received. > > I am not an official reviewer. > > There are a couple issues with the JavaDoc: > > 1. "...{@code OP_CONNECT} then the poll may not proceed concurrently > with either a read or write operation" - no trailing period > 2. @param/@return/@throws should probably be consistent with > capitalization (@throws phrases start with caps but others do not; > probably no phrases should, though maybe this is some weird convention > you guys have) > > A couple semantic issues: > > 1. Thread interruption should stop the poll, but absolutely should not > *not* close the channel - that was a bad idea at its inception and it is > still a bad idea now, and people who do not want this behavior (== sane > people) will still be stuck with using temporary selectors. It is up to > the application to decide what thread interruption means. > InterruptedIOException is more appropriate, and the user can elect to > close the channel if they feel it is appropriate to do so. Inconsistency > is worthless if the existing behavior does not make sense. I mean "consistency is worthless...". > > 2. Not a fan of the triple-behavior-overload of the timeout parameter. > -- - DML From Alan.Bateman at oracle.com Wed Oct 2 20:18:58 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 02 Oct 2013 20:18:58 -0700 Subject: Request for review / feedback: SocketChannel.poll() In-Reply-To: <524C8A2A.8000006@redhat.com> References: <524C70AB.9080108@oracle.com> <524C8A2A.8000006@redhat.com> Message-ID: <524CE222.90009@oracle.com> On 02/10/2013 14:03, David M. Lloyd wrote: > : > > 1. Thread interruption should stop the poll, but absolutely should not > *not* close the channel - that was a bad idea at its inception and it > is still a bad idea now, and people who do not want this behavior (== > sane people) will still be stuck with using temporary selectors. It > is up to the application to decide what thread interruption means. > InterruptedIOException is more appropriate, and the user can elect to > close the channel if they feel it is appropriate to do so. > Inconsistency is worthless if the existing behavior does not make sense. You are correct that there is a bug in the proposal as the poll is supposed to behave in the same way as Selector.select when interrupted (which means it is supposed to wakeup the poll). I don't have time to debate your comments on InterruptedIOException as that was totally broken, we don't want to go back there. -Alan. From rob.mckenna at oracle.com Wed Oct 2 21:13:21 2013 From: rob.mckenna at oracle.com (Rob McKenna) Date: Thu, 03 Oct 2013 05:13:21 +0100 Subject: Request for review / feedback: SocketChannel.poll() In-Reply-To: <524CE222.90009@oracle.com> References: <524C70AB.9080108@oracle.com> <524C8A2A.8000006@redhat.com> <524CE222.90009@oracle.com> Message-ID: <524CEEE1.7090104@oracle.com> Thanks for that David. I'll prepare a new webrev shortly. I've just noticed too that I forgot to remove the pollingThread field. Please ignore that. -Rob On 03/10/13 04:18, Alan Bateman wrote: > On 02/10/2013 14:03, David M. Lloyd wrote: >> : >> >> 1. Thread interruption should stop the poll, but absolutely should >> not *not* close the channel - that was a bad idea at its inception >> and it is still a bad idea now, and people who do not want this >> behavior (== sane people) will still be stuck with using temporary >> selectors. It is up to the application to decide what thread >> interruption means. InterruptedIOException is more appropriate, and >> the user can elect to close the channel if they feel it is >> appropriate to do so. Inconsistency is worthless if the existing >> behavior does not make sense. > You are correct that there is a bug in the proposal as the poll is > supposed to behave in the same way as Selector.select when interrupted > (which means it is supposed to wakeup the poll). > > I don't have time to debate your comments on InterruptedIOException as > that was totally broken, we don't want to go back there. > > -Alan. From Alan.Bateman at oracle.com Wed Oct 2 21:27:23 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 02 Oct 2013 21:27:23 -0700 Subject: Request for review / feedback: SocketChannel.poll() In-Reply-To: <524CEEE1.7090104@oracle.com> References: <524C70AB.9080108@oracle.com> <524C8A2A.8000006@redhat.com> <524CE222.90009@oracle.com> <524CEEE1.7090104@oracle.com> Message-ID: <524CF22B.4020102@oracle.com> On 02/10/2013 21:13, Rob McKenna wrote: > Thanks for that David. I'll prepare a new webrev shortly. I've just > noticed too that I forgot to remove the pollingThread field. Please > ignore that. > > -Rob One other thing, the webrev also includes what appears to be an update to SocketChannel.getLocalAddress but that change is already in 8. -Alan From david.lloyd at redhat.com Thu Oct 3 07:37:31 2013 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 03 Oct 2013 09:37:31 -0500 Subject: Request for review / feedback: SocketChannel.poll() In-Reply-To: <524CE222.90009@oracle.com> References: <524C70AB.9080108@oracle.com> <524C8A2A.8000006@redhat.com> <524CE222.90009@oracle.com> Message-ID: <524D812B.2090303@redhat.com> On 10/02/2013 10:18 PM, Alan Bateman wrote: > On 02/10/2013 14:03, David M. Lloyd wrote: >> : >> >> 1. Thread interruption should stop the poll, but absolutely should not >> *not* close the channel - that was a bad idea at its inception and it >> is still a bad idea now, and people who do not want this behavior (== >> sane people) will still be stuck with using temporary selectors. It >> is up to the application to decide what thread interruption means. >> InterruptedIOException is more appropriate, and the user can elect to >> close the channel if they feel it is appropriate to do so. >> Inconsistency is worthless if the existing behavior does not make sense. > You are correct that there is a bug in the proposal as the poll is > supposed to behave in the same way as Selector.select when interrupted > (which means it is supposed to wakeup the poll). > > I don't have time to debate your comments on InterruptedIOException as > that was totally broken, we don't want to go back there. Ha, so much hate for InterruptedIOEException. I actually use it commonly (with a transferred count of 0) as a general purpose indication that an I/O op was interrupted. In my code I do not support partial transfers (unless I am implementing an API that does), so the user doesn't actually have to introspect the count. Unfortunately there does not seem to be an alternative exception type that indicates interruption (without other nasty effects like channel closure) and also extends IOException, which is the reason I've used it. Simply causing a wakeup is OK, however it does cause the user to have to check the thread interrupted status after every poll, which they are going to forget to do. -- - DML From philippe.marschall at gmail.com Fri Oct 4 04:32:49 2013 From: philippe.marschall at gmail.com (Philippe Marschall) Date: Fri, 4 Oct 2013 13:32:49 +0200 Subject: MacOS file system changes between 7u10 and 7u40? In-Reply-To: <52450D37.2020904@oracle.com> References: <52450D37.2020904@oracle.com> Message-ID: On Fri, Sep 27, 2013 at 6:44 AM, Xueming Shen wrote: > What we did in 7130915 is to normalize the native macos file name (in NFD) > back into > NFC for the File and Path, but passing the File/Path Java path name (in NFC) > into macos's > file system API directly (as it appears those APIs just work fine with NFC > file name, though > it stores them in NFD internally). This serves the purpose of having Java > file name > (File and Path, and their String representation) in NFC form consistently, > cross all platforms. This is not the behavior I'm seeing on Linux and Windows. They don't seem to do any normalization at all (I read that Windows does NFC but my tests seem to contradict this) and in addition you can have NFC and NFD file names side by side. That means you can't have normalized Paths in NFC on those platform at all times because then certain files would not be accessible to Java. I could not check ZFS which seems to be normalization-insensitive / normalization-preserving [1]. This would open the option to normalize. > As the consequence, we can have a reasonable implement of file name > equals() without > involving string normalization, which is expensive. In "normal" use > scenario, you should > not have a file/path name (in String) passing around in NFD form. [1] https://blogs.oracle.com/nico/entry/normalization_insensitivity_should_be_the Cheers Philippe From martinrb at google.com Fri Oct 4 08:30:40 2013 From: martinrb at google.com (Martin Buchholz) Date: Fri, 4 Oct 2013 08:30:40 -0700 Subject: MacOS file system changes between 7u10 and 7u40? In-Reply-To: References: <52450D37.2020904@oracle.com> Message-ID: It is already the case that you cannot access all possible Unix file names from Java because by design, file names are represented by Java strings (UTF-16), but at the OS level filenames are actually arbitrary byte sequences with no concept of encoding. On Fri, Oct 4, 2013 at 4:32 AM, Philippe Marschall < philippe.marschall at gmail.com> wrote: > On Fri, Sep 27, 2013 at 6:44 AM, Xueming Shen > wrote: > > What we did in 7130915 is to normalize the native macos file name (in > NFD) > > back into > > NFC for the File and Path, but passing the File/Path Java path name (in > NFC) > > into macos's > > file system API directly (as it appears those APIs just work fine with > NFC > > file name, though > > it stores them in NFD internally). This serves the purpose of having Java > > file name > > (File and Path, and their String representation) in NFC form > consistently, > > cross all platforms. > > This is not the behavior I'm seeing on Linux and Windows. They don't > seem to do any normalization at all (I read that Windows does NFC but > my tests seem to contradict this) and in addition you can have NFC and > NFD file names side by side. That means you can't have normalized > Paths in NFC on those platform at all times because then certain files > would not be accessible to Java. > > I could not check ZFS which seems to be normalization-insensitive / > normalization-preserving [1]. This would open the option to normalize. > > > As the consequence, we can have a reasonable implement of file name > > equals() without > > involving string normalization, which is expensive. In "normal" use > > scenario, you should > > not have a file/path name (in String) passing around in NFD form. > > [1] > https://blogs.oracle.com/nico/entry/normalization_insensitivity_should_be_the > > Cheers > Philippe > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20131004/b7edee0f/attachment.html From philippe.marschall at gmail.com Fri Oct 4 09:29:17 2013 From: philippe.marschall at gmail.com (Philippe Marschall) Date: Fri, 4 Oct 2013 18:29:17 +0200 Subject: MacOS file system changes between 7u10 and 7u40? In-Reply-To: References: <52450D37.2020904@oracle.com> Message-ID: On Fri, Oct 4, 2013 at 5:30 PM, Martin Buchholz wrote: > It is already the case that you cannot access all possible Unix file names > from Java because by design, file names are represented by Java strings > (UTF-16), but at the OS level filenames are actually arbitrary byte > sequences with no concept of encoding. I don't think this is true on HFS+ and ZFS. As they do normalization when a file is created, they need to have a concept of encoding. Cheers Philippe From philippe.marschall at gmail.com Fri Oct 4 09:32:29 2013 From: philippe.marschall at gmail.com (Philippe Marschall) Date: Fri, 4 Oct 2013 18:32:29 +0200 Subject: MacOS file system changes between 7u10 and 7u40? In-Reply-To: References: <52450D37.2020904@oracle.com> Message-ID: On Fri, Oct 4, 2013 at 6:29 PM, Philippe Marschall wrote: > On Fri, Oct 4, 2013 at 5:30 PM, Martin Buchholz wrote: >> It is already the case that you cannot access all possible Unix file names >> from Java because by design, file names are represented by Java strings >> (UTF-16), but at the OS level filenames are actually arbitrary byte >> sequences with no concept of encoding. > > I don't think this is true on HFS+ and ZFS. As they do normalization > when a file is created, they need to have a concept of encoding. Well ZFS not when the file is created but on look up. And on creation ZFS still needs to check if a file name equal after normalization is present. Cheers Philippe From martinrb at google.com Fri Oct 4 12:47:40 2013 From: martinrb at google.com (Martin Buchholz) Date: Fri, 4 Oct 2013 12:47:40 -0700 Subject: MacOS file system changes between 7u10 and 7u40? In-Reply-To: References: <52450D37.2020904@oracle.com> Message-ID: Interesting. ZFS apparently supports a wide variety of behavior. Apparently for Windows support ?! http://docs.oracle.com/cd/E19253-01/816-5166/zfs-1m/index.html casesensitivity=sensitive | insensitive | mixed Indicates whether the file name matching algorithm used by the file system should be case-sensitive, case-insensitive, or allow a combination of both styles of matching. The default value for the casesensitivity property is sensitive. Traditionally, UNIX and POSIX file systems have case-sensitive file names. The mixed value for the casesensitivity property indicates that the file system can support requests for both case-sensitive and case-insensitive matching behavior. Currently, case-insensitive matching behavior on a file system that supports mixed behavior is limited to the Oracle Solaris SMB server product, which is not supported in the Oracle Solaris 10 release. For more information about the mixed value behavior, see the Oracle Solaris ZFS Administration Guide. This SMB related property is not fully functional in the Oracle Solaris 10 release because the Oracle Solaris SMB server is not supported in the Oracle Solaris 10 release. normalization = none | formC | formD | formKC | formKD Indicates whether the file system should perform a unicode normalization of file names whenever two file names are compared, and which normalization algorithm should be used. File names are always stored unmodified, names are normalized as part of any comparison process. If this property is set to a legal value other than none, and the utf8only property was left unspecified, the utf8only property is automatically set to on. The default value of the normalization property is none. This property cannot be changed after the file system is created. This SMB related property is not fully functional in the Oracle Solaris 10 release because the Oracle Solaris SMB server is not supported in the Oracle Solaris 10 release. utf8only=on | off Indicates whether the file system should reject file names that include characters that are not present in the UTF-8 character code set. If this property is explicitly set to off, the normalization property must either not be explicitly set or be set to none. The default value for the utf8only property is off. This property cannot be changed after the file system is created. This SMB related property is not fully functional in the Oracle Solaris 10 release because the Oracle Solaris SMB server is not supported in the Oracle Solaris 10 release. On Fri, Oct 4, 2013 at 9:32 AM, Philippe Marschall < philippe.marschall at gmail.com> wrote: > On Fri, Oct 4, 2013 at 6:29 PM, Philippe Marschall > wrote: > > On Fri, Oct 4, 2013 at 5:30 PM, Martin Buchholz > wrote: > >> It is already the case that you cannot access all possible Unix file > names > >> from Java because by design, file names are represented by Java strings > >> (UTF-16), but at the OS level filenames are actually arbitrary byte > >> sequences with no concept of encoding. > > > > I don't think this is true on HFS+ and ZFS. As they do normalization > > when a file is created, they need to have a concept of encoding. > > Well ZFS not when the file is created but on look up. And on creation > ZFS still needs to check if a file name equal after normalization is > present. > > Cheers > Philippe > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20131004/4174cd95/attachment.html From sean.coffey at oracle.com Sat Oct 5 08:01:35 2013 From: sean.coffey at oracle.com (=?ISO-8859-1?Q?Se=E1n_Coffey?=) Date: Sat, 5 Oct 2013 08:01:35 -0700 (PDT) Subject: RFR : 8012326 Deadlock occurs when Charset.availableCharsets() is called by several threads at the same time In-Reply-To: <523C4FF5.8020908@oracle.com> References: <523C4FF5.8020908@oracle.com> Message-ID: <525029CF.9090608@oracle.com> Sherman, would you have cycles for review ? regards, Sean. On 20/09/2013 14:39, Se?n Coffey wrote: > I'd like to port this from jdk8 to jdk7u-dev. > > https://bugs.openjdk.java.net/browse/JDK-8012326 > > The backport is similar to JDK 8 fix with the exception of > ISO2022_JP_2 & MSISO2022JP classes. The static variables in those > classes (DEC02{12|08}/ENC02{12|08}) came in via the JDK-6653797 fix > which is only in jdk8. It doesn't appear to be applicable to JDK 7u as > a result. (we're not loading JIS_X_0208() class in ISO2022_JP > initialization for jdk7u from what I see) > > In summary, the jdk8 webrev looks applicable to jdk7u with exception > of ISO2022_JP_2.java / MSISO2022JP.java changes. > > webrev : > http://cr.openjdk.java.net/~coffeys/webrev.8012326.jdk7u/webrev/ > > > regards, > Sean. > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20131005/8f2e4792/attachment.html From philippe.marschall at gmail.com Sun Oct 6 10:27:34 2013 From: philippe.marschall at gmail.com (Philippe Marschall) Date: Sun, 6 Oct 2013 19:27:34 +0200 Subject: MacOS file system changes between 7u10 and 7u40? In-Reply-To: References: <52450D37.2020904@oracle.com> Message-ID: On Fri, Oct 4, 2013 at 5:30 PM, Martin Buchholz wrote: > It is already the case that you cannot access all possible Unix file names > from Java because by design, file names are represented by Java strings > (UTF-16), but at the OS level filenames are actually arbitrary byte > sequences with no concept of encoding. While file names are Strings and java.io.File is String based sun.nio.fs.UnixPath is actually byte[] based. This means so you can access files whos name is not valid in the respective encoding given you can get a hold of the path. The "easy way" is through DirectoryStream the other through the package protected constructor. (Yes I did some checking on a Linux box with file names that are invalid UTF-8). Cheers Philippe From xueming.shen at oracle.com Sun Oct 6 10:58:49 2013 From: xueming.shen at oracle.com (Xueming Shen) Date: Sun, 06 Oct 2013 10:58:49 -0700 Subject: MacOS file system changes between 7u10 and 7u40? In-Reply-To: References: <52450D37.2020904@oracle.com> Message-ID: <5251A4D9.7000104@oracle.com> On 10/6/13 10:27 AM, Philippe Marschall wrote: > On Fri, Oct 4, 2013 at 5:30 PM, Martin Buchholz wrote: >> It is already the case that you cannot access all possible Unix file names >> from Java because by design, file names are represented by Java strings >> (UTF-16), but at the OS level filenames are actually arbitrary byte >> sequences with no concept of encoding. > While file names are Strings and java.io.File is String based > sun.nio.fs.UnixPath is actually byte[] based. This means so you can > access files whos name is not valid in the respective encoding given > you can get a hold of the path. The "easy way" is through > DirectoryStream the other through the package protected constructor. The byte[] representation is really an internal implementation detail to have much better performance when the path is pushed back and forth between the Java level and the native level. Any String level file name access to the nio Path still involves the charset's encoding and decoding, which normally do not handle the nfc/nfd at all, with the exception that the utf based encoding that can do a code point to code point mapping. While we had put lots of thoughts into the encoding/decoding issue for the byte[] <=> String conversion the nfc/nfd issue just kicked in "recently" after the MacOS filesystem started get attraction, with a NFD internal representation (and an interesting flipflop-able case sensitiveness). Again, the idea here is to try to keep the consistency of the file name representation at Java level, which I personally feel more important. So maybe the question here is do we want to see the NFD file name at Java level, which means developer/end user will probably be forced to handle nfd/nfc handling themselves and the file name hashing/equality operation will be way more "expensive", and then the "visual" representation... do you want to see the file names being displayed as nfd or nfc... -Sherman > > (Yes I did some checking on a Linux box with file names that are invalid UTF-8). > > Cheers > Philippe From Alan.Bateman at oracle.com Mon Oct 7 01:15:40 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 07 Oct 2013 09:15:40 +0100 Subject: 8025983: Typo in Javadoc of Files.isRegularFile() Message-ID: <52526DAC.3040504@oracle.com> I need a reviewer for a trivial fix to the javadoc where it reads "is it" instead of "it is". The proposed patch is attached. -Alan diff --git a/src/share/classes/java/nio/file/Files.java b/src/share/classes/java/nio/file/Files.java --- a/src/share/classes/java/nio/file/Files.java +++ b/src/share/classes/java/nio/file/Files.java @@ -2128,7 +2128,7 @@ /** * Tests whether a file is a symbolic link. * - *

Where is it required to distinguish an I/O exception from the case + *

Where it is required to distinguish an I/O exception from the case * that the file is not a symbolic link then the file attributes can be * read with the {@link #readAttributes(Path,Class,LinkOption[]) * readAttributes} method and the file type tested with the {@link @@ -2164,7 +2164,7 @@ * of the link is read. If the option {@link LinkOption#NOFOLLOW_LINKS * NOFOLLOW_LINKS} is present then symbolic links are not followed. * - *

Where is it required to distinguish an I/O exception from the case + *

Where it is required to distinguish an I/O exception from the case * that the file is not a directory then the file attributes can be * read with the {@link #readAttributes(Path,Class,LinkOption[]) * readAttributes} method and the file type tested with the {@link @@ -2201,7 +2201,7 @@ * of the link is read. If the option {@link LinkOption#NOFOLLOW_LINKS * NOFOLLOW_LINKS} is present then symbolic links are not followed. * - *

Where is it required to distinguish an I/O exception from the case + *

Where it is required to distinguish an I/O exception from the case * that the file is not a regular file then the file attributes can be * read with the {@link #readAttributes(Path,Class,LinkOption[]) * readAttributes} method and the file type tested with the {@link diff --git a/src/share/classes/java/nio/file/Path.java b/src/share/classes/java/nio/file/Path.java --- a/src/share/classes/java/nio/file/Path.java +++ b/src/share/classes/java/nio/file/Path.java @@ -315,7 +315,7 @@ * and parent directory. In such file systems all occurrences of "{@code .}" * are considered redundant. If a "{@code ..}" is preceded by a * non-"{@code ..}" name then both names are considered redundant (the - * process to identify such names is repeated until is it no longer + * process to identify such names is repeated until it is no longer * applicable). * *

This method does not access the file system; the path may not locate From mandy.chung at oracle.com Mon Oct 7 01:25:06 2013 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 07 Oct 2013 01:25:06 -0700 Subject: 8025983: Typo in Javadoc of Files.isRegularFile() In-Reply-To: <52526DAC.3040504@oracle.com> References: <52526DAC.3040504@oracle.com> Message-ID: <52526FE2.40403@oracle.com> Thumbs up. Mandy On 10/7/2013 1:15 AM, Alan Bateman wrote: > > I need a reviewer for a trivial fix to the javadoc where it reads "is > it" instead of "it is". The proposed patch is attached. > > -Alan > > > diff --git a/src/share/classes/java/nio/file/Files.java > b/src/share/classes/java/nio/file/Files.java > --- a/src/share/classes/java/nio/file/Files.java > +++ b/src/share/classes/java/nio/file/Files.java > @@ -2128,7 +2128,7 @@ > /** > * Tests whether a file is a symbolic link. > * > - *

Where is it required to distinguish an I/O exception from > the case > + *

Where it is required to distinguish an I/O exception from > the case > * that the file is not a symbolic link then the file attributes > can be > * read with the {@link #readAttributes(Path,Class,LinkOption[]) > * readAttributes} method and the file type tested with the {@link > @@ -2164,7 +2164,7 @@ > * of the link is read. If the option {@link > LinkOption#NOFOLLOW_LINKS > * NOFOLLOW_LINKS} is present then symbolic links are not followed. > * > - *

Where is it required to distinguish an I/O exception from > the case > + *

Where it is required to distinguish an I/O exception from > the case > * that the file is not a directory then the file attributes can be > * read with the {@link #readAttributes(Path,Class,LinkOption[]) > * readAttributes} method and the file type tested with the {@link > @@ -2201,7 +2201,7 @@ > * of the link is read. If the option {@link > LinkOption#NOFOLLOW_LINKS > * NOFOLLOW_LINKS} is present then symbolic links are not followed. > * > - *

Where is it required to distinguish an I/O exception from > the case > + *

Where it is required to distinguish an I/O exception from > the case > * that the file is not a regular file then the file attributes > can be > * read with the {@link #readAttributes(Path,Class,LinkOption[]) > * readAttributes} method and the file type tested with the {@link > diff --git a/src/share/classes/java/nio/file/Path.java > b/src/share/classes/java/nio/file/Path.java > --- a/src/share/classes/java/nio/file/Path.java > +++ b/src/share/classes/java/nio/file/Path.java > @@ -315,7 +315,7 @@ > * and parent directory. In such file systems all occurrences of > "{@code .}" > * are considered redundant. If a "{@code ..}" is preceded by a > * non-"{@code ..}" name then both names are considered redundant > (the > - * process to identify such names is repeated until is it no longer > + * process to identify such names is repeated until it is no longer > * applicable). > * > *

This method does not access the file system; the path may > not locate From chris.hegarty at oracle.com Mon Oct 7 01:25:25 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 07 Oct 2013 09:25:25 +0100 Subject: 8025983: Typo in Javadoc of Files.isRegularFile() In-Reply-To: <52526DAC.3040504@oracle.com> References: <52526DAC.3040504@oracle.com> Message-ID: <52526FF5.3090802@oracle.com> The changes look good to me. Trivially, there is an additional space after the third typo fix, "it is__...." -Chris. On 10/07/2013 09:15 AM, Alan Bateman wrote: > > I need a reviewer for a trivial fix to the javadoc where it reads "is > it" instead of "it is". The proposed patch is attached. > > -Alan > > > diff --git a/src/share/classes/java/nio/file/Files.java > b/src/share/classes/java/nio/file/Files.java > --- a/src/share/classes/java/nio/file/Files.java > +++ b/src/share/classes/java/nio/file/Files.java > @@ -2128,7 +2128,7 @@ > /** > * Tests whether a file is a symbolic link. > * > - *

Where is it required to distinguish an I/O exception from > the case > + *

Where it is required to distinguish an I/O exception from > the case > * that the file is not a symbolic link then the file attributes > can be > * read with the {@link #readAttributes(Path,Class,LinkOption[]) > * readAttributes} method and the file type tested with the {@link > @@ -2164,7 +2164,7 @@ > * of the link is read. If the option {@link > LinkOption#NOFOLLOW_LINKS > * NOFOLLOW_LINKS} is present then symbolic links are not followed. > * > - *

Where is it required to distinguish an I/O exception from > the case > + *

Where it is required to distinguish an I/O exception from > the case > * that the file is not a directory then the file attributes can be > * read with the {@link #readAttributes(Path,Class,LinkOption[]) > * readAttributes} method and the file type tested with the {@link > @@ -2201,7 +2201,7 @@ > * of the link is read. If the option {@link > LinkOption#NOFOLLOW_LINKS > * NOFOLLOW_LINKS} is present then symbolic links are not followed. > * > - *

Where is it required to distinguish an I/O exception from > the case > + *

Where it is required to distinguish an I/O exception from > the case > * that the file is not a regular file then the file attributes > can be > * read with the {@link #readAttributes(Path,Class,LinkOption[]) > * readAttributes} method and the file type tested with the {@link > diff --git a/src/share/classes/java/nio/file/Path.java > b/src/share/classes/java/nio/file/Path.java > --- a/src/share/classes/java/nio/file/Path.java > +++ b/src/share/classes/java/nio/file/Path.java > @@ -315,7 +315,7 @@ > * and parent directory. In such file systems all occurrences of > "{@code .}" > * are considered redundant. If a "{@code ..}" is preceded by a > * non-"{@code ..}" name then both names are considered redundant > (the > - * process to identify such names is repeated until is it no longer > + * process to identify such names is repeated until it is no longer > * applicable). > * > *

This method does not access the file system; the path may > not locate From Alan.Bateman at oracle.com Mon Oct 7 07:13:29 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 07 Oct 2013 15:13:29 +0100 Subject: 8025983: Typo in Javadoc of Files.isRegularFile() In-Reply-To: <52526FF5.3090802@oracle.com> References: <52526DAC.3040504@oracle.com> <52526FF5.3090802@oracle.com> Message-ID: <5252C189.7040206@oracle.com> On 07/10/2013 09:25, Chris Hegarty wrote: > The changes look good to me. > > Trivially, there is an additional space after the third typo fix, > "it is__...." > > -Chris. Thanks for spotting that, I fixed it before pushing. -Alan. From Alan.Bateman at oracle.com Tue Oct 8 00:52:25 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 08 Oct 2013 08:52:25 +0100 Subject: 8024788: (fs) Files.readAllBytes uses FileChannel which may not be supported by all providers Message-ID: <5253B9B9.5050109@oracle.com> I need a reviewer for a small fix to Files.readAllBytes so that it works when the reading from a file associated with a FileSystemProvider that doesn't support FileChannel (which is allowed, FileSystemProvider.newFileChannel is allowed to throw UOE). This is one of the issues that Colin Decker brought up on this list a few weeks ago. The readAllBytes implementation can be trivially changed to see newByteChannel as per the discussion. The webrev with the changes is here: http://cr.openjdk.java.net/~alanb/8024788/webrev/ Thanks, Alan. From chris.hegarty at oracle.com Tue Oct 8 02:09:36 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 08 Oct 2013 10:09:36 +0100 Subject: 8024788: (fs) Files.readAllBytes uses FileChannel which may not be supported by all providers In-Reply-To: <5253B9B9.5050109@oracle.com> References: <5253B9B9.5050109@oracle.com> Message-ID: <5253CBD0.7020203@oracle.com> Looks fine to me Alan. I presume this will eventually make its was into 7u60 too. -Chris. On 10/08/2013 08:52 AM, Alan Bateman wrote: > > I need a reviewer for a small fix to Files.readAllBytes so that it works > when the reading from a file associated with a FileSystemProvider that > doesn't support FileChannel (which is allowed, > FileSystemProvider.newFileChannel is allowed to throw UOE). This is one > of the issues that Colin Decker brought up on this list a few weeks ago. > The readAllBytes implementation can be trivially changed to see > newByteChannel as per the discussion. > > The webrev with the changes is here: > > http://cr.openjdk.java.net/~alanb/8024788/webrev/ > > Thanks, > Alan. From Alan.Bateman at oracle.com Tue Oct 8 02:16:11 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 08 Oct 2013 10:16:11 +0100 Subject: 8024788: (fs) Files.readAllBytes uses FileChannel which may not be supported by all providers In-Reply-To: <5253CBD0.7020203@oracle.com> References: <5253B9B9.5050109@oracle.com> <5253CBD0.7020203@oracle.com> Message-ID: <5253CD5B.50806@oracle.com> On 08/10/2013 10:09, Chris Hegarty wrote: > Looks fine to me Alan. > > I presume this will eventually make its was into 7u60 too. Thanks Chris. It is candidate to push to jdk7u-dev once it is in 8. -Alan From ygaevsky at azulsystems.com Thu Oct 10 13:28:13 2013 From: ygaevsky at azulsystems.com (Yuri Gaevsky) Date: Thu, 10 Oct 2013 20:28:13 +0000 Subject: [PATCH] 7052549 "(aio) AssertionError in sun.nio.ch.PendingIoCache.clearPendingIoMap (win)" Message-ID: Hello, I'm trying to follow [1] in order to propose a patch for JDK-7052549 [2]. The code in clearPendingIoMap() doesn't use 'synchronized' access to internal data as all other methods do and the 'assert Thread.holdsLock(this)' statement in the method can fail when "-esa" command-line option enabled. The proposed patch moves all accesses to 'pendingIoMap' and 'closePending' under synchronized block, and removes the 'assert !result.isDone()' statement. The patch has been verified by JCK testrun with '-esa' option. Thanks, Yuri Gaevsky, Azul Systems, Inc. [1] http://openjdk.java.net/contribute/ [2] https://bugs.openjdk.java.net/browse/JDK-7052549 diff --git a/src/windows/classes/sun/nio/ch/PendingIoCache.java b/src/windows/classes/sun/nio/ch/PendingIoCache.java --- a/src/windows/classes/sun/nio/ch/PendingIoCache.java +++ b/src/windows/classes/sun/nio/ch/PendingIoCache.java @@ -125,38 +125,38 @@ } private void clearPendingIoMap() { - assert Thread.holdsLock(this); + synchronized (this) { + // wait up to 50ms for the I/O operations to complete + closePending = true; + try { + this.wait(50); + } catch (InterruptedException x) { + Thread.currentThread().interrupt(); + } + closePending = false; - // wait up to 50ms for the I/O operations to complete - closePending = true; - try { - this.wait(50); - } catch (InterruptedException x) { - Thread.currentThread().interrupt(); + // cause all pending I/O operations to fail + // simulate the failure of all pending I/O operations. + if (!pendingIoMap.isEmpty()) { + for (Long ov: pendingIoMap.keySet()) { + PendingFuture result = pendingIoMap.get(ov); + + // make I/O port aware of the stale OVERLAPPED structure + Iocp iocp = (Iocp)((Groupable)result.channel()).group(); + iocp.makeStale(ov); + + // execute a task that invokes the result handler's failed method + final Iocp.ResultHandler rh = (Iocp.ResultHandler)result.getContext(); + Runnable task = new Runnable() { + public void run() { + rh.failed(-1, new AsynchronousCloseException()); + } + }; + iocp.executeOnPooledThread(task); + } + pendingIoMap.clear(); + } } - closePending = false; - if (pendingIoMap.isEmpty()) - return; - - // cause all pending I/O operations to fail - // simulate the failure of all pending I/O operations. - for (Long ov: pendingIoMap.keySet()) { - PendingFuture result = pendingIoMap.get(ov); - assert !result.isDone(); - - // make I/O port aware of the stale OVERLAPPED structure - Iocp iocp = (Iocp)((Groupable)result.channel()).group(); - iocp.makeStale(ov); - - // execute a task that invokes the result handler's failed method - final Iocp.ResultHandler rh = (Iocp.ResultHandler)result.getContext(); - Runnable task = new Runnable() { - public void run() { - rh.failed(-1, new AsynchronousCloseException()); - } - }; - iocp.executeOnPooledThread(task); - } - pendingIoMap.clear(); } } + From Alan.Bateman at oracle.com Fri Oct 11 07:46:59 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 11 Oct 2013 15:46:59 +0100 Subject: 8019526: (fs) Files.lines, etc without Charset parameter Message-ID: <52580F63.9090109@oracle.com> The Files class defines a number of very useful methods to operate on text files, all of which take a Charset parameter for the charset to use when encoding or decoding. It's come many times that folks want variants of these that don't have to specify the Charset. Additionally we now have Files.lines, added as part of the Lambda libraries work, and a variant without the Charset parameter is required to use method references. The proposal here is to define variants of each of these methods that use UTF-8 for encoding and decoding instead of taking a Charset parameter. The rational for UTF-8 is that methods that use the default charset are always problematic. With many platforms using UTF-8 now then it is less of an issue from a few years ago but still problematic due to the dependency on the locale. The proposal does mean an inconsistency with the legacy APIs but it should only be an issue if there is mixing of APIs without due consideration. Going forward then it may actually be time to consider changing the JDK's default charset to UTF-8 (I'm not suggesting this for JDK 8 of course, that is a mich bigger issue than what is proposed here and would require deeper consideration). The webrev with the proposed changes is here. The updates to the Files class are trivial (one-liner methods), most of the changes are in the tests. I've expanding Henry's StreamTest to exercise Files.lines(Path) and also converted the BytesAndLines to TestNG. More could be done on the BytesAndLines but it's not critical for this patch. http://cr.openjdk.java.net/~alanb/8019526/webrev/ Thanks, Alan. From paul.sandoz at oracle.com Fri Oct 11 08:02:10 2013 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 11 Oct 2013 17:02:10 +0200 Subject: 8019526: (fs) Files.lines, etc without Charset parameter In-Reply-To: <52580F63.9090109@oracle.com> References: <52580F63.9090109@oracle.com> Message-ID: <8B1FEC11-D6ED-43EA-9567-13F548979390@oracle.com> Change to Files looks good. I did not look too closely at the changes to the test files, just focused on the areas where UTF-8 is the default. Paul. On Oct 11, 2013, at 4:46 PM, Alan Bateman wrote: > > The Files class defines a number of very useful methods to operate on text files, all of which take a Charset parameter for the charset to use when encoding or decoding. It's come many times that folks want variants of these that don't have to specify the Charset. Additionally we now have Files.lines, added as part of the Lambda libraries work, and a variant without the Charset parameter is required to use method references. > > The proposal here is to define variants of each of these methods that use UTF-8 for encoding and decoding instead of taking a Charset parameter. The rational for UTF-8 is that methods that use the default charset are always problematic. With many platforms using UTF-8 now then it is less of an issue from a few years ago but still problematic due to the dependency on the locale. The proposal does mean an inconsistency with the legacy APIs but it should only be an issue if there is mixing of APIs without due consideration. Going forward then it may actually be time to consider changing the JDK's default charset to UTF-8 (I'm not suggesting this for JDK 8 of course, that is a mich bigger issue than what is proposed here and would require deeper consideration). > > The webrev with the proposed changes is here. The updates to the Files class are trivial (one-liner methods), most of the changes are in the tests. I've expanding Henry's StreamTest to exercise Files.lines(Path) and also converted the BytesAndLines to TestNG. More could be done on the BytesAndLines but it's not critical for this patch. > > http://cr.openjdk.java.net/~alanb/8019526/webrev/ > > Thanks, > Alan. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20131011/25a1344d/signature.asc From henry.jen at Oracle.com Fri Oct 11 10:54:48 2013 From: henry.jen at Oracle.com (Henry Jen) Date: Fri, 11 Oct 2013 10:54:48 -0700 Subject: 8019526: (fs) Files.lines, etc without Charset parameter In-Reply-To: <52580F63.9090109@oracle.com> References: <52580F63.9090109@oracle.com> Message-ID: Looks good to me. Cheers, Henry On Oct 11, 2013, at 7:46 AM, Alan Bateman wrote: > > The Files class defines a number of very useful methods to operate on text files, all of which take a Charset parameter for the charset to use when encoding or decoding. It's come many times that folks want variants of these that don't have to specify the Charset. Additionally we now have Files.lines, added as part of the Lambda libraries work, and a variant without the Charset parameter is required to use method references. > > The proposal here is to define variants of each of these methods that use UTF-8 for encoding and decoding instead of taking a Charset parameter. The rational for UTF-8 is that methods that use the default charset are always problematic. With many platforms using UTF-8 now then it is less of an issue from a few years ago but still problematic due to the dependency on the locale. The proposal does mean an inconsistency with the legacy APIs but it should only be an issue if there is mixing of APIs without due consideration. Going forward then it may actually be time to consider changing the JDK's default charset to UTF-8 (I'm not suggesting this for JDK 8 of course, that is a mich bigger issue than what is proposed here and would require deeper consideration). > > The webrev with the proposed changes is here. The updates to the Files class are trivial (one-liner methods), most of the changes are in the tests. I've expanding Henry's StreamTest to exercise Files.lines(Path) and also converted the BytesAndLines to TestNG. More could be done on the BytesAndLines but it's not critical for this patch. > > http://cr.openjdk.java.net/~alanb/8019526/webrev/ > > Thanks, > Alan. From philippe.marschall at gmail.com Sun Oct 13 04:24:07 2013 From: philippe.marschall at gmail.com (Philippe Marschall) Date: Sun, 13 Oct 2013 13:24:07 +0200 Subject: MacOS file system changes between 7u10 and 7u40? In-Reply-To: <5251A4D9.7000104@oracle.com> References: <52450D37.2020904@oracle.com> <5251A4D9.7000104@oracle.com> Message-ID: On Sun, Oct 6, 2013 at 7:58 PM, Xueming Shen wrote: > On 10/6/13 10:27 AM, Philippe Marschall wrote: >> >> On Fri, Oct 4, 2013 at 5:30 PM, Martin Buchholz >> wrote: >>> >>> It is already the case that you cannot access all possible Unix file >>> names >>> from Java because by design, file names are represented by Java strings >>> (UTF-16), but at the OS level filenames are actually arbitrary byte >>> sequences with no concept of encoding. >> >> While file names are Strings and java.io.File is String based >> sun.nio.fs.UnixPath is actually byte[] based. This means so you can >> access files whos name is not valid in the respective encoding given >> you can get a hold of the path. The "easy way" is through >> DirectoryStream the other through the package protected constructor. > > > The byte[] representation is really an internal implementation detail to > have much better > performance when the path is pushed back and forth between the Java level > and the native > level. Any String level file name access to the nio Path still involves the > charset's encoding > and decoding, which normally do not handle the nfc/nfd at all, with the > exception that > the utf based encoding that can do a code point to code point mapping. > While we had > put lots of thoughts into the encoding/decoding issue for the byte[] <=> > String conversion > the nfc/nfd issue just kicked in "recently" after the MacOS filesystem > started get attraction, > with a NFD internal representation (and an interesting flipflop-able case > sensitiveness). > Again, the idea here is to try to keep the consistency of the file name > representation at > Java level, which I personally feel more important. As I said before I don't see how you can do that when Linux and Windows allow NFC and NFD side by side. Additionally if that is really a goal then I think you should commit to that and put it into the contract. As it is right now it's just an implementation artifact that could change at any point, it is therefore useless because you can't rely on it. An other JVM could do something completely different. > So maybe the question > here is do we > want to see the NFD file name at Java level, which means developer/end user > will probably > be forced to handle nfd/nfc handling themselves and the file name > hashing/equality Well, there's Files.isSameFile and Path.toRealPath for this. I would not expect anything else to work especially because the Path.equals contract is so lose (which is fine). > operation will be way more "expensive", and then the "visual" > representation... do you > want to see the file names being displayed as nfd or nfc... For rendering or displaying I don't really care, NFC and NFD should render and display the same (at least that's my understanding). As for the Java String-level I don't really know. On one hand we can't and shouldn't really make any assumptions on how the underlying Java file system works as expressed in various Javadocs. On the other hand it would be intuitive (at least to me) if it behaved like the underlying native file system so you know what to expect. As I said before I don't think you can make it consistent across all platforms. Cheers Philippe From Alan.Bateman at oracle.com Tue Oct 15 08:47:48 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 15 Oct 2013 16:47:48 +0100 Subject: Bug in interrupt handling in =?UTF-8?B?RmlsZUNoYW5uZWxJbXBsLg==?= =?UTF-8?B?bWFwKOKApik=?= In-Reply-To: References: Message-ID: <525D63A4.9010705@oracle.com> On 23/09/2013 14:33, Chris Dennis wrote: > Sure, no problem. There are a couple of oddities to note in the test? > > 1. It doesn't actually end up mapping anything. I'm intentionally trying > to extend the file from zero length to length one, when the file is > read-only. This way I pass through the nested size call, but don't > actually end up mapping anything. Seemed to me like this was the simplest > way to test exactly this bug and nothing else, and also avoid all the > windows specific mapping behaviors causing any problems. > 2. I'm not closing the file in a finally block because when the code is > broken the locking will block the close call, hence I'm only closing on a > successful run. > Just coming back to this. The patch to FileChannelImpl.map looks fine (and the update to when map needs to extend the file is okay too). I'm happy to sponsor this and get it in. I'm not sure about the test yet, it appears to fails intermittently (on several platforms), typically because the FileChannel isn't closed. Have you seen this? I haven't dug into it yet but we would need the test to be very reliable to include it with the patch. -Alan. From xueming.shen at oracle.com Tue Oct 15 15:20:11 2013 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 15 Oct 2013 15:20:11 -0700 Subject: RFR : 8012326 Deadlock occurs when Charset.availableCharsets() is called by several threads at the same time In-Reply-To: <523C4FF5.8020908@oracle.com> References: <523C4FF5.8020908@oracle.com> Message-ID: <525DBF9B.7080009@oracle.com> Looks fine. My apology for the be-lated review. -Sherman On 09/20/2013 06:39 AM, Se?n Coffey wrote: > I'd like to port this from jdk8 to jdk7u-dev. > > https://bugs.openjdk.java.net/browse/JDK-8012326 > > The backport is similar to JDK 8 fix with the exception of ISO2022_JP_2 & MSISO2022JP classes. The static variables in those classes (DEC02{12|08}/ENC02{12|08}) came in via the JDK-6653797 fix which is only in jdk8. It doesn't appear to be applicable to JDK 7u as a result. (we're not loading JIS_X_0208() class in ISO2022_JP initialization for jdk7u from what I see) > > In summary, the jdk8 webrev looks applicable to jdk7u with exception of ISO2022_JP_2.java / MSISO2022JP.java changes. > > webrev : http://cr.openjdk.java.net/~coffeys/webrev.8012326.jdk7u/webrev/ > > regards, > Sean. > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20131015/87693474/attachment.html From chris.w.dennis at gmail.com Wed Oct 16 05:52:52 2013 From: chris.w.dennis at gmail.com (Chris Dennis) Date: Wed, 16 Oct 2013 08:52:52 -0400 Subject: Bug in interrupt handling in FileChannelImpl.map(=?UTF-8?B?4oCm?=) In-Reply-To: <525D63A4.9010705@oracle.com> Message-ID: I'll look in to the test when I get a chance. I haven't seen it fail this way - but like I said the close logic is a bit weird in order to try and make the failure 'clean'. I'll look through it and see if I can figure out what might be going wrong. Chris On 10/15/13 11:47 AM, "Alan Bateman" wrote: >On 23/09/2013 14:33, Chris Dennis wrote: >> Sure, no problem. There are a couple of oddities to note in the test? >> >> 1. It doesn't actually end up mapping anything. I'm intentionally trying >> to extend the file from zero length to length one, when the file is >> read-only. This way I pass through the nested size call, but don't >> actually end up mapping anything. Seemed to me like this was the >>simplest >> way to test exactly this bug and nothing else, and also avoid all the >> windows specific mapping behaviors causing any problems. >> 2. I'm not closing the file in a finally block because when the code is >> broken the locking will block the close call, hence I'm only closing on >>a >> successful run. >> >Just coming back to this. The patch to FileChannelImpl.map looks fine >(and the update to when map needs to extend the file is okay too). I'm >happy to sponsor this and get it in. > >I'm not sure about the test yet, it appears to fails intermittently (on >several platforms), typically because the FileChannel isn't closed. Have >you seen this? I haven't dug into it yet but we would need the test to >be very reliable to include it with the patch. > >-Alan. > > From Alan.Bateman at oracle.com Fri Oct 18 04:04:36 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 18 Oct 2013 12:04:36 +0100 Subject: 8026859: (fs) test/java/nio/file/Files/StreamTest.java fails to compile intermittently Message-ID: <526115C4.3070604@oracle.com> When I updated this test recently then I removed the @library tag as it didn't appear to be used. However one of its dependencies (FaultyFileSystem.java) does make use of the test utilities for this area and so the test will not compile without @library. The reason it wasn't noticed is because other tests running before it also use FaultyFileSystem and so it was compiled already. So a simple patch to restore @library for now. -Alan diff --git a/test/java/nio/file/Files/StreamTest.java b/test/java/nio/file/Files/StreamTest.java --- a/test/java/nio/file/Files/StreamTest.java +++ b/test/java/nio/file/Files/StreamTest.java @@ -23,6 +23,7 @@ /* @test * @bug 8006884 8019526 + * @library .. * @build PassThroughFileSystem FaultyFileSystem * @run testng StreamTest * @summary Unit test for java.nio.file.Files methods that return a Stream From paul.sandoz at oracle.com Fri Oct 18 05:06:38 2013 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 18 Oct 2013 14:06:38 +0200 Subject: 8026859: (fs) test/java/nio/file/Files/StreamTest.java fails to compile intermittently In-Reply-To: <526115C4.3070604@oracle.com> References: <526115C4.3070604@oracle.com> Message-ID: <8C8666CD-C09F-4863-9B34-F92963F55520@oracle.com> On Oct 18, 2013, at 1:04 PM, Alan Bateman wrote: > > When I updated this test recently then I removed the @library tag as it didn't appear to be used. However one of its dependencies (FaultyFileSystem.java) does make use of the test utilities for this area and so the test will not compile without @library. The reason it wasn't noticed is because other tests running before it also use FaultyFileSystem and so it was compiled already. > We have poor dependency management :-) > So a simple patch to restore @library for now. > +1 Paul. > -Alan > > > diff --git a/test/java/nio/file/Files/StreamTest.java b/test/java/nio/file/Files/StreamTest.java > --- a/test/java/nio/file/Files/StreamTest.java > +++ b/test/java/nio/file/Files/StreamTest.java > @@ -23,6 +23,7 @@ > > /* @test > * @bug 8006884 8019526 > + * @library .. > * @build PassThroughFileSystem FaultyFileSystem > * @run testng StreamTest > * @summary Unit test for java.nio.file.Files methods that return a Stream -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20131018/52007e01/signature.asc From Alan.Bateman at oracle.com Mon Oct 21 00:54:38 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 21 Oct 2013 08:54:38 +0100 Subject: Question about FileSystemProvider API in JDK7 In-Reply-To: References: Message-ID: <5264DDBE.7080508@oracle.com> On 21/10/2013 06:30, Pisarev, Vitaliy wrote: > Hello, > > I was directed to this list by Brian Goetz, I am a new member and I am not yet acquainted with how things go in these mailing lists. > I am an SW Engineer and I am looking for some insight about the new FileSystemProvider API. I posted a question on SO: > > http://stackoverflow.com/questions/19425836/tweaking-the-behavior-of-the-default-file-system-in-java-7 > > I hope this reaches you and your answer reaches me back. > The service provider interface allows you to replace the default provider or interpose on it (see the FileSystems.getDefault docs for the details on how this is configured). When you interpose on the default provider then you have the opportunity to do your customization although it can be tricky to ensure that you get all the delegation right. As a starting point then look at the PassThroughFileSystem in jdk/test tree, this is a provider used by some of the tests and may be what you are looking for. If on the other hand that you are looking for something at the lower level, say to intercept calls to the operating system then there isn't anything in the APi for that. To do customizations at that level requires using a native interposer library, or if Windows then it means using something like Detours [1]. -Alan. [1] http://research.microsoft.com/pubs/68568/huntusenixnt99.pdf From chris.w.dennis at gmail.com Mon Oct 21 12:02:39 2013 From: chris.w.dennis at gmail.com (Chris Dennis) Date: Mon, 21 Oct 2013 15:02:39 -0400 Subject: Bug in interrupt handling in FileChannelImpl.map(=?UTF-8?B?4oCm?=) In-Reply-To: Message-ID: Okay looks like I had an incorrect assertion in the test, (an artifact due to the copy paste I performed from the InterruptDeadlock test). I cannot assert that the channel is closed after all the threads have completed (they could all get through the map call before the interrupting thread gets to them). Attached is a corrected version of the patch. Chris On 10/16/13 8:52 AM, "Chris Dennis" wrote: >I'll look in to the test when I get a chance. I haven't seen it fail this >way - but like I said the close logic is a bit weird in order to try and >make the failure 'clean'. I'll look through it and see if I can figure >out what might be going wrong. > >Chris > >On 10/15/13 11:47 AM, "Alan Bateman" wrote: > >>On 23/09/2013 14:33, Chris Dennis wrote: >>> Sure, no problem. There are a couple of oddities to note in the test? >>> >>> 1. It doesn't actually end up mapping anything. I'm intentionally >>>trying >>> to extend the file from zero length to length one, when the file is >>> read-only. This way I pass through the nested size call, but don't >>> actually end up mapping anything. Seemed to me like this was the >>>simplest >>> way to test exactly this bug and nothing else, and also avoid all the >>> windows specific mapping behaviors causing any problems. >>> 2. I'm not closing the file in a finally block because when the code is >>> broken the locking will block the close call, hence I'm only closing on >>>a >>> successful run. >>> >>Just coming back to this. The patch to FileChannelImpl.map looks fine >>(and the update to when map needs to extend the file is okay too). I'm >>happy to sponsor this and get it in. >> >>I'm not sure about the test yet, it appears to fails intermittently (on >>several platforms), typically because the FileChannel isn't closed. Have >>you seen this? I haven't dug into it yet but we would need the test to >>be very reliable to include it with the patch. >> >>-Alan. >> >> > > -------------- next part -------------- A non-text attachment was scrubbed... Name: 8024833_v2.patch Type: application/octet-stream Size: 7313 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20131021/05198520/8024833_v2.patch From Alan.Bateman at oracle.com Tue Oct 22 02:06:08 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 22 Oct 2013 10:06:08 +0100 Subject: 7074436: (sc) SocketChannel can do short gathering writes when channel configured blocking (win) Message-ID: <52664000.9000203@oracle.com> On Windows then SocketChannel.write has to split large writes into 128k chunks in order to support async close when the channel is configured blocking. This wasn't implemented completely for gathering writes: if the total number of bytes remaining in the buffers exceeds 128k then it just writes up to 128k and returns that as the number of bytes written. This is problematic for applications that don't expect a short write (meaning they don't use the idiom to write until the number of bytes remaining is 0 or they don't check the return value). I'd like to update the gathering write so that it works like the write, this means that if the total remaining is >128k then there will be multiple calls to WSASend to write the bytes. http://cr.openjdk.java.net/~alanb/7074436/webrev/ Thanks, Alan. From Alan.Bateman at oracle.com Tue Oct 22 02:21:43 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 22 Oct 2013 10:21:43 +0100 Subject: Bug in interrupt handling in =?UTF-8?B?RmlsZUNoYW5uZWxJbXBsLg==?= =?UTF-8?B?bWFwKOKApik=?= In-Reply-To: References: Message-ID: <526643A7.2040207@oracle.com> On 21/10/2013 20:02, Chris Dennis wrote: > Okay looks like I had an incorrect assertion in the test, (an artifact due > to the copy paste I performed from the InterruptDeadlock test). I cannot > assert that the channel is closed after all the threads have completed > (they could all get through the map call before the interrupting thread > gets to them). Attached is a corrected version of the patch. > > Chris Thanks for the update, the test looks much better now. As this is a low-risk change then it should be okay to get it into jdk8. I'll let it run on all platforms first to make sure that the test is reliable. -Alan From Alan.Bateman at oracle.com Tue Oct 22 02:36:53 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 22 Oct 2013 10:36:53 +0100 Subject: Question about FileSystemProvider API in JDK7 In-Reply-To: References: <5264DDBE.7080508@oracle.com> Message-ID: <52664735.8080003@oracle.com> On 21/10/2013 20:50, Pisarev, Vitaliy wrote: > I appreciate the quick response. > > I looked at the code of PassThroughFileSystem, and I must say it underlines my point here. In case someone wants to interpose on the default provider, he has to write all this boiler plate code. > And unless they are aware of the PassThroughFileSystem, they will probably not do it right. The point being that the mechanism is not friendly for extension by developers. > > Since delegation is the only feasible method available, I wonder whether you have any plans for introducing facilities that will allow extension by inheritance, or put some other hooks in place? > > The reason I am insisting is because I think that a file system abstraction is a very powerful concept and I would definitely like to know whether you are planning to invest in this in the future. > Replacing or interposing on the default file system provider is really something for more advanced libraries, I don't think it's something that the vast majority of developers will likely do. So this is why there isn't a DelegatingFileSystemProvider or some such support in the API. If there is further effort put into this area then a higher priority (in my view) would be the base classes to use when developing your own file system provider. The feedback from those that developed their own file system providers (for custom file systems usually) is that it's a lot of effort, particular to the get Path operations right. -Alan. From Alan.Bateman at oracle.com Tue Oct 22 04:15:03 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 22 Oct 2013 12:15:03 +0100 Subject: Bug in interrupt handling in =?UTF-8?B?RmlsZUNoYW5uZWxJbXBsLg==?= =?UTF-8?B?bWFwKOKApik=?= In-Reply-To: <526643A7.2040207@oracle.com> References: <526643A7.2040207@oracle.com> Message-ID: <52665E37.6060403@oracle.com> On 22/10/2013 10:21, Alan Bateman wrote: > Thanks for the update, the test looks much better now. As this is a > low-risk change then it should be okay to get it into jdk8. I'll let > it run on all platforms first to make sure that the test is reliable. Thanks again for the update, it ran okay on all platforms so I have pushed the change to jdk8/tl [1], listing you as contributor. -Alan. [1] http://hg.openjdk.java.net/jdk8/tl/jdk/rev/f15ad52cffed From chris.hegarty at oracle.com Tue Oct 22 05:26:44 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 22 Oct 2013 13:26:44 +0100 Subject: 7074436: (sc) SocketChannel can do short gathering writes when channel configured blocking (win) In-Reply-To: <52664000.9000203@oracle.com> References: <52664000.9000203@oracle.com> Message-ID: <52666F04.6060701@oracle.com> Looks good to me Alan, this will be a nice surprise for Windows users. -Chris. On 22/10/2013 10:06, Alan Bateman wrote: > > On Windows then SocketChannel.write has to split large writes into 128k > chunks in order to support async close when the channel is configured > blocking. This wasn't implemented completely for gathering writes: if > the total number of bytes remaining in the buffers exceeds 128k then it > just writes up to 128k and returns that as the number of bytes written. > This is problematic for applications that don't expect a short write > (meaning they don't use the idiom to write until the number of bytes > remaining is 0 or they don't check the return value). > > I'd like to update the gathering write so that it works like the write, > this means that if the total remaining is >128k then there will be > multiple calls to WSASend to write the bytes. > http://cr.openjdk.java.net/~alanb/7074436/webrev/ > > Thanks, > Alan. From Alan.Bateman at oracle.com Mon Oct 28 08:03:51 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 28 Oct 2013 15:03:51 +0000 Subject: [PATCH] 7052549 "(aio) AssertionError in sun.nio.ch.PendingIoCache.clearPendingIoMap (win)" In-Reply-To: References: Message-ID: <526E7CD7.90906@oracle.com> On 10/10/2013 21:28, Yuri Gaevsky wrote: > Hello, > > I'm trying to follow [1] in order to propose a patch for JDK-7052549 [2]. > > The code in clearPendingIoMap() doesn't use 'synchronized' access to internal > data as all other methods do and the 'assert Thread.holdsLock(this)' statement > in the method can fail when "-esa" command-line option enabled. The proposed > patch moves all accesses to 'pendingIoMap' and 'closePending' under synchronized > block, and removes the 'assert !result.isDone()' statement. > > The patch has been verified by JCK testrun with '-esa' option. Yuri - apologies for taking time to get back to you on this. Just to double check, are you 100% sure that its the "assert Thread.holdsLock(this)" that is being being triggered? I ask because this method method is only every called while holding the lock. I wonder if instead you are seeing the "assert !result.isDone()" being triggered. That is what JDK-7052549 is about. I'm also interested to know if you can duplicate this easily? If so then maybe we try out a few theories and get the root of this issue. -Alan.