From philippe.marschall at gmail.com Mon Jul 1 00:57:33 2013 From: philippe.marschall at gmail.com (Philippe Marschall) Date: Mon, 1 Jul 2013 09:57:33 +0200 Subject: mapped io for non-default file system In-Reply-To: <51CB3493.9000004@oracle.com> References: <51CB3493.9000004@oracle.com> Message-ID: On Wed, Jun 26, 2013 at 8:36 PM, Alan Bateman wrote: > On 26/06/2013 19:12, Philippe Marschall wrote: >> >> Hi >> >> Currently it's not possible for a non-default file system to provide >> mapped io. A custom FileSystemProvider can override #newFileChannel >> and return a custom FileChannel. This channel class has to implement >> #map (since it's abstract in FileChannel) and return a >> MappedByteBuffer. However even though MappedByteBuffer is abstract it >> can not be implemented by custom file systems because the constructors >> are package protected and it has final methods that call native >> methods (and require a FileDescriptor). >> >> So my question is whether this is intentional because the semantics >> are supposed to be mmap() or rather an oversight. If it's intentional >> then maybe a comment on FileChannel#map would be helpful to document >> the expected behaviour for non-defaulft file systems >> (UnsupportedOperationException or IOException). >> >> Cheers >> Philippe > > FileSystemProvider's newFileChannel method is specified to throw UOE when > the provider does not support file channels and only the default provider is > required to support file channels. > > So the intention is that if a provider supports FileChannels then it should > implement it completely. I don't think the issue of FileChannels not support > map has come up before. Ok, very well. What about AsynchronousFileChannel then? It's the only way to do asynchronous file IO and offers the same functionality as FileChannel except for #map (although in an asynchronous fashion). So unlike FileChannel it can be implemented fully by non-default file systems. Is the idea that non-default file systems can implement AsynchronousFileChannel but not FileChannel (because they can't implement #map) or should they implement neither? Cheers Philippe From philippe.marschall at gmail.com Mon Jul 1 01:03:40 2013 From: philippe.marschall at gmail.com (Philippe Marschall) Date: Mon, 1 Jul 2013 10:03:40 +0200 Subject: mapped io for non-default file system In-Reply-To: <51CB37C2.3090005@univ-mlv.fr> References: <51CB37C2.3090005@univ-mlv.fr> Message-ID: On Wed, Jun 26, 2013 at 8:49 PM, Remi Forax wrote: > On 06/26/2013 08:12 PM, Philippe Marschall wrote: >> >> Hi >> >> Currently it's not possible for a non-default file system to provide >> mapped io. A custom FileSystemProvider can override #newFileChannel >> and return a custom FileChannel. This channel class has to implement >> #map (since it's abstract in FileChannel) and return a >> MappedByteBuffer. However even though MappedByteBuffer is abstract it >> can not be implemented by custom file systems because the constructors >> are package protected and it has final methods that call native >> methods (and require a FileDescriptor). >> >> So my question is whether this is intentional because the semantics >> are supposed to be mmap() or rather an oversight. If it's intentional >> then maybe a comment on FileChannel#map would be helpful to document >> the expected behaviour for non-defaulft file systems >> (UnsupportedOperationException or IOException). > > > I believe it's intentional, in order to avoid a dynamic dispatch > (or at least some class checks) when doing a get or a put (which will be a > real perf killer), > all NIO buffers inherits from MappedByteBuffer and not the opposite. > It's not pretty, but you can not let users to freely subclass > MappedByteBuffer because of that. Wut? I can't believe HotSpot is still that na?ve, surely it must at least use IC or PIC. Given the overhead of all the JNI methods in MappedByteBuffer I have my doubts. Cheers Philippe From Alan.Bateman at oracle.com Mon Jul 1 01:16:24 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 01 Jul 2013 09:16:24 +0100 Subject: mapped io for non-default file system In-Reply-To: References: <51CB3493.9000004@oracle.com> Message-ID: <51D13AD8.4050304@oracle.com> On 01/07/2013 08:57, Philippe Marschall wrote: > Ok, very well. What about AsynchronousFileChannel then? It's the only > way to do asynchronous file IO and offers the same functionality as > FileChannel except for #map (although in an asynchronous fashion). So > unlike FileChannel it can be implemented fully by non-default file > systems. Is the idea that non-default file systems can implement > AsynchronousFileChannel but not FileChannel (because they can't > implement #map) or should they implement neither? > The intention is that the provider can support both, either or none. We included this statement in the javadoc for both of the factory methods that FileSystemProvider defines: "A provider that does not support all the features required to construct a XXX throws UnsupportedOperationException". Clearly most of the providers that come up will not support either as it's just not practical (even SeekableByteChannel is a challenge in some cases). -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130701/44de4fbb/attachment.html From forax at univ-mlv.fr Mon Jul 1 02:45:21 2013 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 01 Jul 2013 11:45:21 +0200 Subject: mapped io for non-default file system In-Reply-To: References: <51CB37C2.3090005@univ-mlv.fr> Message-ID: <51D14FB1.60502@univ-mlv.fr> On 07/01/2013 10:03 AM, Philippe Marschall wrote: > On Wed, Jun 26, 2013 at 8:49 PM, Remi Forax wrote: >> On 06/26/2013 08:12 PM, Philippe Marschall wrote: >>> Hi >>> >>> Currently it's not possible for a non-default file system to provide >>> mapped io. A custom FileSystemProvider can override #newFileChannel >>> and return a custom FileChannel. This channel class has to implement >>> #map (since it's abstract in FileChannel) and return a >>> MappedByteBuffer. However even though MappedByteBuffer is abstract it >>> can not be implemented by custom file systems because the constructors >>> are package protected and it has final methods that call native >>> methods (and require a FileDescriptor). >>> >>> So my question is whether this is intentional because the semantics >>> are supposed to be mmap() or rather an oversight. If it's intentional >>> then maybe a comment on FileChannel#map would be helpful to document >>> the expected behaviour for non-defaulft file systems >>> (UnsupportedOperationException or IOException). >> >> I believe it's intentional, in order to avoid a dynamic dispatch >> (or at least some class checks) when doing a get or a put (which will be a >> real perf killer), >> all NIO buffers inherits from MappedByteBuffer and not the opposite. >> It's not pretty, but you can not let users to freely subclass >> MappedByteBuffer because of that. > Wut? I can't believe HotSpot is still that na?ve, surely it must at least use IC or PIC. yes, Hotspot will use an inlining cache. ByteBuffer get/put try to compete with C/C++ unsafe array access, so the overhead of such cache, even if in the best case of the monomorphic inlining cache the overhead is just a pointer check against a constant, this overhead is for that particular case just an unnecessary overhead. > Given the overhead of all the JNI methods in MappedByteBuffer I have my doubts. These methods are not called often or exactly should not be called often. > > Cheers > Philippe cheers, R?mi From philippe.marschall at gmail.com Mon Jul 1 05:37:57 2013 From: philippe.marschall at gmail.com (Philippe Marschall) Date: Mon, 1 Jul 2013 14:37:57 +0200 Subject: mapped io for non-default file system In-Reply-To: <51D14FB1.60502@univ-mlv.fr> References: <51CB37C2.3090005@univ-mlv.fr> <51D14FB1.60502@univ-mlv.fr> Message-ID: On Mon, Jul 1, 2013 at 11:45 AM, Remi Forax wrote: > On 07/01/2013 10:03 AM, Philippe Marschall wrote: >> >> On Wed, Jun 26, 2013 at 8:49 PM, Remi Forax wrote: >>> >>> On 06/26/2013 08:12 PM, Philippe Marschall wrote: >>>> >>>> Hi >>>> >>>> Currently it's not possible for a non-default file system to provide >>>> mapped io. A custom FileSystemProvider can override #newFileChannel >>>> and return a custom FileChannel. This channel class has to implement >>>> #map (since it's abstract in FileChannel) and return a >>>> MappedByteBuffer. However even though MappedByteBuffer is abstract it >>>> can not be implemented by custom file systems because the constructors >>>> are package protected and it has final methods that call native >>>> methods (and require a FileDescriptor). >>>> >>>> So my question is whether this is intentional because the semantics >>>> are supposed to be mmap() or rather an oversight. If it's intentional >>>> then maybe a comment on FileChannel#map would be helpful to document >>>> the expected behaviour for non-defaulft file systems >>>> (UnsupportedOperationException or IOException). >>> >>> >>> I believe it's intentional, in order to avoid a dynamic dispatch >>> (or at least some class checks) when doing a get or a put (which will be >>> a >>> real perf killer), >>> all NIO buffers inherits from MappedByteBuffer and not the opposite. >>> It's not pretty, but you can not let users to freely subclass >>> MappedByteBuffer because of that. >> >> Wut? I can't believe HotSpot is still that na?ve, surely it must at least >> use IC or PIC. > > > yes, Hotspot will use an inlining cache. > ByteBuffer get/put try to compete with C/C++ unsafe array access, so the > overhead of such cache, > even if in the best case of the monomorphic inlining cache the overhead is > just a pointer check against a constant, > this overhead is for that particular case just an unnecessary overhead. Can you even measure that on a modern CPU? And that's assuming the worst case when you actually have another implementation class loaded and CHA does not kick in. >> Given the overhead of all the JNI methods in MappedByteBuffer I have my >> doubts. > > > These methods are not called often or exactly should not be called often. Why do we care about their call overhead then? Cheers Philippe From forax at univ-mlv.fr Mon Jul 1 06:36:19 2013 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 01 Jul 2013 15:36:19 +0200 Subject: mapped io for non-default file system In-Reply-To: References: <51CB37C2.3090005@univ-mlv.fr> <51D14FB1.60502@univ-mlv.fr> Message-ID: <51D185D3.3090109@univ-mlv.fr> On 07/01/2013 02:37 PM, Philippe Marschall wrote: > On Mon, Jul 1, 2013 at 11:45 AM, Remi Forax wrote: >> On 07/01/2013 10:03 AM, Philippe Marschall wrote: >>> On Wed, Jun 26, 2013 at 8:49 PM, Remi Forax wrote: >>>> On 06/26/2013 08:12 PM, Philippe Marschall wrote: >>>>> Hi >>>>> >>>>> Currently it's not possible for a non-default file system to provide >>>>> mapped io. A custom FileSystemProvider can override #newFileChannel >>>>> and return a custom FileChannel. This channel class has to implement >>>>> #map (since it's abstract in FileChannel) and return a >>>>> MappedByteBuffer. However even though MappedByteBuffer is abstract it >>>>> can not be implemented by custom file systems because the constructors >>>>> are package protected and it has final methods that call native >>>>> methods (and require a FileDescriptor). >>>>> >>>>> So my question is whether this is intentional because the semantics >>>>> are supposed to be mmap() or rather an oversight. If it's intentional >>>>> then maybe a comment on FileChannel#map would be helpful to document >>>>> the expected behaviour for non-defaulft file systems >>>>> (UnsupportedOperationException or IOException). >>>> >>>> I believe it's intentional, in order to avoid a dynamic dispatch >>>> (or at least some class checks) when doing a get or a put (which will be >>>> a >>>> real perf killer), >>>> all NIO buffers inherits from MappedByteBuffer and not the opposite. >>>> It's not pretty, but you can not let users to freely subclass >>>> MappedByteBuffer because of that. >>> Wut? I can't believe HotSpot is still that na?ve, surely it must at least >>> use IC or PIC. >> >> yes, Hotspot will use an inlining cache. >> ByteBuffer get/put try to compete with C/C++ unsafe array access, so the >> overhead of such cache, >> even if in the best case of the monomorphic inlining cache the overhead is >> just a pointer check against a constant, >> this overhead is for that particular case just an unnecessary overhead. > Can you even measure that on a modern CPU? And that's assuming the > worst case when you actually have another implementation class loaded > and CHA does not kick in. get and put are basic primitives that are used every-where in IO code, so all the compiled codes dealing with IOs will need to be trashed if CHA/IC failed, this will create a giant deoptimization flood (not something pretty believe me). > >>> Given the overhead of all the JNI methods in MappedByteBuffer I have my >>> doubts. >> >> These methods are not called often or exactly should not be called often. > Why do we care about their call overhead then? these methods => JNI methods. get/put are called very often in loops, force by example is not called very often. > > Cheers > Philippe cheers, R?mi From philippe.marschall at gmail.com Mon Jul 1 08:54:23 2013 From: philippe.marschall at gmail.com (Philippe Marschall) Date: Mon, 1 Jul 2013 17:54:23 +0200 Subject: mapped io for non-default file system In-Reply-To: <51D185D3.3090109@univ-mlv.fr> References: <51CB37C2.3090005@univ-mlv.fr> <51D14FB1.60502@univ-mlv.fr> <51D185D3.3090109@univ-mlv.fr> Message-ID: On Mon, Jul 1, 2013 at 3:36 PM, Remi Forax wrote: > On 07/01/2013 02:37 PM, Philippe Marschall wrote: >> >> On Mon, Jul 1, 2013 at 11:45 AM, Remi Forax wrote: >>> >>> On 07/01/2013 10:03 AM, Philippe Marschall wrote: >>>> >>>> On Wed, Jun 26, 2013 at 8:49 PM, Remi Forax wrote: >>>>> >>>>> On 06/26/2013 08:12 PM, Philippe Marschall wrote: >>>>>> >>>>>> Hi >>>>>> >>>>>> Currently it's not possible for a non-default file system to provide >>>>>> mapped io. A custom FileSystemProvider can override #newFileChannel >>>>>> and return a custom FileChannel. This channel class has to implement >>>>>> #map (since it's abstract in FileChannel) and return a >>>>>> MappedByteBuffer. However even though MappedByteBuffer is abstract it >>>>>> can not be implemented by custom file systems because the constructors >>>>>> are package protected and it has final methods that call native >>>>>> methods (and require a FileDescriptor). >>>>>> >>>>>> So my question is whether this is intentional because the semantics >>>>>> are supposed to be mmap() or rather an oversight. If it's intentional >>>>>> then maybe a comment on FileChannel#map would be helpful to document >>>>>> the expected behaviour for non-defaulft file systems >>>>>> (UnsupportedOperationException or IOException). >>>>> >>>>> >>>>> I believe it's intentional, in order to avoid a dynamic dispatch >>>>> (or at least some class checks) when doing a get or a put (which will >>>>> be >>>>> a >>>>> real perf killer), >>>>> all NIO buffers inherits from MappedByteBuffer and not the opposite. >>>>> It's not pretty, but you can not let users to freely subclass >>>>> MappedByteBuffer because of that. >>>> >>>> Wut? I can't believe HotSpot is still that na?ve, surely it must at >>>> least >>>> use IC or PIC. >>> >>> >>> yes, Hotspot will use an inlining cache. >>> ByteBuffer get/put try to compete with C/C++ unsafe array access, so the >>> overhead of such cache, >>> even if in the best case of the monomorphic inlining cache the overhead >>> is >>> just a pointer check against a constant, >>> this overhead is for that particular case just an unnecessary overhead. >> >> Can you even measure that on a modern CPU? And that's assuming the >> worst case when you actually have another implementation class loaded >> and CHA does not kick in. > > > get and put are basic primitives that are used every-where in IO code, > so all the compiled codes dealing with IOs will need to be trashed if CHA/IC > failed, > this will create a giant deoptimization flood (not something pretty believe > me). > > >> >>>> Given the overhead of all the JNI methods in MappedByteBuffer I have >>>> my >>>> doubts. >>> >>> >>> These methods are not called often or exactly should not be called often. >> >> Why do we care about their call overhead then? > > > > these methods => JNI methods. > get/put are called very often in loops, force by example is not called very > often. But get/put are abstract in ByteBuffer and have two implementations, one in HeapByteBuffer and one in DirectByteBuffer. (sorry for OT) Cheers Philippe From Alan.Bateman at oracle.com Thu Jul 4 02:42:49 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 04 Jul 2013 10:42:49 +0100 Subject: possible NIO selector leak in 7u25 In-Reply-To: References: Message-ID: <51D54399.20302@oracle.com> On 04/07/2013 09:36, Bernd Eckenfels wrote: > Hello, > > we see a possible handle/selector leak very similiar to this bug: > > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7118373 > > We see on linux unix domain sockets and on windows /dev/afd handles > which are not backed up by any socket/selector/handle/channel in the > heapdump. This is a applicxation using NIO (via JBoss XNIO). We would > like to nail down the source of it and therefore it would be good if > we have the actual code from the old bug above "e.java" to see if this > helps us to reproduce the problem. Can any Oracle developer with > access to it sent it? (and cvonfirm the bug is fixed) > > In our case we see >500 of those internal sockets, but the related tcp > sockets (and java objects) are gone. > > Bernd Do you know if JBoss XNIO uses socket adapters to do timed reads? These were implemented as temporary Selectors and cached on a per thread basis. We've replaced this in jdk8 but for jdk7 and older then it is possible to have scenarios where there are a lot of threads or short-lived threads and the temporarily Selectors hang around until they are GC'ed. If you are using lsof or equivalent then it would be visible as an apparently build up of Unix domain sockets. You mention that the heap dump doesn't appear to have references and that make sense if the heap dump is generated from only the live objects (would be interesting to know if the Unix domain sockets are closed as as result of a heap dump that does a full GC first). There isn't an "e.java" attached to the bug. This bug is actually a "shadow bug" for something that came via another support/customer system so it has been filtered. When Rob fixed 7118373 then he wasn't able to come up with a reliable test case that demonstrated the issue in a reasonable amount of time. So are you able to duplicate the issue in your environment? I'm just wondering if you could try a preview build of 8 or event 7u40 to double check that the issue still duplicates. -Alan From david.lloyd at redhat.com Thu Jul 4 11:43:33 2013 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 04 Jul 2013 13:43:33 -0500 Subject: possible NIO selector leak in 7u25 In-Reply-To: <51D54399.20302@oracle.com> References: <51D54399.20302@oracle.com> Message-ID: <51D5C255.4000306@redhat.com> On 7/4/13 4:42 AM, Alan Bateman wrote: > On 04/07/2013 09:36, Bernd Eckenfels wrote: >> Hello, >> >> we see a possible handle/selector leak very similiar to this bug: >> >> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7118373 >> >> We see on linux unix domain sockets and on windows /dev/afd handles >> which are not backed up by any socket/selector/handle/channel in the >> heapdump. This is a applicxation using NIO (via JBoss XNIO). We would >> like to nail down the source of it and therefore it would be good if >> we have the actual code from the old bug above "e.java" to see if this >> helps us to reproduce the problem. Can any Oracle developer with >> access to it sent it? (and cvonfirm the bug is fixed) >> >> In our case we see >500 of those internal sockets, but the related tcp >> sockets (and java objects) are gone. >> >> Bernd > > Do you know if JBoss XNIO uses socket adapters to do timed reads? These > were implemented as temporary Selectors and cached on a per thread > basis. We've replaced this in jdk8 but for jdk7 and older then it is > possible to have scenarios where there are a lot of threads or > short-lived threads and the temporarily Selectors hang around until they > are GC'ed. If you are using lsof or equivalent then it would be visible > as an apparently build up of Unix domain sockets. You mention that the > heap dump doesn't appear to have references and that make sense if the > heap dump is generated from only the live objects (would be interesting > to know if the Unix domain sockets are closed as as result of a heap > dump that does a full GC first). > > There isn't an "e.java" attached to the bug. This bug is actually a > "shadow bug" for something that came via another support/customer system > so it has been filtered. When Rob fixed 7118373 then he wasn't able to > come up with a reliable test case that demonstrated the issue in a > reasonable amount of time. > > So are you able to duplicate the issue in your environment? I'm just > wondering if you could try a preview build of 8 or event 7u40 to double > check that the issue still duplicates. XNIO uses Selectors (usually PollSelectorImpls) which are cached per thread in order to mix blocking and non-blocking I/O. If you are starting many short-lived threads and doing blocking operations on XNIO channels then this might explain what is happening. The answer is basically "don't do that". -- - DML From Alan.Bateman at oracle.com Thu Jul 4 11:53:15 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 04 Jul 2013 19:53:15 +0100 Subject: possible NIO selector leak in 7u25 In-Reply-To: <51D5C255.4000306@redhat.com> References: <51D54399.20302@oracle.com> <51D5C255.4000306@redhat.com> Message-ID: <51D5C49B.9080700@oracle.com> On 04/07/2013 19:43, David M. Lloyd wrote: > > XNIO uses Selectors (usually PollSelectorImpls) which are cached per > thread in order to mix blocking and non-blocking I/O. If you are > starting many short-lived threads and doing blocking operations on > XNIO channels then this might explain what is happening. The answer > is basically "don't do that". > When you say "usually PollSelectorImpls" then it normally run with it configured to use the poll based Selector? BTW: Sean Coffey mailed me off-list so say that he tried the original support/customer test case that lead to 7118373 and confirms that it doesn't duplicate with 7u25. Anyway, it would be good to hear more from Bernd to know if this duplicates with 7u40 or 8. Also interesting to know if this is a case of lots of short-lived threads doing timed reads on socket adapters (which was the motive for replace this part of the implementation in 8). -Alan. From david.lloyd at redhat.com Thu Jul 4 13:50:08 2013 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 04 Jul 2013 15:50:08 -0500 Subject: possible NIO selector leak in 7u25 In-Reply-To: <51D5C49B.9080700@oracle.com> References: <51D54399.20302@oracle.com> <51D5C255.4000306@redhat.com> <51D5C49B.9080700@oracle.com> Message-ID: <51D5E000.9050907@redhat.com> On 7/4/13 1:53 PM, Alan Bateman wrote: > On 04/07/2013 19:43, David M. Lloyd wrote: >> >> XNIO uses Selectors (usually PollSelectorImpls) which are cached per >> thread in order to mix blocking and non-blocking I/O. If you are >> starting many short-lived threads and doing blocking operations on >> XNIO channels then this might explain what is happening. The answer >> is basically "don't do that". >> > When you say "usually PollSelectorImpls" then it normally run with it > configured to use the poll based Selector? Yeah it will prefer PollSelectorImpl for temporary selectors, but will ultimately fall back to whatever the default provider gives. > BTW: Sean Coffey mailed me off-list so say that he tried the original > support/customer test case that lead to 7118373 and confirms that it > doesn't duplicate with 7u25. Good to know, thanks. -- - DML From bernd-2013 at eckenfels.net Thu Jul 4 13:31:03 2013 From: bernd-2013 at eckenfels.net (Bernd Eckenfels) Date: Thu, 04 Jul 2013 22:31:03 +0200 Subject: possible NIO selector leak in 7u25 In-Reply-To: <51D5C49B.9080700@oracle.com> References: <51D54399.20302@oracle.com> <51D5C255.4000306@redhat.com> <51D5C49B.9080700@oracle.com> Message-ID: Hello, I will have a look at 7u40 and 8 as soon as I get some time. But I dont think it is thread(caching) related as it does not show up in the heapdump (and we have a controled thread creation rate). It looks more like some close/cancel interactions (and a real leak). I will let you know what I find. Thanks David, Alan and Sean! Greetings Bernd From Alan.Bateman at oracle.com Fri Jul 5 00:31:03 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 05 Jul 2013 08:31:03 +0100 Subject: possible NIO selector leak in 7u25 In-Reply-To: <51D5E000.9050907@redhat.com> References: <51D54399.20302@oracle.com> <51D5C255.4000306@redhat.com> <51D5C49B.9080700@oracle.com> <51D5E000.9050907@redhat.com> Message-ID: <51D67637.6020600@oracle.com> On 04/07/2013 21:50, David M. Lloyd wrote: > On 7/4/13 1:53 PM, Alan Bateman wrote: >> On 04/07/2013 19:43, David M. Lloyd wrote: >>> >>> XNIO uses Selectors (usually PollSelectorImpls) which are cached per >>> thread in order to mix blocking and non-blocking I/O. If you are >>> starting many short-lived threads and doing blocking operations on >>> XNIO channels then this might explain what is happening. The answer >>> is basically "don't do that". >>> >> When you say "usually PollSelectorImpls" then it normally run with it >> configured to use the poll based Selector? > > Yeah it will prefer PollSelectorImpl for temporary selectors, but will > ultimately fall back to whatever the default provider gives. So I guess it must be make direct use of sun.nio.ch? Just curious as I can't see how this would work otherwise. -Alan. From david.lloyd at redhat.com Fri Jul 5 06:37:29 2013 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 05 Jul 2013 08:37:29 -0500 Subject: possible NIO selector leak in 7u25 In-Reply-To: <51D67637.6020600@oracle.com> References: <51D54399.20302@oracle.com> <51D5C255.4000306@redhat.com> <51D5C49B.9080700@oracle.com> <51D5E000.9050907@redhat.com> <51D67637.6020600@oracle.com> Message-ID: <51D6CC19.708@redhat.com> On 7/5/13 2:31 AM, Alan Bateman wrote: > On 04/07/2013 21:50, David M. Lloyd wrote: >> On 7/4/13 1:53 PM, Alan Bateman wrote: >>> On 04/07/2013 19:43, David M. Lloyd wrote: >>>> >>>> XNIO uses Selectors (usually PollSelectorImpls) which are cached per >>>> thread in order to mix blocking and non-blocking I/O. If you are >>>> starting many short-lived threads and doing blocking operations on >>>> XNIO channels then this might explain what is happening. The answer >>>> is basically "don't do that". >>>> >>> When you say "usually PollSelectorImpls" then it normally run with it >>> configured to use the poll based Selector? >> >> Yeah it will prefer PollSelectorImpl for temporary selectors, but will >> ultimately fall back to whatever the default provider gives. > So I guess it must be make direct use of sun.nio.ch? Just curious as I > can't see how this would work otherwise. Correct, though in a somewhat defensive manner. It'll still work if that package is missing, just by using the default. -- - DML From bernd-2013 at eckenfels.net Fri Jul 5 07:05:29 2013 From: bernd-2013 at eckenfels.net (Bernd Eckenfels) Date: Fri, 5 Jul 2013 16:05:29 +0200 Subject: possible NIO selector leak in 7u25 In-Reply-To: <51D6CC19.708@redhat.com> References: <51D54399.20302@oracle.com> <51D5C255.4000306@redhat.com> <51D5C49B.9080700@oracle.com> <51D5E000.9050907@redhat.com> <51D67637.6020600@oracle.com> <51D6CC19.708@redhat.com> Message-ID: Hello, I have two updates on the Issue: A) same Problem exists with June 7u40 EA JDK B) there are no threads starting/terminating in the test In our test we run a workload with 1 transaction per second and it -- bernd.eckenfels.net Am 05.07.2013 um 15:37 schrieb "David M. Lloyd" : > On 7/5/13 2:31 AM, Alan Bateman wrote: >> On 04/07/2013 21:50, David M. Lloyd wrote: >>> On 7/4/13 1:53 PM, Alan Bateman wrote: >>>> On 04/07/2013 19:43, David M. Lloyd wrote: >>>>> >>>>> XNIO uses Selectors (usually PollSelectorImpls) which are cached per >>>>> thread in order to mix blocking and non-blocking I/O. If you are >>>>> starting many short-lived threads and doing blocking operations on >>>>> XNIO channels then this might explain what is happening. The answer >>>>> is basically "don't do that". >>>>> >>>> When you say "usually PollSelectorImpls" then it normally run with it >>>> configured to use the poll based Selector? >>> >>> Yeah it will prefer PollSelectorImpl for temporary selectors, but will >>> ultimately fall back to whatever the default provider gives. >> So I guess it must be make direct use of sun.nio.ch? Just curious as I >> can't see how this would work otherwise. > > Correct, though in a somewhat defensive manner. It'll still work if that package is missing, just by using the default. > -- > - DML From bernd-2013 at eckenfels.net Fri Jul 5 07:27:06 2013 From: bernd-2013 at eckenfels.net (Bernd Eckenfels) Date: Fri, 5 Jul 2013 16:27:06 +0200 Subject: possible NIO selector leak in 7u25 In-Reply-To: References: <51D54399.20302@oracle.com> <51D5C255.4000306@redhat.com> <51D5C49B.9080700@oracle.com> <51D5E000.9050907@redhat.com> <51D67637.6020600@oracle.com> <51D6CC19.708@redhat.com> Message-ID: <916A84EB-20A4-4DA9-85B7-5459CDCA963D@eckenfels.net> Sorry I should not write those mails on the mobile - full version below. Am 05.07.2013 um 16:05 schrieb Bernd Eckenfels : > Hello, > > I have two updates on the Issue: > > A) same Problem exists with June 7u40 EA JDK > B) there are no threads starting/terminating in the test > > In our test we run a workload with 1 transaction per second and it leaks one socket each. They don't get cleaned up in a FullGC and when stopping the load generator they don't clear up after longer time. What we found is, that we don't pass shutdownRead/Write to NioTCPChannel (only close) the leak is gone. From rob.mckenna at oracle.com Fri Jul 5 07:51:45 2013 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 05 Jul 2013 15:51:45 +0100 Subject: possible NIO selector leak in 7u25 In-Reply-To: <916A84EB-20A4-4DA9-85B7-5459CDCA963D@eckenfels.net> References: <51D54399.20302@oracle.com> <51D5C255.4000306@redhat.com> <51D5C49B.9080700@oracle.com> <51D5E000.9050907@redhat.com> <51D67637.6020600@oracle.com> <51D6CC19.708@redhat.com> <916A84EB-20A4-4DA9-85B7-5459CDCA963D@eckenfels.net> Message-ID: <51D6DD81.6070801@oracle.com> Hi Bernd, Is there any chance you could supply this test? Also, can you confirm whether this is a regression from 7u25? -Rob On 05/07/13 15:27, Bernd Eckenfels wrote: > Sorry I should not write those mails on the mobile - full version below. > > Am 05.07.2013 um 16:05 schrieb Bernd Eckenfels : > >> Hello, >> >> I have two updates on the Issue: >> >> A) same Problem exists with June 7u40 EA JDK >> B) there are no threads starting/terminating in the test >> >> In our test we run a workload with 1 transaction per second and it leaks one socket each. They don't get cleaned up in a FullGC and when stopping the load generator they don't clear up after longer time. What we found is, that we don't pass shutdownRead/Write to NioTCPChannel (only close) the leak is gone. From Alan.Bateman at oracle.com Fri Jul 5 08:11:27 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 05 Jul 2013 16:11:27 +0100 Subject: possible NIO selector leak in 7u25 In-Reply-To: <51D6DD81.6070801@oracle.com> References: <51D54399.20302@oracle.com> <51D5C255.4000306@redhat.com> <51D5C49B.9080700@oracle.com> <51D5E000.9050907@redhat.com> <51D67637.6020600@oracle.com> <51D6CC19.708@redhat.com> <916A84EB-20A4-4DA9-85B7-5459CDCA963D@eckenfels.net> <51D6DD81.6070801@oracle.com> Message-ID: <51D6E21F.1090300@oracle.com> On 05/07/2013 15:51, Rob McKenna wrote: > Hi Bernd, > > Is there any chance you could supply this test? Also, can you confirm > whether this is a regression from 7u25? > > -Rob That's a good point, I don't think Bernd was clear in the original mail as to whether he believed this was a regression in 7u25 or not. -Alan. From Alan.Bateman at oracle.com Fri Jul 5 08:13:38 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 05 Jul 2013 16:13:38 +0100 Subject: possible NIO selector leak in 7u25 In-Reply-To: <51D6CC19.708@redhat.com> References: <51D54399.20302@oracle.com> <51D5C255.4000306@redhat.com> <51D5C49B.9080700@oracle.com> <51D5E000.9050907@redhat.com> <51D67637.6020600@oracle.com> <51D6CC19.708@redhat.com> Message-ID: <51D6E2A2.7070508@oracle.com> On 05/07/2013 14:37, David M. Lloyd wrote: > > Correct, though in a somewhat defensive manner. It'll still work if > that package is missing, just by using the default. Do you know if there is anything that Bernd can configured to disable that? Just on the off-chance that this is something in the poll Selector (which tends not to be used much these days). -Alan From bernd-2013 at eckenfels.net Mon Jul 8 15:22:36 2013 From: bernd-2013 at eckenfels.net (Bernd Eckenfels) Date: Tue, 09 Jul 2013 00:22:36 +0200 Subject: possible NIO selector leak in 7u25 In-Reply-To: <916A84EB-20A4-4DA9-85B7-5459CDCA963D@eckenfels.net> References: <51D54399.20302@oracle.com> <51D5C255.4000306@redhat.com> <51D5C49B.9080700@oracle.com> <51D5E000.9050907@redhat.com> <51D67637.6020600@oracle.com> <51D6CC19.708@redhat.com> <916A84EB-20A4-4DA9-85B7-5459CDCA963D@eckenfels.net> Message-ID: Hello, we found a cause for the leak, we did not use the latest xnio-nio release. Looking at the NioTcpChannel code I guess that for example this commit could fix a potential problem (we shutdown r+w before we close, which would not be passed on by the older xnio-nio): https://github.com/xnio/xnio/commit/71ebef70d11eedce9a0eb7e5de4d37ab22648b73 When we update the implementation to 3.0.8.GA we dont see the handles accumulate anymore. So sorry for blaming NIO (I am still not sure why this does not show up in the heapdumps. I have done some test programs where I havent closed() the channels or havent selected on the canceled keys. And in all those conditions I see Java heap objects representing the (open) FDs. But I dont see them in our test environment). Oh well... thanks for listening, it always helps to explain your problems to others, especially if they are home made :) Bernd From rob.mckenna at oracle.com Tue Jul 9 10:03:10 2013 From: rob.mckenna at oracle.com (Rob McKenna) Date: Tue, 09 Jul 2013 18:03:10 +0100 Subject: possible NIO selector leak in 7u25 In-Reply-To: References: <51D54399.20302@oracle.com> <51D5C255.4000306@redhat.com> <51D5C49B.9080700@oracle.com> <51D5E000.9050907@redhat.com> <51D67637.6020600@oracle.com> <51D6CC19.708@redhat.com> <916A84EB-20A4-4DA9-85B7-5459CDCA963D@eckenfels.net> Message-ID: <51DC424E.3000009@oracle.com> Glad to hear you got to the bottom of it. -Rob On 08/07/13 23:22, Bernd Eckenfels wrote: > Hello, > > we found a cause for the leak, we did not use the latest xnio-nio > release. Looking at the NioTcpChannel code I guess that for example > this commit could fix a potential problem (we shutdown r+w before we > close, which would not be passed on by the older xnio-nio): > > https://github.com/xnio/xnio/commit/71ebef70d11eedce9a0eb7e5de4d37ab22648b73 > > > When we update the implementation to 3.0.8.GA we dont see the handles > accumulate anymore. So sorry for blaming NIO (I am still not sure why > this does not show up in the heapdumps. I have done some test programs > where I havent closed() the channels or havent selected on the > canceled keys. And in all those conditions I see Java heap objects > representing the (open) FDs. But I dont see them in our test > environment). > > Oh well... thanks for listening, it always helps to explain your > problems to others, especially if they are home made :) > > Bernd From chris.hegarty at oracle.com Sun Jul 21 09:11:01 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sun, 21 Jul 2013 17:11:01 +0100 Subject: RFR [8020669] java.nio.file.Files.readAllBytes() does not read any data when Files.size() is 0 In-Reply-To: <51EB0BFF.3030106@oracle.com> References: <51E97416.20506@oracle.com> <51E98043.6030102@oracle.com> <51E9853E.8020409@redhat.com> <51E985AD.60401@redhat.com> <51EB0BFF.3030106@oracle.com> Message-ID: <293FF6B8-918A-4A2C-9541-A3B36629C9FB@oracle.com> [ cc'ing nio-dev ] On 20 Jul 2013, at 23:15, Ivan Gerasimov wrote: > Roger, David thanks for suggestions! > > Would you please take a look at an updated webrev? > http://cr.openjdk.java.net/~igerasim/8020669/1/webrev/ This looks much better to me. -Chris. > > - File size is used as an initial size of BAOS's buffer. > - BAOS avoids copying its buffer in toByteArray(), if size is correct . > > I don't want to initialize BAOS with a positive number if size happened to be zero. > Because zero could indicate that the file is really empty. > > Sincerely yours, > Ivan > > On 19.07.2013 22:30, David M. Lloyd wrote: >> My mistake, we're not talking about strings. Still you can subclass and determine whether the buffer size guess was right, and if so return the array as-is (swap in an empty array or something as needed). >> >> On 07/19/2013 01:28 PM, David M. Lloyd wrote: >>> It's trivial to subclass ByteArrayOutputStream and add a method which >>> converts its contents to a string using the two protected fields which >>> give you all the info you need to do so. So no extra copy is needed >>> that you aren't already doing. >>> >>> On 07/19/2013 01:06 PM, roger riggs wrote: >>>> Hi Ivan, >>>> >>>> I think this change takes too big a hit for the cases where the size is >>>> correct. >>>> >>>> No real file system can be wrong about the size of a file so this is a >>>> problem >>>> only for special file systems. If the problem is with size reporting zero >>>> then maybe using the incremental read only for size == would be a better >>>> fix. >>>> >>>> At least you could pass the size to the constructor for BAOS and avoid >>>> the thrashing for every re-size; but still it will allocate and create >>>> an extra copy >>>> of the every time. >>>> >>>> $.02, Roger >>>> >>>> >>>> On 7/19/2013 1:15 PM, Ivan Gerasimov wrote: >>>>> Hello everybody! >>>>> >>>>> Would you please review a fix for the problem with >>>>> j.n.f.Files.readAllBytes() function? >>>>> The current implementation relies on FileChannel.size() to preallocate >>>>> a buffer for the whole file's content. >>>>> However, some special filesystems can report a wrong size. >>>>> An example is procfs under Linux, which reports many files under /proc >>>>> to be zero sized. >>>>> >>>>> Thus it is proposed not to rely on the size() and instead continuously >>>>> read until EOF. >>>>> >>>>> The downside is reallocation and copying file content between buffers. >>>>> But taking into account that the doc says: "It is not intended for >>>>> reading in large files." it should not be a big problem. >>>>> >>>>> http://bugs.sun.com/view_bug.do?bug_id=8020669 >>>>> http://cr.openjdk.java.net/~igerasim/8020669/0/webrev/ >>>>> >>>>> The fix is for JDK8. If it is approved, it can be applied to JDK7 as >>>>> well. >>>>> >>>>> Sincerely yours, >>>>> Ivan Gerasimov > From Alan.Bateman at oracle.com Thu Jul 25 21:54:09 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 25 Jul 2013 21:54:09 -0700 Subject: RFR [8020669] java.nio.file.Files.readAllBytes() does not read any data when Files.size() is 0 In-Reply-To: <51F1BB22.6040405@oracle.com> References: <51E97416.20506@oracle.com> <51E98043.6030102@oracle.com> <51E9853E.8020409@redhat.com> <51E985AD.60401@redhat.com> <51EB0BFF.3030106@oracle.com> <51ED57D8.90001@oracle.com> <51EE55AC.5000902@oracle.com> <51EE8E9A.6040806@redhat.com> <51EF1B43.9030200@oracle.com> <51F06789.1050403@oracle.com> <51F12B3E.4010603@oracle.com> <51F1BB22.6040405@oracle.com> Message-ID: <51F200F1.8010500@oracle.com> On 25/07/2013 16:56, Ivan Gerasimov wrote: > > Would you please help review an updated webrev? > http://cr.openjdk.java.net/~igerasim/8020669/4/webrev/ Sorry for the late response to your mails on this, I was on vacation. As you have found, in the original implementation we allowed for resizing (truncation or extending) while the file was open but when it was optimized to avoid an unnecessary resize of the array then we lost the ability to deal with extending case (which is the essentially the same as when the size is reported is less than the actual size). Reverting to the original implementation and doing a single read to double check for EOF seems reasonable. My only real comment on the latest webrev (4) is that the computation of the new capacity is not obvious to the reader (and since this is the uncommon case then it doesn't need to be super-optimized). So I think it would be a lot clearer if "newCapacity" were re-introduced and then computed to a value that is a minimum of capacity+2, maximum of capacity+BUFFER_SIZE (with OOME handling when newCapacity is greater than MAX_BUFFER_SIZE). I think that would be clearer than re-using the remaining count (rem). A minor comment is that the input stream is named "fis" but probably should be "in" as it is not a FileInputStream (I see David Llyod suggested using a FileInputStream but the path may be to a file in a custom file system so it can't be a FileInputStream). I see you've changed the javadoc from "Read" to "Reads" on several methods. I don't know if you meant to include this in the webrev but personally prefer "Read". Thanks for moving the test to BytesAndLines as that is the test for these methods. One suggestion is to move the new test for readAllBytes into testReadAndWriteBytes and the new test for readAllLines into testReadLines. It doesn't matter of course, it just keeps them together. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130725/e91e258f/attachment.html From martinrb at google.com Fri Jul 26 06:10:14 2013 From: martinrb at google.com (Martin Buchholz) Date: Fri, 26 Jul 2013 06:10:14 -0700 Subject: RFR [8020669] java.nio.file.Files.readAllBytes() does not read any data when Files.size() is 0 In-Reply-To: <51F200F1.8010500@oracle.com> References: <51E97416.20506@oracle.com> <51E98043.6030102@oracle.com> <51E9853E.8020409@redhat.com> <51E985AD.60401@redhat.com> <51EB0BFF.3030106@oracle.com> <51ED57D8.90001@oracle.com> <51EE55AC.5000902@oracle.com> <51EE8E9A.6040806@redhat.com> <51EF1B43.9030200@oracle.com> <51F06789.1050403@oracle.com> <51F12B3E.4010603@oracle.com> <51F1BB22.6040405@oracle.com> <51F200F1.8010500@oracle.com> Message-ID: On Thu, Jul 25, 2013 at 9:54 PM, Alan Bateman wrote: > > I see you've changed the javadoc from "Read" to "Reads" on several > methods. I don't know if you meant to include this in the webrev but > personally prefer "Read". > I'm surprised to see you say that, since the "Reads" form has been standard practice for public javadoc for a long time, and private javadoc, while much more informal than public, may as well not be gratuitously different from public. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130726/ea846549/attachment.html From ivan.gerasimov at oracle.com Sun Jul 28 15:08:58 2013 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 29 Jul 2013 02:08:58 +0400 Subject: RFR [8020669] java.nio.file.Files.readAllBytes() does not read any data when Files.size() is 0 In-Reply-To: <51F200F1.8010500@oracle.com> References: <51E97416.20506@oracle.com> <51E98043.6030102@oracle.com> <51E9853E.8020409@redhat.com> <51E985AD.60401@redhat.com> <51EB0BFF.3030106@oracle.com> <51ED57D8.90001@oracle.com> <51EE55AC.5000902@oracle.com> <51EE8E9A.6040806@redhat.com> <51EF1B43.9030200@oracle.com> <51F06789.1050403@oracle.com> <51F12B3E.4010603@oracle.com> <51F1BB22.6040405@oracle.com> <51F200F1.8010500@oracle.com> Message-ID: <51F5967A.6060705@oracle.com> Hello Alan and everyone else Thank you for review! Here's another version of webrev: http://cr.openjdk.java.net/~igerasim/8020669/5/webrev/ The comments are inline. On 26.07.2013 8:54, Alan Bateman wrote: > On 25/07/2013 16:56, Ivan Gerasimov wrote: >> >> Would you please help review an updated webrev? >> http://cr.openjdk.java.net/~igerasim/8020669/4/webrev/ > Sorry for the late response to your mails on this, I was on vacation. > > As you have found, in the original implementation we allowed for > resizing (truncation or extending) while the file was open but when it > was optimized to avoid an unnecessary resize of the array then we lost > the ability to deal with extending case (which is the essentially the > same as when the size is reported is less than the actual size). > > Reverting to the original implementation and doing a single read to > double check for EOF seems reasonable. > > My only real comment on the latest webrev (4) is that the computation > of the new capacity is not obvious to the reader (and since this is > the uncommon case then it doesn't need to be super-optimized). So I > think it would be a lot clearer if "newCapacity" were re-introduced > and then computed to a value that is a minimum of capacity+2, maximum > of capacity+BUFFER_SIZE (with OOME handling when newCapacity is > greater than MAX_BUFFER_SIZE). I think that would be clearer than > re-using the remaining count (rem). OK. I simplified it once again. Now the new capacity is set to one of {BUFFER_SIZE, 2 * old capacity, MAX_BUFFER_SIZE}, whatever is appropriate. I think that doubling the capacity is better than increasing it by BUFFER_SIZE, since this will allow to allocate needed buffer in less number of iterations. > > A minor comment is that the input stream is named "fis" but probably > should be "in" as it is not a FileInputStream (I see David Llyod > suggested using a FileInputStream but the path may be to a file in a > custom file system so it can't be a FileInputStream). > Renamed. > I see you've changed the javadoc from "Read" to "Reads" on several > methods. I don't know if you meant to include this in the webrev but > personally prefer "Read". > Files.java has mixed style. Most methods are documented with "Does" form, but some have "Do". I left "Reads" on the methods I modify for consistency, but reverted changes from other places as I'm not going to change all the occurrences in this webrev. > Thanks for moving the test to BytesAndLines as that is the test for > these methods. One suggestion is to move the new test for readAllBytes > into testReadAndWriteBytes and the new test for readAllLines into > testReadLines. It doesn't matter of course, it just keeps them together. > Sure, done. Sincerely yours, Ivan > -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130729/2080f267/attachment.html From Alan.Bateman at oracle.com Sun Jul 28 16:43:26 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 28 Jul 2013 16:43:26 -0700 Subject: RFR [8020669] java.nio.file.Files.readAllBytes() does not read any data when Files.size() is 0 In-Reply-To: <51F5967A.6060705@oracle.com> References: <51E97416.20506@oracle.com> <51E98043.6030102@oracle.com> <51E9853E.8020409@redhat.com> <51E985AD.60401@redhat.com> <51EB0BFF.3030106@oracle.com> <51ED57D8.90001@oracle.com> <51EE55AC.5000902@oracle.com> <51EE8E9A.6040806@redhat.com> <51EF1B43.9030200@oracle.com> <51F06789.1050403@oracle.com> <51F12B3E.4010603@oracle.com> <51F1BB22.6040405@oracle.com> <51F200F1.8010500@oracle.com> <51F5967A.6060705@oracle.com> Message-ID: <51F5AC9E.1010505@oracle.com> On 28/07/2013 15:08, Ivan Gerasimov wrote: > Hello Alan and everyone else > > Thank you for review! > > Here's another version of webrev: > http://cr.openjdk.java.net/~igerasim/8020669/5/webrev/ > Thanks for the updated the webrev, the expansion of the capacity is much nicer now. As regards whether to double or increase by BUFFER_SIZE then it's probably a coin toss. One could come up with cases where someone is using this method on a file that is extended by another entity. It would depend on the extend rate as to which approach would be more efficient (clearly such cases are going to be problematic as file locking should be used to coordinate the access). As regards the comment on "Reads" vs "Read", that was more just I've wasn't sure if you had intended to include this or not. So I'm happy to sponsor this for you (assuming we are done). -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130728/6b07a52a/attachment.html From ivan.gerasimov at oracle.com Mon Jul 29 01:48:54 2013 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 29 Jul 2013 12:48:54 +0400 Subject: RFR [8020669] java.nio.file.Files.readAllBytes() does not read any data when Files.size() is 0 In-Reply-To: <51F5AC9E.1010505@oracle.com> References: <51E97416.20506@oracle.com> <51E98043.6030102@oracle.com> <51E9853E.8020409@redhat.com> <51E985AD.60401@redhat.com> <51EB0BFF.3030106@oracle.com> <51ED57D8.90001@oracle.com> <51EE55AC.5000902@oracle.com> <51EE8E9A.6040806@redhat.com> <51EF1B43.9030200@oracle.com> <51F06789.1050403@oracle.com> <51F12B3E.4010603@oracle.com> <51F1BB22.6040405@oracle.com> <51F200F1.8010500@oracle.com> <51F5967A.6060705@oracle.com> <51F5AC9E.1010505@oracle.com> Message-ID: <51F62C76.7030401@oracle.com> Thank you, Alan. I really appreciate it. Here's the mercurial export: http://cr.openjdk.java.net/~igerasim/2commit/8020669-jdk8-readAllBytes.patch Sincerely yours, Ivan On 29.07.2013 3:43, Alan Bateman wrote: > On 28/07/2013 15:08, Ivan Gerasimov wrote: >> Hello Alan and everyone else >> >> Thank you for review! >> >> Here's another version of webrev: >> http://cr.openjdk.java.net/~igerasim/8020669/5/webrev/ >> > Thanks for the updated the webrev, the expansion of the capacity is > much nicer now. As regards whether to double or increase by > BUFFER_SIZE then it's probably a coin toss. One could come up with > cases where someone is using this method on a file that is extended by > another entity. It would depend on the extend rate as to which > approach would be more efficient (clearly such cases are going to be > problematic as file locking should be used to coordinate the access). > > As regards the comment on "Reads" vs "Read", that was more just I've > wasn't sure if you had intended to include this or not. > > So I'm happy to sponsor this for you (assuming we are done). > > -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130729/b73307be/attachment.html From chris.hegarty at oracle.com Mon Jul 29 03:27:45 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 29 Jul 2013 11:27:45 +0100 Subject: RFR [8020669] java.nio.file.Files.readAllBytes() does not read any data when Files.size() is 0 In-Reply-To: <51F5AC9E.1010505@oracle.com> References: <51E97416.20506@oracle.com> <51E98043.6030102@oracle.com> <51E9853E.8020409@redhat.com> <51E985AD.60401@redhat.com> <51EB0BFF.3030106@oracle.com> <51ED57D8.90001@oracle.com> <51EE55AC.5000902@oracle.com> <51EE8E9A.6040806@redhat.com> <51EF1B43.9030200@oracle.com> <51F06789.1050403@oracle.com> <51F12B3E.4010603@oracle.com> <51F1BB22.6040405@oracle.com> <51F200F1.8010500@oracle.com> <51F5967A.6060705@oracle.com> <51F5AC9E.1010505@oracle.com> Message-ID: <51F643A1.30404@oracle.com> On 29/07/2013 00:43, Alan Bateman wrote: > On 28/07/2013 15:08, Ivan Gerasimov wrote: >> Hello Alan and everyone else >> >> Thank you for review! >> >> Here's another version of webrev: >> http://cr.openjdk.java.net/~igerasim/8020669/5/webrev/ >> The change looks fine to me, but then again I was happy with previous version too ;-) -Chris. > Thanks for the updated the webrev, the expansion of the capacity is much > nicer now. As regards whether to double or increase by BUFFER_SIZE then > it's probably a coin toss. One could come up with cases where someone is > using this method on a file that is extended by another entity. It would > depend on the extend rate as to which approach would be more efficient > (clearly such cases are going to be problematic as file locking should > be used to coordinate the access). > > As regards the comment on "Reads" vs "Read", that was more just I've > wasn't sure if you had intended to include this or not. > > So I'm happy to sponsor this for you (assuming we are done). > > -Alan. From ivan.gerasimov at oracle.com Mon Jul 29 07:26:17 2013 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 29 Jul 2013 18:26:17 +0400 Subject: RFR [8020669] java.nio.file.Files.readAllBytes() does not read any data when Files.size() is 0 In-Reply-To: <51F643A1.30404@oracle.com> References: <51E97416.20506@oracle.com> <51E98043.6030102@oracle.com> <51E9853E.8020409@redhat.com> <51E985AD.60401@redhat.com> <51EB0BFF.3030106@oracle.com> <51ED57D8.90001@oracle.com> <51EE55AC.5000902@oracle.com> <51EE8E9A.6040806@redhat.com> <51EF1B43.9030200@oracle.com> <51F06789.1050403@oracle.com> <51F12B3E.4010603@oracle.com> <51F1BB22.6040405@oracle.com> <51F200F1.8010500@oracle.com> <51F5967A.6060705@oracle.com> <51F5AC9E.1010505@oracle.com> <51F643A1.30404@oracle.com> Message-ID: <51F67B89.807@oracle.com> On 29.07.2013 14:27, Chris Hegarty wrote: > On 29/07/2013 00:43, Alan Bateman wrote: >> On 28/07/2013 15:08, Ivan Gerasimov wrote: >>> Hello Alan and everyone else >>> >>> Thank you for review! >>> >>> Here's another version of webrev: >>> http://cr.openjdk.java.net/~igerasim/8020669/5/webrev/ >>> > > The change looks fine to me, but then again I was happy with previous > version too ;-) > Thank you, Chris! Ivan > -Chris. > > >> Thanks for the updated the webrev, the expansion of the capacity is much >> nicer now. As regards whether to double or increase by BUFFER_SIZE then >> it's probably a coin toss. One could come up with cases where someone is >> using this method on a file that is extended by another entity. It would >> depend on the extend rate as to which approach would be more efficient >> (clearly such cases are going to be problematic as file locking should >> be used to coordinate the access). >> >> As regards the comment on "Reads" vs "Read", that was more just I've >> wasn't sure if you had intended to include this or not. >> >> So I'm happy to sponsor this for you (assuming we are done). >> >> -Alan. > >