From xueming.shen at oracle.com Fri Apr 11 22:42:59 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 11 Apr 2014 15:42:59 -0700 Subject: RFR: JDK-8038500: (zipfs) Upgrade ZIP provider to be a supported provider Message-ID: <53486FF3.6020203@oracle.com> Hi, Please help review this changeset to upgrade the zip filesystem provider implementation from demo to a supported provider. Back in JDK7 we created a demo file system provider for zip/jar files. It is shipped in two forms, one as a binary under lib/ext that works "out of the box" to support zip/jar file access as a nio filesystem, the second is source form with a BSD license. We are now proposing to drop it as a demo and instead just "release" it as a filesystem provider that shipped with the JDK. issue: https://bugs.openjdk.java.net/browse/JDK-8038500 webrev: http://cr.openjdk.java.net/~sherman/8038500/webrev Thanks! -Sherman From mandy.chung at oracle.com Fri Apr 11 23:29:22 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 11 Apr 2014 16:29:22 -0700 Subject: RFR: JDK-8038500: (zipfs) Upgrade ZIP provider to be a supported provider In-Reply-To: <53486FF3.6020203@oracle.com> References: <53486FF3.6020203@oracle.com> Message-ID: <53487AD2.5060203@oracle.com> On 4/11/2014 3:42 PM, Xueming Shen wrote: > > webrev: http://cr.openjdk.java.net/~sherman/8038500/webrev It's good to see the source of the zip provider moved to the jdk repo. You have made some public classes to package-private which is good. I wonder whether a few remaining public classes can be made package-private (e.g. ZipFileAttributes, ZipInfo etc). Is JarFileSystemProvider used anywhere? ZipFileSystemProvider is the only provider listed in the service configuration. Do you mean to make Demo.java as a regression test? basic.sh - I wonder if you can get rid of this shell test. Do Basic, PathOps, ZipFSTester have any relationship? I was thinking if they can be made as individual java test and constructor the input path of zipfs.jar. With zipfs.jar part of the jdk build, it will be found under ${java.home}/lib/ext/zipfs.jar. When we move to modules, this jar file won't exist. It might be better for the tests to create a JAR file (one to share by all zipfs tests) to get ready for modules. Mandy From Alan.Bateman at oracle.com Sat Apr 12 07:20:06 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 12 Apr 2014 08:20:06 +0100 Subject: RFR: JDK-8038500: (zipfs) Upgrade ZIP provider to be a supported provider In-Reply-To: <53486FF3.6020203@oracle.com> References: <53486FF3.6020203@oracle.com> Message-ID: <5348E926.604@oracle.com> On 11/04/2014 23:42, Xueming Shen wrote: > Hi, > > Please help review this changeset to upgrade the zip filesystem > provider implementation from > demo to a supported provider. > > Back in JDK7 we created a demo file system provider for zip/jar files. > It is shipped in two forms, > one as a binary under lib/ext that works "out of the box" to support > zip/jar file access as a nio > filesystem, the second is source form with a BSD license. > > We are now proposing to drop it as a demo and instead just "release" > it as a filesystem provider > that shipped with the JDK. > > issue: https://bugs.openjdk.java.net/browse/JDK-8038500 > webrev: http://cr.openjdk.java.net/~sherman/8038500/webrev This looks good. Mandy makes a good point about that only com.sun.nio.zipfs.ZipFileSystemProvider needs to be public. That might avoid having to put com.sun.nio.zipfs on the restricted package list too. On basic.sh then this was originally needed in order to ensure that zipfs.jar was on the class path. Since it in the ext directory then it is not needed so Mandy's point about just moving the @test to the individual tests is good. -Alan. From xueming.shen at oracle.com Mon Apr 14 17:52:37 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 14 Apr 2014 10:52:37 -0700 Subject: RFR: JDK-8038500: (zipfs) Upgrade ZIP provider to be a supported provider In-Reply-To: <53487AD2.5060203@oracle.com> References: <53486FF3.6020203@oracle.com> <53487AD2.5060203@oracle.com> Message-ID: <534C2065.6010705@oracle.com> Thanks Mandy and Alan for the review. webrev has been updated accordingly to (1) Removed the basic.sh. The tests are now using test.jdk to access the zipfs.jar directly (2) Updated most of the class to package package, only ZipFileSystemProvider (the spi interface) and ZipInfo are public. The ZipInfo is a handy utility sometime I find it useful to simply do java com.sun.nio.zipfs.Info xyz.zip to display all cen and loc tables in details. I may take sometime to find a better home for it later. (3) I have not migrated the test target zipfs.jar to an "independent" test source (created during test) yet, will definitely later in a separate thread when we move to modules. (4) Yes, I mean to keep Demo there as an interactive regression test. http://cr.openjdk.java.net/~sherman/8038500/webrev/ -Sherman On 4/11/14 4:29 PM, Mandy Chung wrote: > On 4/11/2014 3:42 PM, Xueming Shen wrote: >> >> webrev: http://cr.openjdk.java.net/~sherman/8038500/webrev > > It's good to see the source of the zip provider moved to the jdk > repo. You have made some public classes to package-private which is > good. I wonder whether a few remaining public classes can be made > package-private (e.g. ZipFileAttributes, ZipInfo etc). Is > JarFileSystemProvider used anywhere? ZipFileSystemProvider is the > only provider listed in the service configuration. > > Do you mean to make Demo.java as a regression test? > > basic.sh - I wonder if you can get rid of this shell test. Do Basic, > PathOps, ZipFSTester have any relationship? I was thinking if they > can be made as individual java test and constructor the input path of > zipfs.jar. With zipfs.jar part of the jdk build, it will be found > under ${java.home}/lib/ext/zipfs.jar. When we move to modules, this > jar file won't exist. It might be better for the tests to create a > JAR file (one to share by all zipfs tests) to get ready for modules. > > Mandy From Alan.Bateman at oracle.com Mon Apr 14 18:56:46 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 14 Apr 2014 19:56:46 +0100 Subject: RFR: JDK-8038500: (zipfs) Upgrade ZIP provider to be a supported provider In-Reply-To: <534C2065.6010705@oracle.com> References: <53486FF3.6020203@oracle.com> <53487AD2.5060203@oracle.com> <534C2065.6010705@oracle.com> Message-ID: <534C2F6E.5090609@oracle.com> On 14/04/2014 18:52, Xueming Shen wrote: > Thanks Mandy and Alan for the review. > > webrev has been updated accordingly to > > (1) Removed the basic.sh. The tests are now using test.jdk to access > the zipfs.jar directly > (2) Updated most of the class to package package, only > ZipFileSystemProvider (the spi interface) > and ZipInfo are public. > The ZipInfo is a handy utility sometime I find it useful to > simply do java com.sun.nio.zipfs.Info xyz.zip > to display all cen and loc tables in details. I may take sometime > to find a better home for it later. > (3) I have not migrated the test target zipfs.jar to an "independent" > test source (created during test) yet, > will definitely later in a separate thread when we move to modules. > (4) Yes, I mean to keep Demo there as an interactive regression test. > > http://cr.openjdk.java.net/~sherman/8038500/webrev/ > > -Sherman Iris asked me off-list about the name space which makes me wonder if we should use the opportunity to move this to jdk.. As it's a service provider then nothing should be accessing these classes directly. The only thing that comes to mind is ZipFileAttributeView/ZipFileAttributes where the API provides a type-safe means to access attributes. In the webrev then these are being changed to package-private so I think would break anyone that might be using them anyway. The removal of basic.sh and the test tags added the tests looks okay. -Alan. From xueming.shen at oracle.com Mon Apr 14 19:15:16 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 14 Apr 2014 12:15:16 -0700 Subject: RFR: JDK-8038500: (zipfs) Upgrade ZIP provider to be a supported provider In-Reply-To: <534C2F6E.5090609@oracle.com> References: <53486FF3.6020203@oracle.com> <53487AD2.5060203@oracle.com> <534C2065.6010705@oracle.com> <534C2F6E.5090609@oracle.com> Message-ID: <534C33C4.7030400@oracle.com> On 4/14/14 11:56 AM, Alan Bateman wrote: > On 14/04/2014 18:52, Xueming Shen wrote: >> Thanks Mandy and Alan for the review. >> >> webrev has been updated accordingly to >> >> (1) Removed the basic.sh. The tests are now using test.jdk to access >> the zipfs.jar directly >> (2) Updated most of the class to package package, only >> ZipFileSystemProvider (the spi interface) >> and ZipInfo are public. >> The ZipInfo is a handy utility sometime I find it useful to >> simply do java com.sun.nio.zipfs.Info xyz.zip >> to display all cen and loc tables in details. I may take >> sometime to find a better home for it later. >> (3) I have not migrated the test target zipfs.jar to an "independent" >> test source (created during test) yet, >> will definitely later in a separate thread when we move to modules. >> (4) Yes, I mean to keep Demo there as an interactive regression test. >> >> http://cr.openjdk.java.net/~sherman/8038500/webrev/ >> >> -Sherman > Iris asked me off-list about the name space which makes me wonder if > we should use the opportunity to move this to jdk.. As it's > a service provider then nothing should be accessing these classes > directly. The only thing that comes to mind is > ZipFileAttributeView/ZipFileAttributes where the API provides a > type-safe means to access attributes. In the webrev then these are > being changed to package-private so I think would break anyone that > might be using them anyway. > go with jdk.nio.zipfs? I'm fine with that if this is the new directory to go. though it appears we have 1000+ classes at com.sun... -sherman From erik.joelsson at oracle.com Tue Apr 15 07:48:37 2014 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 15 Apr 2014 09:48:37 +0200 Subject: RFR: JDK-8038500: (zipfs) Upgrade ZIP provider to be a supported provider In-Reply-To: <534C2065.6010705@oracle.com> References: <53486FF3.6020203@oracle.com> <53487AD2.5060203@oracle.com> <534C2065.6010705@oracle.com> Message-ID: <534CE455.1090809@oracle.com> Hello Sherman, The build changes look good to me. /Erik On 2014-04-14 19:52, Xueming Shen wrote: > Thanks Mandy and Alan for the review. > > webrev has been updated accordingly to > > (1) Removed the basic.sh. The tests are now using test.jdk to access > the zipfs.jar directly > (2) Updated most of the class to package package, only > ZipFileSystemProvider (the spi interface) > and ZipInfo are public. > The ZipInfo is a handy utility sometime I find it useful to > simply do java com.sun.nio.zipfs.Info xyz.zip > to display all cen and loc tables in details. I may take sometime > to find a better home for it later. > (3) I have not migrated the test target zipfs.jar to an "independent" > test source (created during test) yet, > will definitely later in a separate thread when we move to modules. > (4) Yes, I mean to keep Demo there as an interactive regression test. > > http://cr.openjdk.java.net/~sherman/8038500/webrev/ > > -Sherman > > On 4/11/14 4:29 PM, Mandy Chung wrote: >> On 4/11/2014 3:42 PM, Xueming Shen wrote: >>> >>> webrev: http://cr.openjdk.java.net/~sherman/8038500/webrev >> >> It's good to see the source of the zip provider moved to the jdk >> repo. You have made some public classes to package-private which is >> good. I wonder whether a few remaining public classes can be made >> package-private (e.g. ZipFileAttributes, ZipInfo etc). Is >> JarFileSystemProvider used anywhere? ZipFileSystemProvider is the >> only provider listed in the service configuration. >> >> Do you mean to make Demo.java as a regression test? >> >> basic.sh - I wonder if you can get rid of this shell test. Do Basic, >> PathOps, ZipFSTester have any relationship? I was thinking if they >> can be made as individual java test and constructor the input path of >> zipfs.jar. With zipfs.jar part of the jdk build, it will be found >> under ${java.home}/lib/ext/zipfs.jar. When we move to modules, this >> jar file won't exist. It might be better for the tests to create a >> JAR file (one to share by all zipfs tests) to get ready for modules. >> >> Mandy > From Alan.Bateman at oracle.com Tue Apr 15 08:46:13 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 15 Apr 2014 09:46:13 +0100 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: References: Message-ID: <534CF1D5.70605@oracle.com> On 15/04/2014 09:05, Richard Warburton wrote: > : > > The only issue that I'm aware of that is related to this kind of change is > the requirement to recompile all the classes in the hierarchy when making a > change [0]. If you don't do this its possible for an infinite recursion and > eventual stackoverflow to occur. Now as far as I can tell the > entire hierarchy of Buffer classes are all either package scoped classes, > or they are public classes with package scoped constructors. So I'm not > aware of a way to sub class them without the code being part of the JDK. > This would mean that the subclasses all need to be recompiled, satisfying > the safety criteria. > This is a long standing bug to look at this issue [1]. CharBuffer.subSequence was one case that was retrofitted in 7. As you note, buffers are not designed to be extended outside of the java.nio package so this should make a number of options possible. -Alan [1] https://bugs.openjdk.java.net/browse/JDK-4774077 From Alan.Bateman at oracle.com Tue Apr 15 09:00:30 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 15 Apr 2014 10:00:30 +0100 Subject: RFR: JDK-8038500: (zipfs) Upgrade ZIP provider to be a supported provider In-Reply-To: <534CAC1C.8000405@oracle.com> References: <53486FF3.6020203@oracle.com> <53487AD2.5060203@oracle.com> <534C2065.6010705@oracle.com> <534C2F6E.5090609@oracle.com> <534C33C4.7030400@oracle.com> <534CAC1C.8000405@oracle.com> Message-ID: <534CF52E.5040403@oracle.com> On 15/04/2014 04:48, Xueming Shen wrote: > Hi, > > webrev has been updated to use the name space jdk.nio.zipfs. > > http://cr.openjdk.java.net/~sherman/8038500/webrev/ > Thanks, I think this makes sense. For further down the road then consideration could be given to make ZipFileAttributes/ZipFileAttributeView public and supported so as to provide type safe access to zip specific attributes. JarFileSystemProvider will also need discussion at some point to decide whether it should be added to the service configuration file so that JAR URLs are supported. -Alan. ** -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavel.rappo at oracle.com Tue Apr 15 12:52:24 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 15 Apr 2014 13:52:24 +0100 Subject: RFR JDK-8040262: (fs) Misc. typos in comments and implementation Message-ID: <91C49A8D-6EB9-4610-B485-792AB9C0A5A4@oracle.com> Hi everyone, could you please review my change for JDK-8040262: http://cr.openjdk.java.net/~alanb/8040262/webrev/ That's just a fix for a bunch of typos (and deleting 1 unused import statement) I found in java.nio.file and sun.nio.fs during my work on a different bug. Thanks -Pavel From Alan.Bateman at oracle.com Tue Apr 15 12:57:04 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 15 Apr 2014 13:57:04 +0100 Subject: RFR JDK-8040262: (fs) Misc. typos in comments and implementation In-Reply-To: <91C49A8D-6EB9-4610-B485-792AB9C0A5A4@oracle.com> References: <91C49A8D-6EB9-4610-B485-792AB9C0A5A4@oracle.com> Message-ID: <534D2CA0.4010204@oracle.com> On 15/04/2014 13:52, Pavel Rappo wrote: > Hi everyone, > > could you please review my change for JDK-8040262: http://cr.openjdk.java.net/~alanb/8040262/webrev/ > > That's just a fix for a bunch of typos (and deleting 1 unused import statement) I found in java.nio.file and sun.nio.fs during my work on a different bug. > > Thanks > -Pavel Thanks for doing this and good to see that only one of them is in the javadoc. I will sponsor this for you. -Alan. From chris.hegarty at oracle.com Tue Apr 15 13:01:00 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 15 Apr 2014 14:01:00 +0100 Subject: RFR JDK-8040262: (fs) Misc. typos in comments and implementation In-Reply-To: <91C49A8D-6EB9-4610-B485-792AB9C0A5A4@oracle.com> References: <91C49A8D-6EB9-4610-B485-792AB9C0A5A4@oracle.com> Message-ID: Very nice Pavel. Looks good to me. -Chris. On 15 Apr 2014, at 13:52, Pavel Rappo wrote: > Hi everyone, > > could you please review my change for JDK-8040262: http://cr.openjdk.java.net/~alanb/8040262/webrev/ > > That's just a fix for a bunch of typos (and deleting 1 unused import statement) I found in java.nio.file and sun.nio.fs during my work on a different bug. > > Thanks > -Pavel From stuart.marks at oracle.com Thu Apr 17 17:49:33 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 17 Apr 2014 10:49:33 -0700 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: <534CF1D5.70605@oracle.com> References: <534CF1D5.70605@oracle.com> Message-ID: <5350142D.8040706@oracle.com> On 4/15/14 1:46 AM, Alan Bateman wrote: > On 15/04/2014 09:05, Richard Warburton wrote: >> The only issue that I'm aware of that is related to this kind of change is >> the requirement to recompile all the classes in the hierarchy when making a >> change [0]. If you don't do this its possible for an infinite recursion and >> eventual stackoverflow to occur. Now as far as I can tell the >> entire hierarchy of Buffer classes are all either package scoped classes, >> or they are public classes with package scoped constructors. So I'm not >> aware of a way to sub class them without the code being part of the JDK. >> This would mean that the subclasses all need to be recompiled, satisfying >> the safety criteria. >> > This is a long standing bug to look at this issue [1]. CharBuffer.subSequence > was one case that was retrofitted in 7. As you note, buffers are not designed to > be extended outside of the java.nio package so this should make a number of > options possible. > > [1] https://bugs.openjdk.java.net/browse/JDK-4774077 Hi Richard, It looks to me like there are no obvious problems with changing the buffer subclasses to use covariant overrides. As Alan mentioned, one API was modified this way in 7, by Martin Buchholz. The discussion threads for this change are [2] and [3]. It looks like Martin considered additional covariant overrides, but ultimately backed off from this for a variety of incidental reasons; see [4]. There is a certain amount of work here above and beyond just changing the return types, but there don't appear to be any fundamental issues. Ulf posted several links, but these seem mostly related to language change proposals related to implicit method invocation chaining. I don't think these proposals are active at the moment, so I'd suggest that you pursue a library-only change. s'marks [2] http://mail.openjdk.java.net/pipermail/core-libs-dev/2008-July/000596.html [3] http://mail.openjdk.java.net/pipermail/core-libs-dev/2008-August/000599.html [4] http://mail.openjdk.java.net/pipermail/core-libs-dev/2008-August/000612.html From weijun.wang at oracle.com Fri Apr 18 15:21:47 2014 From: weijun.wang at oracle.com (Wang Weijun) Date: Fri, 18 Apr 2014 23:21:47 +0800 Subject: the best way to talk to multiple UDP/TCP servers? Message-ID: I'm writing a client which talks to 3 servers, and would like to write a method that sends request to server 1 at time 0:01 sends request to server 2 at time 0:02 sends request to server 3 at time 0:03 sends request to server 1 at time 0:11 sends request to server 2 at time 0:12 sends request to server 3 at time 0:13 waits until 0:20 During this whole period, whenever a response from a server is received, the method returns and no more request is sent. The protocol can be UDP or TCP. What is the best way I can do this with NIO? Thanks Max From Ulf.Zibis at CoSoCo.de Sat Apr 19 16:00:14 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Sat, 19 Apr 2014 18:00:14 +0200 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> Message-ID: <53529D8E.6020606@CoSoCo.de> Am 19.04.2014 10:09, schrieb Richard Warburton: > Thanks for the feedback guys - its most appreciated and I'll take a look > through the detailed linked you provided. > > Ulf posted several links, but these seem mostly related to language change Not all of them, but yes, mostly. >> proposals related to implicit method invocation chaining. I don't think >> these proposals are active at the moment, so I'd suggest that you pursue a >> library-only change. > > I had no intention in my original email to broaden the scope of this change > to include changing the type system itself! I was only really thinking > about a library level change, though I appreciate the links Ulf, thanks. My proposed language change would ... - include the benefit of covariant overrides - solve situations, where covariant overrides don't help - need less byte code - have less backport issues - avoid massive API reengineering - cover methods which don't return this today - offer interesting extensions e.g. "leading dot" syntax Thanks for reading my proposal. -Ulf From richard.warburton at gmail.com Sun Apr 20 18:56:52 2014 From: richard.warburton at gmail.com (Richard Warburton) Date: Sun, 20 Apr 2014 20:56:52 +0200 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: <5350142D.8040706@oracle.com> References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> Message-ID: Hi, It looks to me like there are no obvious problems with changing the buffer > subclasses to use covariant overrides. As Alan mentioned, one API was > modified this way in 7, by Martin Buchholz. The discussion threads for this > change are [2] and [3]. It looks like Martin considered additional > covariant overrides, but ultimately backed off from this for a variety of > incidental reasons; see [4]. There is a certain amount of work here above > and beyond just changing the return types, but there don't appear to be any > fundamental issues. So I've put a patch on cr at http://cr.openjdk.java.net/~rwarburton/buffer-overrides-0/. It also touches a couple of other classes which have unnecessary casts in. Aside from being a good idea, fixing the casts is also necessary due to -Werror. Let me know if I've missed anything or if there are any outstanding issues. I think the bugs that this resolves are: https://bugs.openjdk.java.net/browse/JDK-4774077 https://bugs.openjdk.java.net/browse/JDK-6451131 regards, Richard Warburton http://insightfullogic.com @RichardWarburto -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ulf.Zibis at CoSoCo.de Sun Apr 20 21:44:24 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Sun, 20 Apr 2014 23:44:24 +0200 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> Message-ID: <53543FB8.7040508@CoSoCo.de> Hi again, one additional reason on top of my list from yesterday why I would prefer the language change: - Removing "final" is a performance decrease. -Ulf Am 20.04.2014 20:56, schrieb Richard Warburton: > Hi, > > It looks to me like there are no obvious problems with changing the buffer >> subclasses to use covariant overrides. As Alan mentioned, one API was >> modified this way in 7, by Martin Buchholz. The discussion threads for this >> change are [2] and [3]. It looks like Martin considered additional >> covariant overrides, but ultimately backed off from this for a variety of >> incidental reasons; see [4]. There is a certain amount of work here above >> and beyond just changing the return types, but there don't appear to be any >> fundamental issues. > > So I've put a patch on cr at > http://cr.openjdk.java.net/~rwarburton/buffer-overrides-0/. It also touches > a couple of other classes which have unnecessary casts in. Aside from being > a good idea, fixing the casts is also necessary due to -Werror. Let me > know if I've missed anything or if there are any outstanding issues. > > I think the bugs that this resolves are: > > https://bugs.openjdk.java.net/browse/JDK-4774077 > https://bugs.openjdk.java.net/browse/JDK-6451131 > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > From Ulf.Zibis at CoSoCo.de Mon Apr 21 12:07:36 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Mon, 21 Apr 2014 14:07:36 +0200 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: <5354FC9F.1020409@gmail.com> References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> <5354FC9F.1020409@gmail.com> Message-ID: <53550A08.3090005@CoSoCo.de> Hi Peter, Am 21.04.2014 13:10, schrieb Peter Levart: > Since the public part of Buffer type hierarchy is (mostly) just two-level, an alternative would be > to generify java.nio.Buffer: Hey wow, some time ago I also tried to find a compiling and working code in that sense, but I stranded. But what for an effort, just to avoid retyping of same object in java code ??? So to me, this is clearly a language matter, not to solve by tricky footprint and/or performance lasting code on APIs: People want a _smart syntax_ for consecutive method invocations on the same object, that's all! -Ulf From richard.warburton at gmail.com Mon Apr 21 21:51:40 2014 From: richard.warburton at gmail.com (Richard Warburton) Date: Mon, 21 Apr 2014 23:51:40 +0200 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: <5354FC9F.1020409@gmail.com> References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> <5354FC9F.1020409@gmail.com> Message-ID: Hi Peter, > I'm suggesting this alternative, because Buffer methods can stay final in > this case. This is more JIT-friendly. And, if I'm not mistaken, client code > compiled using JDK8 onto which this API change was backported, would still > run with JDK8 (or JDK7 when compiled with -target 1.7) onto which the API > change was not back-ported. > Thanks for suggesting this alternative. I think there are a few downsides to this approach as well though. 1. Anyone with code referring to 'ByteBuffer' now gets rawtype generics errors. 2. Anyone with -Werror (like openjdk!) now fails to compile. 3. This is a more complex change than the one I was proposing and smaller, simpler, changes seem to be less risky. 4. Developers do genuinely get confused by generics. Not a reason not to use them ever but a good reason not to introduce them if the issue can be solved by an alternative approach. Happy to be corrected if I've misunderstood anything ;) regards, Richard Warburton http://insightfullogic.com @RichardWarburto -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.lloyd at redhat.com Mon Apr 21 22:02:26 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Mon, 21 Apr 2014 17:02:26 -0500 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> <5354FC9F.1020409@gmail.com> Message-ID: <53559572.2080004@redhat.com> On 04/21/2014 04:51 PM, Richard Warburton wrote: > Hi Peter, > > I'm suggesting this alternative, because Buffer methods can stay > final in this case. This is more JIT-friendly. And, if I'm not > mistaken, client code compiled using JDK8 onto which this API change > was backported, would still run with JDK8 (or JDK7 when compiled > with -target 1.7) onto which the API change was not back-ported. > > > Thanks for suggesting this alternative. I think there are a few > downsides to this approach as well though. > > 1. Anyone with code referring to 'ByteBuffer' now gets rawtype generics > errors. > 2. Anyone with -Werror (like openjdk!) now fails to compile. > 3. This is a more complex change than the one I was proposing and > smaller, simpler, changes seem to be less risky. > 4. Developers do genuinely get confused by generics. Not a reason not to > use them ever but a good reason not to introduce them if the issue can > be solved by an alternative approach. > > Happy to be corrected if I've misunderstood anything ;) Um, do we *know* that there is a performance cost to covariantly overriding these methods? There would seem to be enough information to keep them monomorphic, if the optimizer is smart enough to inline the bridge methods and the delegating override method. The overridden methods in addition can be final, meaning that in the 99% case that you're invoking directly on the desired buffer type, it should be just as optimizable for the same reason that the original methods were optimizable. The only potentially "slow" invocation path is if you call the method on a Buffer reference, and even then it seems like there's enough information to avoid slowness - and if not, then that seems like a HotSpot problem that is very solvable ("if all overrides of this method call super.xxx(), inline & eliminate them"). Is there a real, solid hypothesis that would demonstrate that any of this is not true? -- - DML From zhong.j.yu at gmail.com Tue Apr 22 02:30:47 2014 From: zhong.j.yu at gmail.com (Zhong Yu) Date: Mon, 21 Apr 2014 21:30:47 -0500 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: <53550A08.3090005@CoSoCo.de> References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> <5354FC9F.1020409@gmail.com> <53550A08.3090005@CoSoCo.de> Message-ID: On Mon, Apr 21, 2014 at 7:07 AM, Ulf Zibis wrote: > So to me, this is clearly a language matter, not to solve by tricky > footprint and/or performance lasting code on APIs: > People want a _smart syntax_ for consecutive method invocations on the same > object, that's all! Amen to that! However I'm a little concerned with the mental effort a reader needs to figure out the implicit method receiver. I'd like to see more explicitness in establishing the implicit method receiver, something like Pascal's with statement. expr "." { // expr is the implicit receiver for this block foo(); // interpreted as expr.foo() } so that we could write CharBuffer.allocate(26).{position(2);put("C");position(25);put("Z");} string.{ mySub=subString(indexOf("C"), indexOf("Q")); } new HashMap<>().{ put(n1,v1); put(n2,v2); } new Button().{ color(red); text("hello"); onClick(event->color(blue)); } Zhong Yu From richard.warburton at gmail.com Tue Apr 22 08:55:39 2014 From: richard.warburton at gmail.com (Richard Warburton) Date: Tue, 22 Apr 2014 10:55:39 +0200 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: <535614B1.1010400@gmail.com> References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> <5354FC9F.1020409@gmail.com> <53550A08.3090005@CoSoCo.de> <535614B1.1010400@gmail.com> Message-ID: Hi, > It certainly spawns all kinds of questions that were on the agenda for > lambda expressions, like: > - what's the meaning of "this" inside the .{ ... } block? > - what's the meaning of "break", "continue", "next", "return", .... (I > guess it should be the same as with the ordinary lexical block: { ... }, > although it's rather unusual to be able to for example "break" out of a > loop inside the evaluation of an expression) > I appreciate you guys are excited about the idea of language changes, but can you please discuss them in another thread and not hijack this library discussion thread. regards, Richard Warburton http://insightfullogic.com @RichardWarburto -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.lloyd at redhat.com Tue Apr 22 12:48:15 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 22 Apr 2014 07:48:15 -0500 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: <535633C2.5090501@gmail.com> References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> <5354FC9F.1020409@gmail.com> <53559572.2080004@redhat.com> <535633C2.5090501@gmail.com> Message-ID: <5356650F.5020909@redhat.com> On 04/22/2014 04:17 AM, Peter Levart wrote: > On 04/22/2014 12:02 AM, David M. Lloyd wrote: >> Um, do we *know* that there is a performance cost to covariantly >> overriding these methods? There would seem to be enough information >> to keep them monomorphic, if the optimizer is smart enough to inline >> the bridge methods and the delegating override method. The overridden >> methods in addition can be final, meaning that in the 99% case that >> you're invoking directly on the desired buffer type, it should be just >> as optimizable for the same reason that the original methods were >> optimizable. The only potentially "slow" invocation path is if you >> call the method on a Buffer reference, and even then it seems like >> there's enough information to avoid slowness - and if not, then that >> seems like a HotSpot problem that is very solvable ("if all overrides >> of this method call super.xxx(), inline & eliminate them"). > > It's more complicated than that. Maybe we need an expert for hotspot JIT > to answer this question, but as the code is written in the Rickard's > webrev, then the reasoning behind the JIT to keep the monomorphic > dispatch would have to be more involving. Richard is doing the following > (in ByteBuffer): > > @Override > public ByteBuffer position(int newPosition) { > super.position(newPosition); > return this; > } > > javac compiles each of the covariant overrides as two methods - one that > actually "overrides" the virtual method in superclass (has the same > signature) and calls the covariant-returning method with a virtual > dispatch. So ByteBuffer.position(int) is compiled as: > > public ByteBuffer position(int newPosition) { > super.position(newPosition); > return this; > } > > public Buffer position(int newPosition) { > // this is an invokevirtual for position:(I)Ljava/nio/ByteBuffer; > return (ByteBuffer) position( (int) newPosition); > } If the methods were final, AFAICT it'd be more like this: public final ByteBuffer position(int newPosition) { // iirc super upcall is already bytecoded as invokespecial invokespecial (Buffer)Buffer.position(newPosition); return this; } public synthetic final Buffer position(int newPosition) { return effectively-invokespecial (ByteBuffer)ByteBuffer.position(newPosition); } Since there would only be one possible target for the invokevirtual, my understanding is that the JIT will convert that into an invokespecial, letting the whole works be optimized at worst and inlined at best. -- - DML From paul.sandoz at oracle.com Tue Apr 22 13:24:10 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 22 Apr 2014 15:24:10 +0200 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: <5356650F.5020909@redhat.com> References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> <5354FC9F.1020409@gmail.com> <53559572.2080004@redhat.com> <535633C2.5090501@gmail.com> <5356650F.5020909@redhat.com> Message-ID: <25EE2DD0-5B32-45D7-9A14-C3E9E501F5AB@oracle.com> On Apr 22, 2014, at 2:48 PM, David M. Lloyd wrote: > > If the methods were final, AFAICT it'd be more like this: > > public final ByteBuffer position(int newPosition) { > // iirc super upcall is already bytecoded as invokespecial > invokespecial (Buffer)Buffer.position(newPosition); > return this; > } > > public synthetic final Buffer position(int newPosition) { > return effectively-invokespecial (ByteBuffer)ByteBuffer.position(newPosition); > } > > Since there would only be one possible target for the invokevirtual, my understanding is that the JIT will convert that into an invokespecial, letting the whole works be optimized at worst and inlined at best. > Here is a simple example with the Java 8 byte code public class TestBridge { static class A { public A pos(int i) { return this; } } static class B extends A { public final B pos(int i) { super.pos(i); return this; } } } public final TestBridge$B pos(int); flags: ACC_PUBLIC, ACC_FINAL Code: stack=2, locals=2, args_size=2 0: aload_0 1: iload_1 2: invokespecial #2 // Method TestBridge$A.pos:(I)LTestBridge$A; 5: pop 6: aload_0 7: areturn public TestBridge$A pos(int); flags: ACC_PUBLIC, ACC_BRIDGE, ACC_SYNTHETIC Code: stack=2, locals=2, args_size=2 0: aload_0 1: iload_1 2: invokevirtual #3 // Method pos:(I)LTestBridge$B; 5: areturn I dunno why the synthetic bridge method is not marked as final. -- It's too hard for me to speculate what any performance impact might be. My suspicion is these methods are probably not used that often in hot loops so even if performance results are marginally worse the API improvement may trump that. Paul. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail URL: From david.lloyd at redhat.com Tue Apr 22 13:29:58 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 22 Apr 2014 08:29:58 -0500 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: <25EE2DD0-5B32-45D7-9A14-C3E9E501F5AB@oracle.com> References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> <5354FC9F.1020409@gmail.com> <53559572.2080004@redhat.com> <535633C2.5090501@gmail.com> <5356650F.5020909@redhat.com> <25EE2DD0-5B32-45D7-9A14-C3E9E501F5AB@oracle.com> Message-ID: <53566ED6.3030309@redhat.com> On 04/22/2014 08:24 AM, Paul Sandoz wrote: > On Apr 22, 2014, at 2:48 PM, David M. Lloyd wrote: >> >> If the methods were final, AFAICT it'd be more like this: >> >> public final ByteBuffer position(int newPosition) { >> // iirc super upcall is already bytecoded as invokespecial >> invokespecial (Buffer)Buffer.position(newPosition); >> return this; >> } >> >> public synthetic final Buffer position(int newPosition) { >> return effectively-invokespecial (ByteBuffer)ByteBuffer.position(newPosition); >> } >> >> Since there would only be one possible target for the invokevirtual, my understanding is that the JIT will convert that into an invokespecial, letting the whole works be optimized at worst and inlined at best. >> > > Here is a simple example with the Java 8 byte code > > public class TestBridge { > static class A { > public A pos(int i) { return this; } > } > > static class B extends A { > public final B pos(int i) { super.pos(i); return this; } > } > } > > public final TestBridge$B pos(int); > flags: ACC_PUBLIC, ACC_FINAL > Code: > stack=2, locals=2, args_size=2 > 0: aload_0 > 1: iload_1 > 2: invokespecial #2 // Method TestBridge$A.pos:(I)LTestBridge$A; > 5: pop > 6: aload_0 > 7: areturn > > public TestBridge$A pos(int); > flags: ACC_PUBLIC, ACC_BRIDGE, ACC_SYNTHETIC > Code: > stack=2, locals=2, args_size=2 > 0: aload_0 > 1: iload_1 > 2: invokevirtual #3 // Method pos:(I)LTestBridge$B; > 5: areturn > > I dunno why the synthetic bridge method is not marked as final. Hmm that seems like a bug to me, I'll take it over to compiler-dev and see what they say... > It's too hard for me to speculate what any performance impact might be. My suspicion is these methods are probably not used that often in hot loops so even if performance results are marginally worse the API improvement may trump that. Yeah I am not sure we can say with certainty either way without testing. But if there is a performance cost, IMO it should be done anyway, and the performance cost should be fixed in javac and/or the JIT. -- - DML From david.lloyd at redhat.com Tue Apr 22 15:25:16 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 22 Apr 2014 10:25:16 -0500 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: <53568254.3010106@gmail.com> References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> <5354FC9F.1020409@gmail.com> <53559572.2080004@redhat.com> <535633C2.5090501@gmail.com> <5356650F.5020909@redhat.com> <53568254.3010106@gmail.com> Message-ID: <535689DC.6060907@redhat.com> On 04/22/2014 09:53 AM, Peter Levart wrote: > On 04/22/2014 02:48 PM, David M. Lloyd wrote: >> On 04/22/2014 04:17 AM, Peter Levart wrote: >>> On 04/22/2014 12:02 AM, David M. Lloyd wrote: >>>> Um, do we *know* that there is a performance cost to covariantly >>>> overriding these methods? There would seem to be enough information >>>> to keep them monomorphic, if the optimizer is smart enough to inline >>>> the bridge methods and the delegating override method. The overridden >>>> methods in addition can be final, meaning that in the 99% case that >>>> you're invoking directly on the desired buffer type, it should be just >>>> as optimizable for the same reason that the original methods were >>>> optimizable. The only potentially "slow" invocation path is if you >>>> call the method on a Buffer reference, and even then it seems like >>>> there's enough information to avoid slowness - and if not, then that >>>> seems like a HotSpot problem that is very solvable ("if all overrides >>>> of this method call super.xxx(), inline & eliminate them"). >>> >>> It's more complicated than that. Maybe we need an expert for hotspot JIT >>> to answer this question, but as the code is written in the Rickard's >>> webrev, then the reasoning behind the JIT to keep the monomorphic >>> dispatch would have to be more involving. Richard is doing the following >>> (in ByteBuffer): >>> >>> @Override >>> public ByteBuffer position(int newPosition) { >>> super.position(newPosition); >>> return this; >>> } >>> >>> javac compiles each of the covariant overrides as two methods - one that >>> actually "overrides" the virtual method in superclass (has the same >>> signature) and calls the covariant-returning method with a virtual >>> dispatch. So ByteBuffer.position(int) is compiled as: >>> >>> public ByteBuffer position(int newPosition) { >>> super.position(newPosition); >>> return this; >>> } >>> >>> public Buffer position(int newPosition) { >>> // this is an invokevirtual for position:(I)Ljava/nio/ByteBuffer; >>> return (ByteBuffer) position( (int) newPosition); >>> } >> >> If the methods were final, AFAICT it'd be more like this: >> >> public final ByteBuffer position(int newPosition) { >> // iirc super upcall is already bytecoded as invokespecial >> invokespecial (Buffer)Buffer.position(newPosition); >> return this; >> } >> >> public synthetic final Buffer position(int newPosition) { >> return effectively-invokespecial >> (ByteBuffer)ByteBuffer.position(newPosition); >> } >> >> Since there would only be one possible target for the invokevirtual, >> my understanding is that the JIT will convert that into an >> invokespecial, letting the whole works be optimized at worst and >> inlined at best. >> > > There are multiple possible targets for invokevirtual > position:(I)Ljava/nio/Buffer; - all the methods that override it in all > subclasses loaded. It doesn't matter if they are final or not (only if > they are effectively final or not). The non-finality of a method has a > performance impact only if the method *is* overridden in any of the > loaded subclasses, otherwise it is effectively final and treated as such > by JIT (at least that's how I understand it - any hotspot JIT expert > please debunk my claims). That is only true if you are calling the method on the base class, which is normally seldom done (though it will be nearly always in legacy code, but see below). > That might also be the answer to why the synthetic method need not be > marked as final if the covariant method is. The synthetic method can > never be overridden in a sub-class (at least not by javac) - only the > covariant method can be. Doesn't sound quite right to me, but I'll defer to any experts who might wish to discuss it on compiler-dev. > But as Paul noted, the methods on Buffer are probably not used in hot > loops alone, since they are just for reading/adjusting > position/limit/mark. The hot loops probably also contain methods for > reading/writing the buffer and those are only defined on particular > sub-types of java.nio.Buffer, so it can reasonably be expected that the > static (guaranteed) type of target upon which methods are called in hot > loops is a particular subtype of java.nio.Buffer and JIT only has one > method to choose from in this case. Yeah new code will call the covariant final method directly in almost all cases for sure, having been compiled against it. The tricky case is old code - even if the JIT can figure out that it is really a subclass being invoked upon, it'd still be forced to target the non-covariant synthetic method due to the specific method reference in the bytecode. Hopefully at this point though, as you say, the lack of overriding classes will be enough to optimize the dispatch. -- - DML From zhong.j.yu at gmail.com Thu Apr 24 06:17:35 2014 From: zhong.j.yu at gmail.com (Zhong Yu) Date: Thu, 24 Apr 2014 01:17:35 -0500 Subject: Memory leak in PendingFuture? Message-ID: In sun.nio.ch.PendingFuture, there is a static cached Throwable private static final CancellationException CANCELLED = new CancellationException(); Apparently, its Throwable.backtrace field contains an array of all Classes on the call stack. This will prevent the user classes from being garbage collected. Zhong Yu From zhong.j.yu at gmail.com Thu Apr 24 06:32:46 2014 From: zhong.j.yu at gmail.com (Zhong Yu) Date: Thu, 24 Apr 2014 01:32:46 -0500 Subject: Memory leak in AsynchronousFileChannel? Message-ID: AsynchronousFileChannel.open() with default executor spawns some threads that never exit: java.lang.Thread.State: RUNNABLE at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:744) (win7, jdk8) These threads reference all class loaders on the calling stack of open(), through Thread.inheritedAccessControlContext.context[*].classloader, which prevents user class loaders from being garbage collected. On the other hand, WatchService threads do not have this problem. I guess it's due to doPrivileged(): sun.nio.fs.AbstractPoller public void start() { final Runnable thisRunnable = this; AccessController.doPrivileged(new PrivilegedAction() { @Override public Object run() { Thread thr = new Thread(thisRunnable); ... Regards, Zhong Yu From Alan.Bateman at oracle.com Thu Apr 24 08:16:58 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 24 Apr 2014 09:16:58 +0100 Subject: Memory leak in PendingFuture? In-Reply-To: References: Message-ID: <5358C87A.3070206@oracle.com> On 24/04/2014 07:17, Zhong Yu wrote: > In sun.nio.ch.PendingFuture, there is a static cached Throwable > > private static final CancellationException CANCELLED = > new CancellationException(); > > Apparently, its Throwable.backtrace field contains an array of all > Classes on the call stack. This will prevent the user classes from > being garbage collected. > I'll create a bug for this, it is an oversight that the stack wasn't cleared. -Alan. From Alan.Bateman at oracle.com Thu Apr 24 08:18:38 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 24 Apr 2014 09:18:38 +0100 Subject: Memory leak in AsynchronousFileChannel? In-Reply-To: References: Message-ID: <5358C8DE.6030006@oracle.com> On 24/04/2014 07:32, Zhong Yu wrote: > : > > These threads reference all class loaders on the calling stack of > open(), through > Thread.inheritedAccessControlContext.context[*].classloader, which > prevents user class loaders from being garbage collected. > Can you re-check this with 8u5 or 7u55 as these references should be cleared? -Alan From Alan.Bateman at oracle.com Thu Apr 24 08:19:50 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 24 Apr 2014 09:19:50 +0100 Subject: the best way to talk to multiple UDP/TCP servers? In-Reply-To: References: Message-ID: <5358C926.70402@oracle.com> On 18/04/2014 16:21, Wang Weijun wrote: > I'm writing a client which talks to 3 servers, and would like to write a method that > > sends request to server 1 at time 0:01 > sends request to server 2 at time 0:02 > sends request to server 3 at time 0:03 > sends request to server 1 at time 0:11 > sends request to server 2 at time 0:12 > sends request to server 3 at time 0:13 > waits until 0:20 > > During this whole period, whenever a response from a server is received, the method returns and no more request is sent. The protocol can be UDP or TCP. What is the best way I can do this with NIO? > You should be able to do this with non-blocking I/O and Selector, have you looked into that? -Alan. From weijun.wang at oracle.com Thu Apr 24 08:58:24 2014 From: weijun.wang at oracle.com (Wang Weijun) Date: Thu, 24 Apr 2014 16:58:24 +0800 Subject: the best way to talk to multiple UDP/TCP servers? In-Reply-To: <5358C926.70402@oracle.com> References: <5358C926.70402@oracle.com> Message-ID: <5BA77109-24EA-4E77-8F76-44CE3C36DA57@oracle.com> I'm playing with them now. UDP seems simpler, but TCP has more stages and seems I should carefully change the SelectionKey after each stage. Otherwise I see the select() method returning 0 immediately. --Max On Apr 24, 2014, at 16:19, Alan Bateman wrote: > On 18/04/2014 16:21, Wang Weijun wrote: >> I'm writing a client which talks to 3 servers, and would like to write a method that >> >> sends request to server 1 at time 0:01 >> sends request to server 2 at time 0:02 >> sends request to server 3 at time 0:03 >> sends request to server 1 at time 0:11 >> sends request to server 2 at time 0:12 >> sends request to server 3 at time 0:13 >> waits until 0:20 >> >> During this whole period, whenever a response from a server is received, the method returns and no more request is sent. The protocol can be UDP or TCP. What is the best way I can do this with NIO? >> > You should be able to do this with non-blocking I/O and Selector, have you looked into that? > > -Alan. From zhong.j.yu at gmail.com Thu Apr 24 19:19:43 2014 From: zhong.j.yu at gmail.com (Zhong Yu) Date: Thu, 24 Apr 2014 14:19:43 -0500 Subject: Memory leak in AsynchronousFileChannel? In-Reply-To: <5358C8DE.6030006@oracle.com> References: <5358C8DE.6030006@oracle.com> Message-ID: I tried 8u5 and 8u20b10, win x64. The problem seems still there. Zhong Yu On Thu, Apr 24, 2014 at 3:18 AM, Alan Bateman wrote: > On 24/04/2014 07:32, Zhong Yu wrote: >> >> : >> >> These threads reference all class loaders on the calling stack of >> open(), through >> Thread.inheritedAccessControlContext.context[*].classloader, which >> prevents user class loaders from being garbage collected. >> > Can you re-check this with 8u5 or 7u55 as these references should be > cleared? > > -Alan From Alan.Bateman at oracle.com Thu Apr 24 19:51:00 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 24 Apr 2014 20:51:00 +0100 Subject: Memory leak in AsynchronousFileChannel? In-Reply-To: References: <5358C8DE.6030006@oracle.com> Message-ID: <53596B24.5070604@oracle.com> On 24/04/2014 20:19, Zhong Yu wrote: > I tried 8u5 and 8u20b10, win x64. The problem seems still there. Okay, I see the issue (the change I was thinking about in 8u5 was for the security manager case only). -Alan From zhong.j.yu at gmail.com Thu Apr 24 20:43:01 2014 From: zhong.j.yu at gmail.com (Zhong Yu) Date: Thu, 24 Apr 2014 15:43:01 -0500 Subject: executor for AsynchronousFileChannel.open() In-Reply-To: <4F844F2B.5020807@oracle.com> References: <4F78B01F.1060800@oracle.com> <4F840849.30306@oracle.com> <4F844F2B.5020807@oracle.com> Message-ID: On Tue, Apr 10, 2012 at 10:18 AM, Alan Bateman wrote: > On 10/04/2012 16:12, Zhong Yu wrote: >> >> Thanks Alan. You are right, in the context of heavy I/O operations, my >> concern about threading overhead of the lightweight tasks is >> overblown. >> >> I'll simply use the default thread pool with unbounded number of >> threads. However one problem is that these threads never expire >> >> class sun.nio.ch.ThreadPool >> static ThreadPool createDefault() { >> >> ExecutorService executor = >> new ThreadPoolExecutor(0, Integer.MAX_VALUE, >> Long.MAX_VALUE, TimeUnit.MILLISECONDS, >> new SynchronousQueue(), >> threadFactory); >> >> After peak load, the extra threads will stay forever. Some users might >> find that unsettling. Shouldn't we set a human-scale timeout, like 60 >> seconds? no.... they are still alive after 5 minutes. Also, even if they have a timeout, if the application is active enough, it'll prevent these threads from exiting, right? Then the problem of referencing user classloader won't be alleviated. >> > It probably should, except for older versions of Windows where I/O > operations are tied to the initiating thread so we can't let them terminate > if there are outstanding I/O operations. > > -Alan. From Alan.Bateman at oracle.com Fri Apr 25 09:20:12 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 25 Apr 2014 10:20:12 +0100 Subject: executor for AsynchronousFileChannel.open() In-Reply-To: References: <4F78B01F.1060800@oracle.com> <4F840849.30306@oracle.com> <4F844F2B.5020807@oracle.com> Message-ID: <535A28CC.2090202@oracle.com> On 24/04/2014 21:43, Zhong Yu wrote: > : > no.... they are still alive after 5 minutes. > > Also, even if they have a timeout, if the application is active > enough, it'll prevent these threads from exiting, right? Then the > problem of referencing user classloader won't be alleviated. Yes, if there are several applications or usages in the same VM, all using the default channel group, then the threads may never be idle and so will not terminate. -Alan. From richard.warburton at gmail.com Sun Apr 27 23:08:28 2014 From: richard.warburton at gmail.com (Richard Warburton) Date: Mon, 28 Apr 2014 00:08:28 +0100 Subject: Covariant overrides on the Buffer Hierachy In-Reply-To: <535689DC.6060907@redhat.com> References: <534CF1D5.70605@oracle.com> <5350142D.8040706@oracle.com> <5354FC9F.1020409@gmail.com> <53559572.2080004@redhat.com> <535633C2.5090501@gmail.com> <5356650F.5020909@redhat.com> <53568254.3010106@gmail.com> <535689DC.6060907@redhat.com> Message-ID: Hi, There are multiple possible targets for invokevirtual >> position:(I)Ljava/nio/Buffer; - all the methods that override it in all >> subclasses loaded. It doesn't matter if they are final or not (only if >> they are effectively final or not). The non-finality of a method has a >> performance impact only if the method *is* overridden in any of the >> loaded subclasses, otherwise it is effectively final and treated as such >> by JIT (at least that's how I understand it - any hotspot JIT expert >> please debunk my claims). >> > > That is only true if you are calling the method on the base class, which > is normally seldom done (though it will be nearly always in legacy code, > but see below). > > That might also be the answer to why the synthetic method need not be >> marked as final if the covariant method is. The synthetic method can >> never be overridden in a sub-class (at least not by javac) - only the >> covariant method can be. >> > > Doesn't sound quite right to me, but I'll defer to any experts who might > wish to discuss it on compiler-dev. > > > But as Paul noted, the methods on Buffer are probably not used in hot >> loops alone, since they are just for reading/adjusting >> position/limit/mark. The hot loops probably also contain methods for >> reading/writing the buffer and those are only defined on particular >> sub-types of java.nio.Buffer, so it can reasonably be expected that the >> static (guaranteed) type of target upon which methods are called in hot >> loops is a particular subtype of java.nio.Buffer and JIT only has one >> method to choose from in this case. >> > > Yeah new code will call the covariant final method directly in almost all > cases for sure, having been compiled against it. The tricky case is old > code - even if the JIT can figure out that it is really a subclass being > invoked upon, it'd still be forced to target the non-covariant synthetic > method due to the specific method reference in the bytecode. Hopefully at > this point though, as you say, the lack of overriding classes will be > enough to optimize the dispatch. > So interestingly the original patch didn't add final to the covariant overrides. I've updated the patch to do this. http://cr.openjdk.java.net/~rwarburton/buffer-overrides-1/ I'm not sure what would satisfy people on the performance front. Some guidance would be most appreciated here. regards, Richard Warburton http://insightfullogic.com @RichardWarburto -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavel.rappo at oracle.com Tue Apr 29 10:01:17 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 29 Apr 2014 11:01:17 +0100 Subject: RFR JDK-8034057: Files.getFileStore and Files.isWritable do not work with SUBST'ed drives (win) Message-ID: Hi everyone, could you please review my change for JDK-8034057: http://cr.openjdk.java.net/~alanb/8034057/webrev/ It turns out some of win32 functions don't work well with SUBSTed drives (namely GetVolumePathName). As a result it fails to create WindowsFileStore. This is a simple fallback type solution. It first tries to create WindowsFileStore the old way and if it fails with specific error, tries different approach. Thanks -Pavel From chris.hegarty at oracle.com Tue Apr 29 10:47:23 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 29 Apr 2014 11:47:23 +0100 Subject: RFR JDK-8034057: Files.getFileStore and Files.isWritable do not work with SUBST'ed drives (win) In-Reply-To: References: Message-ID: I agree with the ?fallback? approach, and the changes look good to me Pavel. -Chris. On 29 Apr 2014, at 11:01, Pavel Rappo wrote: > Hi everyone, > > could you please review my change for JDK-8034057: http://cr.openjdk.java.net/~alanb/8034057/webrev/ > > It turns out some of win32 functions don't work well with SUBSTed drives (namely GetVolumePathName). As a result it fails to create WindowsFileStore. This is a simple fallback type solution. It first tries to create WindowsFileStore the old way and if it fails with specific error, tries different approach. > > Thanks > -Pavel From Alan.Bateman at oracle.com Tue Apr 29 11:01:47 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 29 Apr 2014 12:01:47 +0100 Subject: RFR JDK-8034057: Files.getFileStore and Files.isWritable do not work with SUBST'ed drives (win) In-Reply-To: References: Message-ID: <535F869B.1000600@oracle.com> On 29/04/2014 11:01, Pavel Rappo wrote: > Hi everyone, > > could you please review my change for JDK-8034057: http://cr.openjdk.java.net/~alanb/8034057/webrev/ > > It turns out some of win32 functions don't work well with SUBSTed drives (namely GetVolumePathName). As a result it fails to create WindowsFileStore. This is a simple fallback type solution. It first tries to create WindowsFileStore the old way and if it fails with specific error, tries different approach. > The approach looks okay, I can sponsor this for you. One small thing, are you sure you need to specify "file" in the exception message? FileSystemException will already include the file path in the exception message. -Alan From pavel.rappo at oracle.com Tue Apr 29 12:18:05 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 29 Apr 2014 13:18:05 +0100 Subject: RFR JDK-8034057: Files.getFileStore and Files.isWritable do not work with SUBST'ed drives (win) In-Reply-To: <535F869B.1000600@oracle.com> References: <535F869B.1000600@oracle.com> Message-ID: <7C13759E-5B41-4076-B60A-EA29D48917C5@oracle.com> Alan, you're right. That's an overkill. Here's the updated webrev: http://cr.openjdk.java.net/~alanb/8034057/webrev.02/ -Pavel On 29 Apr 2014, at 12:01, Alan Bateman wrote: > On 29/04/2014 11:01, Pavel Rappo wrote: >> Hi everyone, >> >> could you please review my change for JDK-8034057: http://cr.openjdk.java.net/~alanb/8034057/webrev/ >> >> It turns out some of win32 functions don't work well with SUBSTed drives (namely GetVolumePathName). As a result it fails to create WindowsFileStore. This is a simple fallback type solution. It first tries to create WindowsFileStore the old way and if it fails with specific error, tries different approach. >> > The approach looks okay, I can sponsor this for you. > > One small thing, are you sure you need to specify "file" in the exception message? FileSystemException will already include the file path in the exception message. > > -Alan From Alan.Bateman at oracle.com Tue Apr 29 12:24:20 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 29 Apr 2014 13:24:20 +0100 Subject: RFR JDK-8034057: Files.getFileStore and Files.isWritable do not work with SUBST'ed drives (win) In-Reply-To: <7C13759E-5B41-4076-B60A-EA29D48917C5@oracle.com> References: <535F869B.1000600@oracle.com> <7C13759E-5B41-4076-B60A-EA29D48917C5@oracle.com> Message-ID: <535F99F4.9000905@oracle.com> On 29/04/2014 13:18, Pavel Rappo wrote: > Alan, you're right. That's an overkill. Here's the updated webrev: http://cr.openjdk.java.net/~alanb/8034057/webrev.02/ > > -Pavel > > Thanks, let's go with that. -Alan. From Alan.Bateman at oracle.com Tue Apr 29 12:55:59 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 29 Apr 2014 13:55:59 +0100 Subject: 8041772: (ch) PendingFuture.CANCELLED has backtrace that potentially keeps objects alive Message-ID: <535FA15F.1000609@oracle.com> This is an issue that Zhong Yu pointed out recently [1]. Easily fixed by removing the static field and creating the CancellationException when the cancel method is called: http://cr.openjdk.java.net/~alanb/8041772/webrev/ I don't propose to include a test with this specific patch, instead I'd prefer to add a more general test that is long running to ensure that there aren't any unnecessary objects retained. -Alan. [1] http://mail.openjdk.java.net/pipermail/nio-dev/2014-April/002614.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Tue Apr 29 16:09:22 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 29 Apr 2014 17:09:22 +0100 Subject: 8041772: (ch) PendingFuture.CANCELLED has backtrace that potentially keeps objects alive In-Reply-To: <535FA15F.1000609@oracle.com> References: <535FA15F.1000609@oracle.com> Message-ID: <18E00116-CF2D-42C8-92CA-3B79E5611B15@oracle.com> This change looks good to me. -Chris. On 29 Apr 2014, at 13:55, Alan Bateman wrote: > > This is an issue that Zhong Yu pointed out recently [1]. Easily fixed by removing the static field and creating the CancellationException when the cancel method is called: > http://cr.openjdk.java.net/~alanb/8041772/webrev/ > > I don't propose to include a test with this specific patch, instead I'd prefer to add a more general test that is long running to ensure that there aren't any unnecessary objects retained. > > -Alan. > > [1] http://mail.openjdk.java.net/pipermail/nio-dev/2014-April/002614.html From zhong.j.yu at gmail.com Tue Apr 29 21:47:44 2014 From: zhong.j.yu at gmail.com (Zhong Yu) Date: Tue, 29 Apr 2014 16:47:44 -0500 Subject: 8041772: (ch) PendingFuture.CANCELLED has backtrace that potentially keeps objects alive In-Reply-To: <535FA15F.1000609@oracle.com> References: <535FA15F.1000609@oracle.com> Message-ID: This would change the stacktrace the application sees; does it matter? Previously, if thread X calls cancel(), then thread Y calls get(), Y will see an exception with Y's stacktrace. After the change, Y will see an exception with X's stacktrace (which is always confusing). Zhong Yu On Tue, Apr 29, 2014 at 7:55 AM, Alan Bateman wrote: > > This is an issue that Zhong Yu pointed out recently [1]. Easily fixed by > removing the static field and creating the CancellationException when the > cancel method is called: > http://cr.openjdk.java.net/~alanb/8041772/webrev/ > > I don't propose to include a test with this specific patch, instead I'd > prefer to add a more general test that is long running to ensure that there > aren't any unnecessary objects retained. > > -Alan. > > [1] http://mail.openjdk.java.net/pipermail/nio-dev/2014-April/002614.html From Alan.Bateman at oracle.com Wed Apr 30 07:28:59 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 30 Apr 2014 08:28:59 +0100 Subject: 8041772: (ch) PendingFuture.CANCELLED has backtrace that potentially keeps objects alive In-Reply-To: References: <535FA15F.1000609@oracle.com> Message-ID: <5360A63B.7050201@oracle.com> On 29/04/2014 22:47, Zhong Yu wrote: > This would change the stacktrace the application sees; does it matter? > It's a good question and could be confusing to have get throw CancellationException with the stack trace of the thread that called cancel. So I think we'll leave this as it was. -Alan From fweimer at redhat.com Wed Apr 30 18:34:24 2014 From: fweimer at redhat.com (Florian Weimer) Date: Wed, 30 Apr 2014 20:34:24 +0200 Subject: 8041772: (ch) PendingFuture.CANCELLED has backtrace that potentially keeps objects alive In-Reply-To: <5360A63B.7050201@oracle.com> References: <535FA15F.1000609@oracle.com> <5360A63B.7050201@oracle.com> Message-ID: <53614230.2030207@redhat.com> On 04/30/2014 09:28 AM, Alan Bateman wrote: > On 29/04/2014 22:47, Zhong Yu wrote: >> This would change the stacktrace the application sees; does it matter? >> > It's a good question and could be confusing to have get throw > CancellationException with the stack trace of the thread that called > cancel. So I think we'll leave this as it was. Would it make sense to call setStackTrace(new StackTraceElement[0]) on the marker exception? The class retention issue looks it could cause problems in some cases. -- Florian Weimer / Red Hat Product Security Team