From Alan.Bateman at Sun.COM Sat May 3 09:26:07 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sat, 03 May 2008 17:26:07 +0100 Subject: Just FYI In-Reply-To: <481BF136.9050508@sun.com> References: <481BF136.9050508@sun.com> Message-ID: <481C921F.4040901@sun.com> Arkadiy Sutchilin wrote: > Hi Alan, > > I've discovered, that there may be some delay between the cancel(true) > return and the appropriate IoFuture update. E.g. the code below MAY > fail (and actually failed from time to time, while rarely): > > boolean cancelled = future.cancel(true); > if (cancelled && !future.isDone()) { > return Status.failed(...); > } > > : Sorry about that - it was an oversight that I missed in the implementation and only noticed it when you exposed how IoFuture violates contract for cancel. The technical reason is of course that the forceful cancel is closing the channel but isn't setting the result or waiting for the result to be set. I've fixed this and so it will be included in the next drop and build (probably after JavaOne as there is too much going on just now). -Alan. From Alan.Bateman at Sun.COM Sat May 3 12:25:38 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sat, 03 May 2008 20:25:38 +0100 Subject: Multicast testing some tests fail In-Reply-To: References: <4806A87C.6030503@sun.com> <4806FE98.1040804@sun.com> <48070491.4050708@sun.com> <480705BD.8050501@sun.com> <480707CC.9020104@sun.com> <48071318.7080104@sun.com> Message-ID: <481CBC32.6070908@sun.com> Shantaram Pande wrote: > Hi Alan, > > >> Thanks. I'm pretty sure the issue is that the >> > multicast datagrams > >> are >> being dropped. It may be that your tests will only >> > be able to run > >> in the >> same subnet or alternatively only in conditions >> > where you have > >> control >> of the router to allow multicast datagrams to be >> > forwarded. > > I have changed the > dc.setOption(StandardSocketOption.IP_MULTICAST_TTL,16) > > and tested it. > Still the test is failing. > > Did you get a chance to capture network traces from both networks? I assume the issue is that the multicast datagrams aren't going through the router. If you can "snoop -o multicast" to capture the multicast packets on *both* the source and destination network while the test runs then we should be able to figure this out quickly. -Alan. From Shantaram.Pande at Sun.COM Mon May 5 02:30:39 2008 From: Shantaram.Pande at Sun.COM (Shantaram Pande) Date: Mon, 05 May 2008 15:00:39 +0530 Subject: Multicast testing some tests fail In-Reply-To: <481CBC32.6070908@sun.com> References: <4806A87C.6030503@sun.com> <4806FE98.1040804@sun.com> <48070491.4050708@sun.com> <480705BD.8050501@sun.com> <480707CC.9020104@sun.com> <48071318.7080104@sun.com> <481CBC32.6070908@sun.com> Message-ID: <481ED3BF.9070401@sun.com> Alan Bateman wrote: > Shantaram Pande wrote: >> Hi Alan, >> >> >>> Thanks. I'm pretty sure the issue is that the >>> >> multicast datagrams >>> are being dropped. It may be that your tests will only >>> >> be able to run >>> in the same subnet or alternatively only in conditions >>> >> where you have >>> control of the router to allow multicast datagrams to be >>> >> forwarded. >> >> I have changed the >> dc.setOption(StandardSocketOption.IP_MULTICAST_TTL,16) >> >> and tested it. Still the test is failing. >> >> > Did you get a chance to capture network traces from both networks? I > assume the issue is that the multicast datagrams aren't going through > the router. If you can "snoop -o multicast" to capture the > multicast packets on *both* the source and destination network while > the test runs then we should be able to figure this out quickly. > > -Alan. > > > Hi Alan, I tested it today. and verified that the multicast datagram is transmitted but just isn't received on the target network. So the issue is with teh router. Thanks for your help. Thanks and Regards, shantaram From Alan.Bateman at Sun.COM Mon May 5 08:57:58 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 05 May 2008 16:57:58 +0100 Subject: AsynchronousFileChannel Open options In-Reply-To: <481EEE0F.6060609@sun.com> References: <481EEE0F.6060609@sun.com> Message-ID: <481F2E86.9080000@sun.com> Umashankar wrote: > Hi Alan, > Creating an asynchronous file channel without passing any of the Open > options (READ,WRITE) does not open an existing file for reading. > As per the AsynchronousFileChannel#open() specification, it should > have opened the file for reading in this case. Here is what the spec > says : > "The READ and WRITE options determines if the file should be opened > for reading and/or writing. If neither option is contained in the > array then an existing file is opened for reading." > > I checked the behavior of a SeekableByteChannel in this scenario, it > seems to be working fine. > > Thanks, > Umashankar Umashankar - this is Windows only - right? If so, you are correct, and it should default to reading for this case. I just checked the unit tests I created for AsynchonrousFileChannel and they don't exercise the open method with all combinations of options so there may be a few other issues lurking here. The checking and mapping of options needs to be cleaned up anyway so assume this will be resolved for the next drop. I'll send you mail next week on this as I don't think much will happy this week due to JavaOne. -Alan. From Alan.Bateman at Sun.COM Mon May 5 09:12:32 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 05 May 2008 17:12:32 +0100 Subject: Query : AsynchronousFileChannel Open options In-Reply-To: <481EFB6F.5010203@sun.com> References: <481EFB6F.5010203@sun.com> Message-ID: <481F31F0.90009@sun.com> Umashankar wrote: > Hi Alan, > I noticed the following two behaviors of > AsynchronousFileChannel.open() method that seem to be violating the spec: > > 1. Invoking AsynchronousFileChannel.open() method specifying path to a > file that does not exist and CREAT_NEW or CREATE as Open Options > throws NoSuchFileException. As per the spec, if CREATE_NEW or > CREATE option is present then a new file should be created if it does > not exist. > > 2.Invoking AsynchronousFileChannel.open() method specifying path to a > file that exists and CREAT_NEW as Open Option does not throw > FileAlreadyExistsException. > > Please clarify. What other options are included? I ask because the CREATE and CREATE_NEW options are specified to be ignored if the file is opened only for reading. I quickly checked CREATE + WRITE, and CREATE_NEW + WRITE and they are behaving as expected. -Alan. From Umashankar.Ningaiah at Sun.COM Mon May 5 10:11:34 2008 From: Umashankar.Ningaiah at Sun.COM (Umashankar) Date: Mon, 05 May 2008 22:41:34 +0530 Subject: Query : AsynchronousFileChannel Open options In-Reply-To: <481F31F0.90009@sun.com> References: <481EFB6F.5010203@sun.com> <481F31F0.90009@sun.com> Message-ID: <481F3FC6.9010605@sun.com> Alan Bateman wrote: > Umashankar wrote: >> Hi Alan, >> I noticed the following two behaviors of >> AsynchronousFileChannel.open() method that seem to be violating the >> spec: >> >> 1. Invoking AsynchronousFileChannel.open() method specifying path to >> a file that does not exist and CREAT_NEW or CREATE as Open Options >> throws NoSuchFileException. As per the spec, if CREATE_NEW or >> CREATE option is present then a new file should be created if it does >> not exist. >> >> 2.Invoking AsynchronousFileChannel.open() method specifying path to a >> file that exists and CREAT_NEW as Open Option does not throw >> FileAlreadyExistsException. >> >> Please clarify. > What other options are included? I ask because the CREATE and > CREATE_NEW options are specified to be ignored if the file is opened > only for reading. I quickly checked CREATE + WRITE, and CREATE_NEW + > WRITE and they are behaving as expected. > > -Alan. Alan, I am specifying either CREATE_NEW or CREATE option only. No other options included. Thanks, Umashankar From Alan.Bateman at Sun.COM Mon May 5 10:50:27 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 05 May 2008 18:50:27 +0100 Subject: Query : AsynchronousFileChannel Open options In-Reply-To: <481F3FC6.9010605@sun.com> References: <481EFB6F.5010203@sun.com> <481F31F0.90009@sun.com> <481F3FC6.9010605@sun.com> Message-ID: <481F48E3.8060009@sun.com> Umashankar wrote: > Alan, > I am specifying either CREATE_NEW or CREATE option only. No other > options included. > As currently specified, if neither the READ or WRITE option is present then the method defaults to opening an existing file for reading. This makes it easy for the common case but perhaps non-intuitive when other options are present (in particular options that are usually only relevant for writing, or reading + writing). I'll note this as something to re-examine after JavaOne. -Alan. From Alan.Bateman at Sun.COM Wed May 14 07:19:55 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 14 May 2008 15:19:55 +0100 Subject: 6642290 Message-ID: <482AF50B.9020000@sun.com> Umashankar, I've been discussing Solaris bug 6642290 [1] with Prakash. You may recall that this is the underlying bug that causes watch keys to be automatically cancelled on Nevada when the automounter tries, but fails, to unmount a file system because it is in use. One of your tests runs into this when watching files on a home directory and you've asked me about it a few times. It looks like this isn't going to be feasible to fix in the kernel but he has suggested a workaround that I have implemented. The workaround has one one side effect that arises when watching directories an automounted file systems that are otherwise not in use. In that case, if the automounter does unmount the file system then the re-registration in the watch service implementation will cause the file system to be immediately re-mounted. We don't think this will be a big problem. So assume this will be resolved for the next build. Once you have that then you should be able to run that test for >10 minutes (automount's default idle duration). He did alert me to one other bug that you should be aware of. The bug is 6636438 [2]. That one needs 6651501 [3] to be implemented first. I don't plan to put in a workaround for that. -Alan. [1] http://bugs.opensolaris.org/view_bug.do?bug_id=6642290 [2] http://bugs.opensolaris.org/view_bug.do?bug_id=6636438 [3] http://bugs.opensolaris.org/view_bug.do?bug_id=6651501 From Umashankar.Ningaiah at Sun.COM Thu May 15 04:58:36 2008 From: Umashankar.Ningaiah at Sun.COM (Umashankar) Date: Thu, 15 May 2008 17:28:36 +0530 Subject: connect() method behavior during asynchronous close Message-ID: <482C256C.4010906@sun.com> Hi Alan, In one of my AsynchronousSocketChannel tests, I am testing the behavior of the connect() method in the event of an asynchronous close of the socket channel. The test sometimes hangs on windows. I did jstack -l on the hung process. I don't see any deadlocks in the thread dump. Attached is the thread dump of the hung process. Please let me know if there is a bug. Thanks, Umashankar -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: dump.log Url: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20080515/8dca1cb4/attachment.ksh From Alan.Bateman at Sun.COM Thu May 15 05:29:18 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 15 May 2008 13:29:18 +0100 Subject: connect() method behavior during asynchronous close In-Reply-To: <482C256C.4010906@sun.com> References: <482C256C.4010906@sun.com> Message-ID: <482C2C9E.8050804@sun.com> Umashankar wrote: > Hi Alan, > In one of my AsynchronousSocketChannel tests, I am testing the > behavior of the connect() method in the event of an asynchronous close > of the socket channel. > The test sometimes hangs on windows. I did jstack -l on the hung > process. I don't see any deadlocks in the thread dump. > Attached is the thread dump of the hung process. > > Please let me know if there is a bug. > You are right and there is bug here. The deadlock isn't obvious at first glance but the stack trace for Thread-0 reveals all. It is trying to close the channel (which requires the channel's write lock) while holding the read lock as it updates the socket properties after the connection is established. You are right that it is Windows specific and furthermore this does appear to be the only case where this can happen. The fix is simple and I'll include it in the next build. Thanks for persevering with these tests to flush out these lurking issues. -Alan. From neal at gafter.com Sun May 18 16:18:31 2008 From: neal at gafter.com (Neal Gafter) Date: Sun, 18 May 2008 16:18:31 -0700 Subject: jdk build error Message-ID: <15e8b9d20805181618i60c06c3fhf8a2c0cb651b4b84@mail.gmail.com> I'm trying to prepare a patch for closures, but I still can't build openjdk even from the master on my Ubuntu system. Did I do something wrong? Is anybody working on this? Here's where the build fails: /home/gafter/jdk7/jdk7-image/bin/java -client -Xmx896m -Xms128m -XX:PermSize=32m -XX:MaxPermSize=160m -Xbootclasspath/p:/home/gafter/jdk7/tl0.hgf/build/linux-i586/langtools/dist/bootstrap/lib/javac.jar -jar /home/gafter/jdk7/tl0.hgf/build/linux-i586/langtools/dist/bootstrap/lib/javac.jar -source 1.5 -target 5 -encoding ascii -Xbootclasspath:/home/gafter/jdk7/tl0.hgf/build/linux-i586/classes -sourcepath /home/gafter/jdk7/tl0.hgf/build/linux-i586/gensrc:../../../src/solaris/classes:../../../src/share/classes -d /home/gafter/jdk7/tl0.hgf/build/linux-i586/classes @/home/gafter/jdk7/tl0.hgf/build/linux-i586/tmp/java/java.lang/java/.classes.list.filtered /home/gafter/jdk7/tl0.hgf/build/linux-i586/gensrc/java/nio/charset/CharsetEncoder.java:142: cannot find symbol symbol : class $replType$ location: class java.nio.charset.CharsetEncoder private $replType$ replacement; ^ /home/gafter/jdk7/tl0.hgf/build/linux-i586/gensrc/java/nio/charset/CharsetEncoder.java:185: cannot find symbol symbol : class $replType$ location: class java.nio.charset.CharsetEncoder $replType$ replacement) ^ /home/gafter/jdk7/tl0.hgf/build/linux-i586/gensrc/java/nio/charset/CharsetEncoder.java:246: cannot find symbol symbol : class $replType$ location: class java.nio.charset.CharsetEncoder public final $replType$ replacement() { ^ /home/gafter/jdk7/tl0.hgf/build/linux-i586/gensrc/java/nio/charset/CharsetEncoder.java:275: cannot find symbol symbol : class $replType$ location: class java.nio.charset.CharsetEncoder public final CharsetEncoder replaceWith($replType$ newReplacement) { ^ /home/gafter/jdk7/tl0.hgf/build/linux-i586/gensrc/java/nio/charset/CharsetEncoder.java:301: cannot find symbol symbol : class $replType$ location: class java.nio.charset.CharsetEncoder protected void implReplaceWith($replType$ newReplacement) { ^ Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. 5 errors make[3]: *** [.compile.classlist] Error 1 make[3]: Leaving directory `/home/gafter/jdk7/tl0.hgf/jdk/make/java/java' -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20080518/a9d06045/attachment.html From neal at gafter.com Sun May 18 20:52:17 2008 From: neal at gafter.com (Neal Gafter) Date: Sun, 18 May 2008 20:52:17 -0700 Subject: jdk build error In-Reply-To: <4830D6C6.9040500@sun.com> References: <15e8b9d20805181618i60c06c3fhf8a2c0cb651b4b84@mail.gmail.com> <4830D6C6.9040500@sun.com> Message-ID: <15e8b9d20805182052t6ace8d37i49224d248ed7c039@mail.gmail.com> That did it. Thanks Tim and Martin! -Neal On Sun, May 18, 2008 at 6:24 PM, Tim Bell wrote: > Hi Neal: > > /home/gafter/jdk7/tl0.hgf/build/linux-i586/gensrc/java/nio/charset/CharsetEncoder.java:185: >> cannot find symbol >> symbol : class $replType$ >> location: class java.nio.charset.CharsetEncoder >> $replType$ replacement) >> >> > > I believe this is the /bin/bash versus /bin/dash issue on recent release(s) > of Ubuntu. > > There are a couple of related bug IDs: > > http://bugs.sun.com/view_bug.do?bug_id=6681798 > https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 > > Also this thread on build-dev (at) openjdk.java.net: > > http://mail.openjdk.java.net/pipermail/build-dev/2008-May/001047.html > > The workaround (according to Martin) is 'sudo dpkg-reconfigure dash' and > answer no > > HTH - Tim > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20080518/89ccf0b2/attachment.html From Tim.Bell at Sun.COM Sun May 18 18:24:22 2008 From: Tim.Bell at Sun.COM (Tim Bell) Date: Sun, 18 May 2008 18:24:22 -0700 Subject: jdk build error In-Reply-To: <15e8b9d20805181618i60c06c3fhf8a2c0cb651b4b84@mail.gmail.com> References: <15e8b9d20805181618i60c06c3fhf8a2c0cb651b4b84@mail.gmail.com> Message-ID: <4830D6C6.9040500@sun.com> Hi Neal: > /home/gafter/jdk7/tl0.hgf/build/linux-i586/gensrc/java/nio/charset/CharsetEncoder.java:185: > cannot find symbol > symbol : class $replType$ > location: class java.nio.charset.CharsetEncoder > $replType$ replacement) > I believe this is the /bin/bash versus /bin/dash issue on recent release(s) of Ubuntu. There are a couple of related bug IDs: http://bugs.sun.com/view_bug.do?bug_id=6681798 https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 Also this thread on build-dev (at) openjdk.java.net: http://mail.openjdk.java.net/pipermail/build-dev/2008-May/001047.html The workaround (according to Martin) is 'sudo dpkg-reconfigure dash' and answer no HTH - Tim From Alan.Bateman at Sun.COM Wed May 21 08:48:58 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 21 May 2008 16:48:58 +0100 Subject: zip provider Message-ID: <4834446A.7060302@sun.com> Rajendra, I finally got around to integrating your zip provider. It's mostly make file work and it now builds as a demo that can be installed or added to the class path. To catch bit rot I've also added a regression/unit test that exercises the provider. Our Mercurial repository should be setup on hg.openjdk.java.net soon and I'll push the changes there once that is available. Regards, -Alan. From Joe.Darcy at Sun.COM Thu May 22 18:18:17 2008 From: Joe.Darcy at Sun.COM (Joe Darcy) Date: Thu, 22 May 2008 18:18:17 -0700 Subject: network tests Message-ID: <48361B59.1090808@sun.com> Greetings. Recently Mark Wielaard started an effort to run the regression tests on OpenJDK builds and publish the results (http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2008-May/001921.html). Certain tests in the nio and networking area assume Sun-internal hosts like javaweb are available; the shell test test/java/net/InetAddress/ptr/lookup.sh has a string for javaweb and test/java/nio/channels/TestUtil.java has the following constants used by other tests: // Test hosts used by the channels tests - change these when // executing in a different network. public static final String HOST = "javaweb.sfbay.sun.com"; public static final String REFUSING_HOST = "jano.sfbay.sun.com"; public static final String FAR_HOST = "theclub.ireland.sun.com"; public static final String UNRESOLVABLE_HOST = "blah-blah.blah-blah.blah"; Mark observed these three hosts are needed: > With daytime and echo available. The only wrinkle is that some tests > expect to be able to use quick timeouts from HOST, but to need long > timeouts from FAR_HOST. I think we could provide something public for > this. But I might have missed some tests that need other services. Do > you have a full overview? (http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2008-May/001972.html) First, is this all the networking configuration information that is used in the regression tests? Second, it would be helpful is this host information could be configured without changing the sources of the test each time. I've spoken briefly to Jon about ways this kind of information can be passed into a jtreg run. Environment variables can be used, as can system properties. A system property could also be used to specify a file from which the information was read. With a bit of jtreg hacking, it is feasible portions of the jtharness interview procedure could be exposed to configure information that way too. -Joe From Alan.Bateman at Sun.COM Fri May 23 02:20:57 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Fri, 23 May 2008 10:20:57 +0100 Subject: network tests In-Reply-To: <48361B59.1090808@sun.com> References: <48361B59.1090808@sun.com> Message-ID: <48368C79.8050307@sun.com> Joe Darcy wrote: > Greetings. > > Recently Mark Wielaard started an effort to run the regression tests > on OpenJDK builds and publish the results > (http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2008-May/001921.html). > Certain tests in the nio and networking area assume Sun-internal > hosts like javaweb are available; the shell test > > test/java/net/InetAddress/ptr/lookup.sh > > has a string for javaweb and > > test/java/nio/channels/TestUtil.java > > has the following constants used by other tests: > > // Test hosts used by the channels tests - change these when > // executing in a different network. > public static final String HOST = "javaweb.sfbay.sun.com"; > public static final String REFUSING_HOST = "jano.sfbay.sun.com"; > public static final String FAR_HOST = "theclub.ireland.sun.com"; > public static final String UNRESOLVABLE_HOST = > "blah-blah.blah-blah.blah"; > > Mark observed these three hosts are needed: >> With daytime and echo available. The only wrinkle is that some tests >> expect to be able to use quick timeouts from HOST, but to need long >> timeouts from FAR_HOST. I think we could provide something public for >> this. But I might have missed some tests that need other services. Do >> you have a full overview? > (http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2008-May/001972.html) > > > First, is this all the networking configuration information that is > used in the regression tests? For the tests in the java/nio and sun/nio tree then the dependencies are centralized in TestUtil.java. When we moved these tests from closed->open we did talk about moving this configuration into a properties/configuration file but figured there wasn't much difference between editing a config file vs. a source file. For the classic networking, the dependencies are scattered. I believe some of the tests weren't moved from closed->open for this reason. Some of the tests uses an embedded HTTP or FTP server but there are others that depend on DNS, proxy, etc. Michael or Jessie may have more to say on this. BTW: I briefly scanned the summary and most of the java/nio failures appear to be *HOST related. Now that folks are running these tests outside Sun it is time to own up to a few intermittent/timing related failures with 4 or 5 of these tests. I've been meaning to fix these test issues for a while but I haven't had the cycles. I'll dig up my notes on these. One other thing - I notice test/java/nio/channels/DatagramChannel/NotBound.java on the list - that test is failing because it is missing the public modifier. The test is bogus anyway so it might be best to just remove this test from jdk6 and open6. For jdk7/OpenJDK I have spec clarification in the works to fix the spec hole and as part of that work the test will be replaced (see 6621689). > Second, it would be helpful is this host information could be > configured without changing the sources of the test each time. I've > spoken briefly to Jon about ways this kind of information can be > passed into a jtreg run. Environment variables can be used, as can > system properties. A system property could also be used to specify a > file from which the information was read. With a bit of jtreg > hacking, it is feasible portions of the jtharness interview procedure > could be exposed to configure information that way too. I would be concerned with a solution that requires environment variables or jtreg requiring configuration from somewhere other than the test suite. At least in Sun (sorry for being selfish), these tests are run by various teams on hundreds of machines. I don't think we'd want to require the javatest interview be run or have .jti files stored in various places with configuration information that will quickly go stale. If you mean that the configuration can be overridden via some input to jtreg that might be okay but would require the distributions and others to maintain that configuration. Anyway, I'm interested to hear the outcome of the discussion. -Alan. From Joe.Darcy at Sun.COM Fri May 23 16:45:45 2008 From: Joe.Darcy at Sun.COM (Joe Darcy) Date: Fri, 23 May 2008 16:45:45 -0700 Subject: network tests In-Reply-To: <48368C79.8050307@sun.com> References: <48361B59.1090808@sun.com> <48368C79.8050307@sun.com> Message-ID: <48375729.2000301@sun.com> Alan Bateman wrote: > Joe Darcy wrote: >> Greetings. >> >> Recently Mark Wielaard started an effort to run the regression tests >> on OpenJDK builds and publish the results >> (http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2008-May/001921.html). >> Certain tests in the nio and networking area assume Sun-internal >> hosts like javaweb are available; the shell test >> >> test/java/net/InetAddress/ptr/lookup.sh >> >> has a string for javaweb and >> >> test/java/nio/channels/TestUtil.java >> >> has the following constants used by other tests: >> >> // Test hosts used by the channels tests - change these when >> // executing in a different network. >> public static final String HOST = "javaweb.sfbay.sun.com"; >> public static final String REFUSING_HOST = "jano.sfbay.sun.com"; >> public static final String FAR_HOST = "theclub.ireland.sun.com"; >> public static final String UNRESOLVABLE_HOST = >> "blah-blah.blah-blah.blah"; >> >> Mark observed these three hosts are needed: >>> With daytime and echo available. The only wrinkle is that some tests >>> expect to be able to use quick timeouts from HOST, but to need long >>> timeouts from FAR_HOST. I think we could provide something public for >>> this. But I might have missed some tests that need other services. Do >>> you have a full overview? >> (http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2008-May/001972.html) >> >> >> First, is this all the networking configuration information that is >> used in the regression tests? > For the tests in the java/nio and sun/nio tree then the dependencies > are centralized in TestUtil.java. When we moved these tests from > closed->open we did talk about moving this configuration into a > properties/configuration file but figured there wasn't much difference > between editing a config file vs. a source file. I think being able to pass in your own config file would be useful for this use case; it is awkward to have to edit the source to the tests to get them to pass :-) If -Dnetworking.test.config.file were defined, the test could use those values instead of the defaults... > > For the classic networking, the dependencies are scattered. I believe > some of the tests weren't moved from closed->open for this reason. > Some of the tests uses an embedded HTTP or FTP server but there are > others that depend on DNS, proxy, etc. Michael or Jessie may have more > to say on this. Could the classic networking tests be refactored a bit to be centralized like the nio tests? Would it be natural for classic networking and nio to get their host information from the same place? > > BTW: I briefly scanned the summary and most of the java/nio failures > appear to be *HOST related. Now that folks are running these tests > outside Sun it is time to own up to a few intermittent/timing related > failures with 4 or 5 of these tests. I've been meaning to fix these > test issues for a while but I haven't had the cycles. I'll dig up my > notes on these. One other thing - I notice > test/java/nio/channels/DatagramChannel/NotBound.java on the list - > that test is failing because it is missing the public modifier. The > test is bogus anyway so it might be best to just remove this test from > jdk6 and open6. For jdk7/OpenJDK I have spec clarification in the > works to fix the spec hole and as part of that work the test will be > replaced (see 6621689). Yes, fixing the intermittent failures and removing the invalid test in OpenJDK 6 sounds good. >> Second, it would be helpful is this host information could be >> configured without changing the sources of the test each time. I've >> spoken briefly to Jon about ways this kind of information can be >> passed into a jtreg run. Environment variables can be used, as can >> system properties. A system property could also be used to specify a >> file from which the information was read. With a bit of jtreg >> hacking, it is feasible portions of the jtharness interview procedure >> could be exposed to configure information that way too. > I would be concerned with a solution that requires environment > variables or jtreg requiring configuration from somewhere other than > the test suite. At least in Sun (sorry for being selfish), these tests > are run by various teams on hundreds of machines. I don't think we'd > want to require the javatest interview be run or have .jti files > stored in various places with configuration information that will > quickly go stale. If you mean that the configuration can be overridden > via some input to jtreg that might be okay but would require the > distributions and others to maintain that configuration. Anyway, I'm > interested to hear the outcome of the discussion. Even if the Sun-internal settings remain baked in, I think an easy way to pass in a limited number of configuration settings is appropriate and shouldn't be too hard to accommodate in the test code. Presumably an interface like "openjtreg -Dmy.config.options=foo ..." wouldn't be too awkward to use. -Joe From Alan.Bateman at Sun.COM Sat May 24 06:29:15 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sat, 24 May 2008 14:29:15 +0100 Subject: network tests In-Reply-To: <48375729.2000301@sun.com> References: <48361B59.1090808@sun.com> <48368C79.8050307@sun.com> <48375729.2000301@sun.com> Message-ID: <4838182B.8090709@sun.com> Joe Darcy wrote: > : > Could the classic networking tests be refactored a bit to be > centralized like the nio tests? Would it be natural for classic > networking and nio to get their host information from the same place? There was a time when the networking tests had dependencies on quite a few web servers, proxies, etc. but there was effort to replace these tests so there should only be a few dependencies remaining. I briefly looked at the tests today and I was surprised to only find one test that attempts a lookup of a Sun internal host. The other tests with dependencies weren't moved from closed->open. In any case, it should be straight-forward to centralize the dependencies into one place. > > : > Even if the Sun-internal settings remain baked in, I think an easy way > to pass in a limited number of configuration settings is appropriate > and shouldn't be too hard to accommodate in the test code. Presumably > an interface like "openjtreg -Dmy.config.options=foo ..." wouldn't be > too awkward to use. If by "baked in" you mean a default configuration file for when the property or other knob isn't used then it seems reasonable. Are you thinking of passing in the individual parameters as properties or have one property to locate a properties/config file with the settings? -Alan. From Jonathan.Gibbons at Sun.COM Sun May 25 08:08:49 2008 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Sun, 25 May 2008 08:08:49 -0700 Subject: network tests In-Reply-To: <4838182B.8090709@sun.com> References: <48361B59.1090808@sun.com> <48368C79.8050307@sun.com> <48375729.2000301@sun.com> <4838182B.8090709@sun.com> Message-ID: Alan, I can imagine two possibilities here: 1) pass a system property to each test that defines a config file in properties format that canbe read by any test that needs it 2) allow a new option to main/applet/shell tests that define a config value that will be passed to the test via a system property or environment variable. Eg @run main/config=XYZ The former requires less change to the test but does require the test to be able to read files. The latter is a bigger change to the spec but doesn't run into any security manager issues. -- Jon On May 24, 2008, at 6:29 AM, Alan Bateman wrote: > Joe Darcy wrote: >> : >> Could the classic networking tests be refactored a bit to be >> centralized like the nio tests? Would it be natural for classic >> networking and nio to get their host information from the same place? > There was a time when the networking tests had dependencies on quite > a few web servers, proxies, etc. but there was effort to replace > these tests so there should only be a few dependencies remaining. I > briefly looked at the tests today and I was surprised to only find > one test that attempts a lookup of a Sun internal host. The other > tests with dependencies weren't moved from closed->open. In any > case, it should be straight-forward to centralize the dependencies > into one place. > >> >> : >> Even if the Sun-internal settings remain baked in, I think an easy >> way to pass in a limited number of configuration settings is >> appropriate and shouldn't be too hard to accommodate in the test >> code. Presumably an interface like "openjtreg - >> Dmy.config.options=foo ..." wouldn't be too awkward to use. > If by "baked in" you mean a default configuration file for when the > property or other knob isn't used then it seems reasonable. Are you > thinking of passing in the individual parameters as properties or > have one property to locate a properties/config file with the > settings? > > -Alan. From Alan.Bateman at Sun.COM Mon May 26 03:09:54 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 26 May 2008 11:09:54 +0100 Subject: [Fwd: Re: Mime types for gnome_vfs_mime_type_from_name?] Message-ID: <483A8C72.8010209@sun.com> -------- Original Message -------- Subject: Re: Mime types for gnome_vfs_mime_type_from_name? Date: Mon, 26 May 2008 11:00:04 +0100 From: Darren Kenny To: Alan Bateman CC: Darren Kenny References: <48358DDA.2060609 at sun.com> Hi Alan, : I think that what's likely to be wrong here is that the pattern for the x-mozilla-bookmarks is too general, it should be something more specific than it's parent type text/html. (e.g. bookmarks*.html ?) Looks like there is already a bug for this at: https://bugs.freedesktop.org/show_bug.cgi?id=11843 No sign of it being fixed yet though, so I just added a quick comment to see if there's any update. It would seem that the bug is being "deferred" until typing in KDE and GNOME supports looking at the 'magic' in the files - but I'm not sure when that might happen. BTW - as a heads up - you need to be aware that gnome_vfs is on the way out, and could happen in the next year or so on Linux, might stay longer on Solaris due to support needs. It's being replaced with another implementation that is located in glib (much lower down the library stack) rather than the gnome layer. It's now broken in to two elements, GLIB and GIO. For the part you're looking at, it's GIO, and probably need the function g_content_type_guess() [1]. It's possible that this implementation may (given the function signature) look at the file contents, and may work for you. HTH, Darren. [1] http://library.gnome.org/devel/gio/stable/gio-GContentType.html#g-content-type-guess From Alan.Bateman at Sun.COM Mon May 26 03:14:10 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 26 May 2008 11:14:10 +0100 Subject: [Fwd: Re: Mime types for gnome_vfs_mime_type_from_name?] In-Reply-To: <483A8C72.8010209@sun.com> References: <483A8C72.8010209@sun.com> Message-ID: <483A8D72.8010508@sun.com> Rajendra - forgot to add a note here to say that this issue impacts the Files#probeContentType method so you need to be aware of it with any tests that probe .html files expecting their content type to be text/html. From david.lloyd at redhat.com Mon May 26 09:52:52 2008 From: david.lloyd at redhat.com (David M. Lloyd) Date: Mon, 26 May 2008 11:52:52 -0500 Subject: NIO Filesystem API Message-ID: <483AEAE4.8060105@redhat.com> It's probably a bit late in the game to request this... but... is there any chance that you could consider locating all the filesystem stuff in a "javax.filesystems" package heirarchy, as opposed to the current "java.nio.filesystems"? The reason is that it seems to be mostly, if not completely, self-contained (with respect to the other NIO stuff), and it would certainly improve adoption of the new FileSystem API if backport implementations would be allowed (which, unless I'm mistaken, is much more difficult from a licensing perspective when the package in question is located in the "java" space). Also, the filesystems API has little to do with "NIO", which up until now has been concerned primarily with channels, selectors, buffers, and transcoding of buffers. What do you think? - DML From Alan.Bateman at Sun.COM Mon May 26 10:55:46 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 26 May 2008 18:55:46 +0100 Subject: NIO Filesystem API In-Reply-To: <483AEAE4.8060105@redhat.com> References: <483AEAE4.8060105@redhat.com> Message-ID: <483AF9A2.3010205@sun.com> David M. Lloyd wrote: > It's probably a bit late in the game to request this... but... is > there any chance that you could consider locating all the filesystem > stuff in a "javax.filesystems" package heirarchy, as opposed to the > current "java.nio.filesystems"? The reason is that it seems to be > mostly, if not completely, self-contained (with respect to the other > NIO stuff), and it would certainly improve adoption of the new > FileSystem API if backport implementations would be allowed (which, > unless I'm mistaken, is much more difficult from a licensing > perspective when the package in question is located in the "java" space). > > Also, the filesystems API has little to do with "NIO", which up until > now has been concerned primarily with channels, selectors, buffers, > and transcoding of buffers. > > What do you think? The package name is java.nio.file and we don't have any plans to move from that. There are dependencies in that package on new classes that we've added to the channels package so it isn't as self-contained as it might seem (Path#newSeekableByteChannel and FileSystemProvider#newAsynchronousFileChannel come to mind). That said, there is some interest in "back-porting" the file system part of the API so that it can be used with jdk6. Carl may wish to say more about this. Unfortunately any "back-port" does require choosing a package name. -Alan. From kevinb at google.com Mon May 26 12:05:51 2008 From: kevinb at google.com (kevin bourrillion) Date: Mon, 26 May 2008 12:05:51 -0700 Subject: NIO Filesystem API In-Reply-To: <483AF9A2.3010205@sun.com> References: <483AEAE4.8060105@redhat.com> <483AF9A2.3010205@sun.com> Message-ID: <108fcdeb0805261205s4fd7b106w4a411fd0a2f810a4@mail.gmail.com> On Mon, May 26, 2008 at 10:55 AM, Alan Bateman wrote: > > That said, there is some interest in "back-porting" the file system part of > the API so that it can be used with jdk6. Carl may wish to say more about > this. Unfortunately any "back-port" does require choosing a package name. The current plan is to use jsr203.* in place of java.* in the backport. -- Kevin Bourrillion @ Google internal: go/javalibraries google-collections.googlecode.com google-guice.googlecode.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20080526/7fc07782/attachment.html From Joe.Darcy at Sun.COM Tue May 27 09:23:10 2008 From: Joe.Darcy at Sun.COM (Joseph D. Darcy) Date: Tue, 27 May 2008 09:23:10 -0700 Subject: network tests In-Reply-To: <4838182B.8090709@sun.com> References: <48361B59.1090808@sun.com> <48368C79.8050307@sun.com> <48375729.2000301@sun.com> <4838182B.8090709@sun.com> Message-ID: <483C356E.2070300@sun.com> Alan Bateman wrote: > Joe Darcy wrote: >> : >> Could the classic networking tests be refactored a bit to be >> centralized like the nio tests? Would it be natural for classic >> networking and nio to get their host information from the same place? > There was a time when the networking tests had dependencies on quite a > few web servers, proxies, etc. but there was effort to replace these > tests so there should only be a few dependencies remaining. I briefly > looked at the tests today and I was surprised to only find one test that > attempts a lookup of a Sun internal host. The other tests with > dependencies weren't moved from closed->open. In any case, it should be > straight-forward to centralize the dependencies into one place. > >> >> : >> Even if the Sun-internal settings remain baked in, I think an easy way >> to pass in a limited number of configuration settings is appropriate >> and shouldn't be too hard to accommodate in the test code. Presumably >> an interface like "openjtreg -Dmy.config.options=foo ..." wouldn't be >> too awkward to use. > If by "baked in" you mean a default configuration file for when the > property or other knob isn't used then it seems reasonable. Are you > thinking of passing in the individual parameters as properties or have > one property to locate a properties/config file with the settings? I was thinking of one property to locate the properties/config file. What to you think Mark? -Joe From xxxxx at yyyy.zz Wed May 28 03:07:23 2008 From: xxxxx at yyyy.zz (zzzzzzzzzzzzzzzzzzzzzzzzzzzzzz) Date: Wed, 28 May 2008 12:07:23 +0200 Subject: NIO: missing createTempFile() and deleteOnExit() Message-ID: <000801c8c0aa$a5207440$d705a8c0@inet> I'm very interested to play with the NIO in the fields of portable applications to use it for encrypted filesystems in a singel file and so on. To port existing applications and dbs it would be nice to have two more things in the NIO: createTempFile() and deleteOnExit(). If I missed it in the existing NIO please give me a hint. Otherwise my proposals just need very little modification of the existing interfaces. PROPOSALS: static File createTempFile(String prefix, String suffix) static File createTempFile(String prefix, String suffix, File directory) ---> public abstract Path createFile(Attribute... attrs) throws IOException with Attributes for prefix, suffix [and directory] ( or by breaking the existing interface ---> public abstract Path createTempFile(...) throws IOException ) void deleteOnExit() ---> public static final StandardOpenOption DELETE_ON_EXIT - -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20080528/bd9347ec/attachment.html From Alan.Bateman at Sun.COM Wed May 28 05:45:40 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 28 May 2008 13:45:40 +0100 Subject: NIO: missing createTempFile() and deleteOnExit() In-Reply-To: <000801c8c0aa$a5207440$d705a8c0@inet> References: <000801c8c0aa$a5207440$d705a8c0@inet> Message-ID: <483D53F4.6030608@sun.com> zzzzzzzzzzzzzzzzzzzz wrote: > > I'm very interested to play with the NIO in the fields of portable > applications to use it for encrypted filesystems in a singel file and > so on. To port existing applications and dbs it would be nice to have > two more things in the NIO: createTempFile() and deleteOnExit(). > > If I missed it in the existing NIO please give me a hint. > > Otherwise my proposals just need very little modification of the > existing interfaces. > > PROPOSALS: > > static File createTempFile(String prefix, String suffix) > static File createTempFile(String prefix, String suffix, File directory) > ---> public abstract Path createFile(Attribute... attrs) throws > IOException > with Attributes for prefix, suffix [and directory] > ( or by breaking the existing interface > ---> public abstract Path createTempFile(...) throws IOException ) > > > void deleteOnExit() > ---> public static final StandardOpenOption DELETE_ON_EXIT > (I've moved this to nio-discuss as that is the mailing list for feedback) Thank you for bringing up these topics. The summary answer is that they aren't present but they aren't forgotten either. You've suggested adding a DELETE_ON_EXIT option to the standard open options. I may be reading too much into this but did you mean DELETE_ON_CLOSE instead? A delete-on-close option may be useful but it's not supported on many operating systems and isn't really feasible to implement without support from the kernel (deleting the file after close isn't exactly the same thing as the file may have been replaced by an attacker and so we would delete the wrong file). If you mean the File deleteOnExit method then you are correct that this isn't an equivalent. Arguably we haven't really needed the delete on exit mechanism since the addition of the shutdown hooks in 1.3. The shutdown hooks allow for other types of tasks in addition to removing individual files before the VM shuts down. They can also be cancelled which is something that isn't (currently) possible with the delete mechanism. Another point is that with the provider interface there is potential for other types of file systems and it may be that you want is for the file to be registered for deletion when the FileSystem is closed (VM exit is too late). More generally you may have tasks to perform before the file system is closed. It would be good to get more feedback in this area so that we do the right thing. As regards temporary files: File.createFile(prefix,suffix).getFileRef() creates a temporary file and gives you an object in the new API to access that file. If you are instead asking about creating temporary file in a given FileSystem then we currently don't have a method to do that, unless of course you choose the temporary file name yourself and invoke createFile. The notion of "temporary directory" is traditionally something that is configured for the platform rather than a per-file system basis. It could be extended of course so feedback on this area is welcome too. -Alan.