network tests

[Joe Darcy]

Alan Bateman wrote:
> Joe Darcy wrote:
>> Greetings.
>>
>> Recently Mark Wielaard started an effort to run the regression tests 
>> on OpenJDK builds and publish the results 
>> (http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2008-May/001921.html).  
>> Certain tests in the nio and networking area assume Sun-internal 
>> hosts  like javaweb are available; the shell test
>>
>>    test/java/net/InetAddress/ptr/lookup.sh
>>
>> has a string for javaweb and
>>
>>    test/java/nio/channels/TestUtil.java
>>
>> has the following constants used by other tests:
>>
>>    // Test hosts used by the channels tests - change these when
>>    // executing in a different network.
>>    public static final String HOST = "javaweb.sfbay.sun.com";
>>    public static final String REFUSING_HOST = "jano.sfbay.sun.com";
>>    public static final String FAR_HOST = "theclub.ireland.sun.com";
>>    public static final String UNRESOLVABLE_HOST = 
>> "blah-blah.blah-blah.blah";
>>
>> Mark observed these three hosts are needed:
>>> With daytime and echo available. The only wrinkle is that some tests
>>> expect to be able to use quick timeouts from HOST, but to need long
>>> timeouts from FAR_HOST. I think we could provide something public for
>>> this. But I might have missed some tests that need other services. Do
>>> you have a full overview?
>> (http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2008-May/001972.html) 
>>
>>
>> First, is this all the networking configuration information that is 
>> used in the regression tests?  
> For the tests in the java/nio and sun/nio tree then the dependencies 
> are centralized in TestUtil.java. When we moved these tests from 
> closed->open we did talk about moving this configuration into a 
> properties/configuration file but figured there wasn't much difference 
> between editing a config file vs. a source file.

I think being able to pass in your own config file would be useful for 
this use case; it is awkward to have to edit the source to the tests to 
get them to pass :-)  If -Dnetworking.test.config.file were defined, the 
test could use those values instead of the defaults...

>
> For the classic networking, the dependencies are scattered. I believe 
> some of the tests weren't moved from closed->open for this reason. 
> Some of the tests uses an embedded HTTP or FTP server but there are 
> others that depend on DNS, proxy, etc. Michael or Jessie may have more 
> to say on this.

Could the classic networking tests be refactored a bit to be centralized 
like the nio tests?  Would it be natural for classic networking and nio 
to get their host information from the same place?

>
> BTW: I briefly scanned the summary and most of the java/nio failures 
> appear to be *HOST related. Now that folks are running these tests 
> outside Sun it is time to own up to a few intermittent/timing related 
> failures with 4 or 5 of these tests. I've been meaning to fix these 
> test issues for a while but I haven't had the cycles. I'll dig up my 
> notes on these. One other thing - I notice 
> test/java/nio/channels/DatagramChannel/NotBound.java on the list - 
> that test is failing because it is missing the public modifier. The 
> test is bogus anyway so it might be best to just remove this test from 
> jdk6 and open6. For jdk7/OpenJDK I have spec clarification in the 
> works to fix the spec hole and as part of that work the test will be 
> replaced (see 6621689).

Yes, fixing the intermittent failures and removing the invalid test in 
OpenJDK 6 sounds good.

>> Second, it would be helpful is this host information could be 
>> configured without changing the sources of the test each time.  I've 
>> spoken briefly to Jon about ways this kind of information can be 
>> passed into a jtreg run.  Environment variables can be used, as can 
>> system properties.  A system property could also be used to specify a 
>> file from which the information was read.  With a bit of jtreg 
>> hacking, it is feasible portions of the jtharness interview procedure 
>> could be exposed to configure information that way too.
> I would be concerned with a solution that requires environment 
> variables or jtreg requiring configuration from somewhere other than 
> the test suite. At least in Sun (sorry for being selfish), these tests 
> are run by various teams on hundreds of machines. I don't think we'd 
> want to require the javatest interview be run or have .jti files 
> stored in various places with configuration information that will 
> quickly go stale. If you mean that the configuration can be overridden 
> via some input to jtreg that might be okay but would require the 
> distributions and others to maintain that configuration. Anyway, I'm 
> interested to hear the outcome of the discussion.

Even if the Sun-internal settings remain baked in, I think an easy way 
to  pass in a limited number of configuration settings is appropriate 
and shouldn't be too hard to accommodate in the test code.  Presumably 
an interface like "openjtreg -Dmy.config.options=foo ..." wouldn't be 
too awkward to use.

-Joe