RFR 8181299/10, Several jdk tests fail with java.lang.NoClassDefFoundError: jdk/test/lib/process/StreamPumper

[Igor Ignatyev]

> On Jun 2, 2017, at 9:14 AM, Ioi Lam <ioi.lam at oracle.com> wrote:
> 
> On 6/2/17 8:44 AM, Ioi Lam wrote:
>> 
>> On 6/2/17 6:40 AM, Chris Hegarty wrote:
>>> On 02/06/17 00:14, Ioi Lam wrote:
>>>> ...
>>>> 
>>>> The gem is hidden in the compile.0.jta file. It contains something like:
>>>> 
>>>>      -sourcepath <blahblah>:/jdk/foobar/test/lib:<blahblah>
>>>> 
>>>> So if my test refers to a class under /test/lib, such as
>>>> jdk.test.lib.process.ProcessTools, javac will be able to locate it under
>>>> /jdk/foobar/test/lib/jdk/test/lib/process/ProcessTools.java, and will
>>>> build it automatically.
>>>> 
>>>> So really, there's no reason why the test must explicitly do an @build
>>>> of the library classes that it uses.
>>> 
>>> Sure, you're relying on the implicit compilation of dependencies
>>> by javac. Look at the output, where it compiles the library
>>> classes to.  It is part of the classes directory for the
>>> individual test. That means that the library classes will need
>>> to be compiled many many times.  The @build tag will compile
>>> the library classes to a common output directory, where they
>>> can be reused ( unless I'm missing something ).
>>> 
>>> -Chris.
>> Yes, @build will compile classes so that they can be reused. But why should it be the responsibility of every test to do this?
>> 
>> To reuse my malloc metaphore -- is it reasonable for every program that uses malloc to explicitly build libc?
>> 
>> By the way, jtreg arranges the output directory of the test by the directory they sit in, so
>> 
>>  jdk/test/foo/bar/XTest.java
>>  jdk/test/foo/bar/YTest.java
>> 
>> will all output their .class files to the same directory. Therefore, the amount of duplicated classes is not as bad as you might think. We've been omitting the @build tags in the hotspot tests and we haven't seen any problems.
>> 
>> - Ioi
> To avoid repeat compilation of the library classes, a more reasonable solution would be:
> 
> [1] Before test execution -- scan all the selected test to find all libraries specified by @library tags
> 
> [2] Fully compile all the libraries into their own output directories
> 
> [3] Then, start execution of the selected tests

unfortunately, it is not that simple, there are at least 2 problems w/ that approach:
 1. some of library classes have extra module dependency, e.g. jdk.test.lib.management.* depend on jdk.management module, ExtendedRobot (from jdk/test/testlibrary) depends on java.desktop. so compiling the whole library will require extra module dependency, which might be unneeded for the selected tests, as a result we won't be able to run these tests on configurations w/ limited module set.
 2. to make our tests packagefull, we had to add '@library /' to many hotspot/test/compiler tests, so we will have to compile all files from hotspot/test.

my take on all of this is that determination of output directory for classes is buggy, it uses directory of a @build or @run target to decide where put all produced classes files, but it should have mapping between source and destination paths instead, so all classes from jdk/test/foo/bar/ will go to a test scratch directory and all classes from /test/lib/ (assuming they are declared as @library) and /jdk/test/lib/ to different common directories which will be later added to classpath for the tests which use these libraries.

Thanks,
-- Igor