RFR 8181299/10, Several jdk tests fail with java.lang.NoClassDefFoundError: jdk/test/lib/process/StreamPumper

[Igor Ignatyev]

I have measured how much time it takes to run :tier1 w/ and and w/o the fix which removes @build for jdk.test.lib.** classes[1-2]:
 - w/o 8181391, i.e. w/ @build: real 33m4.624s, user 111m56.758s, sys 6m24.022s. [3] is a breakdown for jtreg actions
 - w/ 8181391, i.e. w/o @build: real 32m17.259s, user 109m18.236s, sys 6m9.669s. [4] is a breakdown for jtreg actions
as you can see there is no much difference in execution time, and the run w/o @build action was even a bit faster. the total time spend on build was lower. hence I'd say removing @build actions does not impact overall execution time. Even if it did, I don't think I'd prefer us to choose isolation and determinism over small performance improvements.

as Ioi and I stated before, removing @build actions did not help in all cases in hotspot. the root cause of this is having @run actions whose target is a class from library, this is identical to have explicit @build action for this class. if this class has dependency on other classes from testlibrary, you can get a testlibrary split into different locations and as a results NCDFE in runtime due to CODETOOLS-7901986. Fortunately, it is not the case for jdk tests, the only test library class which is used in @run is ClassFileInstaller, which does not have any dependencies. Therefore I think removing explicit @build is a more reliable and clearer way to work around current problems and it does not have a big drawback if any.

PS measurements were done on my mac 3.1 GHz Intel Core i7, 16 GB 1867 MHz DDR3, jtreg was run w/ "-conc:8 -agentvm"

[1] http://scaab055.us.oracle.com:9502/vmsqe/home/iignatye/webrev//8181391/webrev.00/index.html <http://scaab055.us.oracle.com:9502/vmsqe/home/iignatye/webrev//8181391/webrev.00/index.html>
[2] https://bugs.openjdk.java.net/browse/JDK-8181391
[3] 
compile: 826.206
build: 776.955
testng: 5362.58
junit: 640.705
shell: 861.206
main: 6823.19
clean: 0.004
driver: 6.578
[4] 
compile: 829.317
build: 774.904
testng: 5251
junit: 648.888
shell: 852.658
main: 6686.99
clean: 0.002
driver: 5.973

Thanks,
-- Igor

> On Jun 2, 2017, at 8:44 AM, Ioi Lam <ioi.lam at oracle.com> wrote:
> 
> On 6/2/17 6:40 AM, Chris Hegarty wrote:
>> On 02/06/17 00:14, Ioi Lam wrote:
>>> ...
>>> 
>>> The gem is hidden in the compile.0.jta file. It contains something like:
>>> 
>>>      -sourcepath <blahblah>:/jdk/foobar/test/lib:<blahblah>
>>> 
>>> So if my test refers to a class under /test/lib, such as
>>> jdk.test.lib.process.ProcessTools, javac will be able to locate it under
>>> /jdk/foobar/test/lib/jdk/test/lib/process/ProcessTools.java, and will
>>> build it automatically.
>>> 
>>> So really, there's no reason why the test must explicitly do an @build
>>> of the library classes that it uses.
>> 
>> Sure, you're relying on the implicit compilation of dependencies
>> by javac. Look at the output, where it compiles the library
>> classes to.  It is part of the classes directory for the
>> individual test. That means that the library classes will need
>> to be compiled many many times.  The @build tag will compile
>> the library classes to a common output directory, where they
>> can be reused ( unless I'm missing something ).
>> 
>> -Chris.
> Yes, @build will compile classes so that they can be reused. But why should it be the responsibility of every test to do this?
> 
> To reuse my malloc metaphore -- is it reasonable for every program that uses malloc to explicitly build libc?
> 
> By the way, jtreg arranges the output directory of the test by the directory they sit in, so
> 
>  jdk/test/foo/bar/XTest.java
>  jdk/test/foo/bar/YTest.java
> 
> will all output their .class files to the same directory. Therefore, the amount of duplicated classes is not as bad as you might think. We've been omitting the @build tags in the hotspot tests and we haven't seen any problems.
> 
> - Ioi

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/security-dev/attachments/20170602/54d5ebd8/attachment.htm>