RFR 8181299/10, Several jdk tests fail with java.lang.NoClassDefFoundError: jdk/test/lib/process/StreamPumper

[Igor Ignatyev]

just out of curiosity I have removed @build for all jdk.testlibary classes as well and run :tier1, it took approximately the same amount of time[1], the breakdown[2] shows that we spend 70 seconds more on build actions, which is not that big comparing to total execution time.

-- Igor

[1]
real	33m39.697s
user	112m5.722s
sys	6m26.973s
[2] 
compile: 903.9
build: 846.53
testng: 5515.98
junit: 674.429
shell: 885.103
main: 6973.12
clean: 0.001
driver: 9.125

> On Jun 2, 2017, at 6:45 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
> 
> I have measured how much time it takes to run :tier1 w/ and and w/o the fix which removes @build for jdk.test.lib.** classes[1-2]:
>  - w/o 8181391, i.e. w/ @build: real 33m4.624s, user 111m56.758s, sys 6m24.022s. [3] is a breakdown for jtreg actions
>  - w/ 8181391, i.e. w/o @build: real 32m17.259s, user 109m18.236s, sys 6m9.669s. [4] is a breakdown for jtreg actions
> as you can see there is no much difference in execution time, and the run w/o @build action was even a bit faster. the total time spend on build was lower. hence I'd say removing @build actions does not impact overall execution time. Even if it did, I don't think I'd prefer us to choose isolation and determinism over small performance improvements.
> 
> as Ioi and I stated before, removing @build actions did not help in all cases in hotspot. the root cause of this is having @run actions whose target is a class from library, this is identical to have explicit @build action for this class. if this class has dependency on other classes from testlibrary, you can get a testlibrary split into different locations and as a results NCDFE in runtime due to CODETOOLS-7901986. Fortunately, it is not the case for jdk tests, the only test library class which is used in @run is ClassFileInstaller, which does not have any dependencies. Therefore I think removing explicit @build is a more reliable and clearer way to work around current problems and it does not have a big drawback if any.
> 
> PS measurements were done on my mac 3.1 GHz Intel Core i7, 16 GB 1867 MHz DDR3, jtreg was run w/ "-conc:8 -agentvm"
> 
> [1] http://scaab055.us.oracle.com:9502/vmsqe/home/iignatye/webrev//8181391/webrev.00/index.html <http://scaab055.us.oracle.com:9502/vmsqe/home/iignatye/webrev//8181391/webrev.00/index.html>
> [2] https://bugs.openjdk.java.net/browse/JDK-8181391 <https://bugs.openjdk.java.net/browse/JDK-8181391>
> [3] 
> compile: 826.206
> build: 776.955
> testng: 5362.58
> junit: 640.705
> shell: 861.206
> main: 6823.19
> clean: 0.004
> driver: 6.578
> [4] 
> compile: 829.317
> build: 774.904
> testng: 5251
> junit: 648.888
> shell: 852.658
> main: 6686.99
> clean: 0.002
> driver: 5.973
> 
> Thanks,
> -- Igor
> 
>> On Jun 2, 2017, at 8:44 AM, Ioi Lam <ioi.lam at oracle.com <mailto:ioi.lam at oracle.com>> wrote:
>> 
>> On 6/2/17 6:40 AM, Chris Hegarty wrote:
>>> On 02/06/17 00:14, Ioi Lam wrote:
>>>> ...
>>>> 
>>>> The gem is hidden in the compile.0.jta file. It contains something like:
>>>> 
>>>>      -sourcepath <blahblah>:/jdk/foobar/test/lib:<blahblah>
>>>> 
>>>> So if my test refers to a class under /test/lib, such as
>>>> jdk.test.lib.process.ProcessTools, javac will be able to locate it under
>>>> /jdk/foobar/test/lib/jdk/test/lib/process/ProcessTools.java, and will
>>>> build it automatically.
>>>> 
>>>> So really, there's no reason why the test must explicitly do an @build
>>>> of the library classes that it uses.
>>> 
>>> Sure, you're relying on the implicit compilation of dependencies
>>> by javac. Look at the output, where it compiles the library
>>> classes to.  It is part of the classes directory for the
>>> individual test. That means that the library classes will need
>>> to be compiled many many times.  The @build tag will compile
>>> the library classes to a common output directory, where they
>>> can be reused ( unless I'm missing something ).
>>> 
>>> -Chris.
>> Yes, @build will compile classes so that they can be reused. But why should it be the responsibility of every test to do this?
>> 
>> To reuse my malloc metaphore -- is it reasonable for every program that uses malloc to explicitly build libc?
>> 
>> By the way, jtreg arranges the output directory of the test by the directory they sit in, so
>> 
>>  jdk/test/foo/bar/XTest.java
>>  jdk/test/foo/bar/YTest.java
>> 
>> will all output their .class files to the same directory. Therefore, the amount of duplicated classes is not as bad as you might think. We've been omitting the @build tags in the hotspot tests and we haven't seen any problems.
>> 
>> - Ioi
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/security-dev/attachments/20170605/1205102c/attachment.htm>