JTreg spawns java processes and uses all memory with CONCURRENCY=auto

[Iris Clark]

Hi, Jon.

I think that the new page looks great!

Nit.  Looks like a space is missing between "othervmtest" in the "Execution Mode" section.

Any recommendations for concurrency for the hotspot/ test suite?  

Also in the test suite section, I would say that you are referring to test suites in the JDK 8 and 7u Project forests. (I think that the same recommendations hold for both, but maybe not...)

Thanks,
iris

-----Original Message-----
From: Jonathan Gibbons 
Sent: Tuesday, May 21, 2013 7:47 PM
To: jtreg-use at openjdk.java.net
Subject: Re: JTreg spawns java processes and uses all memory with CONCURRENCY=auto

I've published a new page on the OpenJDK jtreg pages that attempts to summarize the wisdom collected in this email thread.

The page is http://openjdk.java.net/jtreg/concurrency.html

The page is not yet linked in to other any pages, to give folk here a chance to give feedback before the page becomes more generally available.

-- Jon


On 05/21/2013 12:33 AM, Alan Bateman wrote:
> On 20/05/2013 21:59, Tomasz Kowalczewski wrote:
>> Hello,
>>
>> during a recent "Adopt OpenJDK" event held in Krakow I was using 
>> jtreg for running OpenJDK test cases. It was set up as per 
>> instructions published here:
>> https://java.net/projects/adoptopenjdk/pages/InstallJtreg
>>
>> OpenJDK repository was cloned from http://hg.openjdk.java.net/jdk8/tl
>> and jtreg version was jtreg-4.1-bin-b05_29_nov_2012.zip
>>
>> I have set 'CONCURRENCY=auto' environment variable.
>>
>> When running a test suite (.e.g.: 'cd jdk8_tl/test; make jdk_util') 
>> jtreg spawns a lot of java processes and it seems not to stop. Ever.
>> It uses all 8 GB of RAM on my machine at which point Windows starts 
>> killing random processes.
>>
>> With 'CONCURRENCY=1' jtreg will spawn only a few java processes and 
>> will happily run all the test.
>>
>> What is the behavior for 'CONCURRENCY=auto'? Which value is used as 
>> default? Maybe it is not a bug its just that OpenJDK tests should be 
>> run on a big server machine with lots of RAM?
>>
>> Any suggestions or pointers to documentation will be very appreciated.
>>
>> --
>> Regards,
>> Tomasz Kowalczewski
> If you are using -concurrency then I would also suggest adding 
> -vmoption:-Xmx512m so that you limit the heap size of the VMs. I think 
> Mike has a patch coming for the test Makefile to make this the default.
>
> On concurrency I personally find that -concurrency:auto works well for 
> the jdk tests, assuming you limit the heap size. The reason is that 
> the tests are a wild mix of work loads (some tests are I/O bound, some 
> are compute bound, some spend a lot of time sleeping). The exception 
> is Windows 32-bit where you can limit the heap but you might still run 
> into the virtual memory limits on the system.
>
> -Alan