JTreg spawns java processes and uses all memory with CONCURRENCY=auto

[Jonathan Gibbons]

Iris,

Thanks for the feedback; I'll incorporate it later today.

The other section I thought to add as I was driving home last night was 
a section on writing tests with concurrency in mind. For example, we 
have one langtools test that can utilize all the processors by itself.  
I'll add some notes on that aspect.    That brings up a doc-design 
issue. There's a similar page I would like to write on choosing the best 
execution mode, and again, there are two potential audiences: those 
running jtreg, and those writing tests for jtreg. So the question is, 
how to organize notes like this, given the potentially different audiences?

-- Jon



On 05/21/2013 08:22 PM, Iris Clark wrote:
> Hi, Jon.
>
> I think that the new page looks great!
>
> Nit.  Looks like a space is missing between "othervmtest" in the "Execution Mode" section.
>
> Any recommendations for concurrency for the hotspot/ test suite?
>
> Also in the test suite section, I would say that you are referring to test suites in the JDK 8 and 7u Project forests. (I think that the same recommendations hold for both, but maybe not...)
>
> Thanks,
> iris
>
> -----Original Message-----
> From: Jonathan Gibbons
> Sent: Tuesday, May 21, 2013 7:47 PM
> To: jtreg-use at openjdk.java.net
> Subject: Re: JTreg spawns java processes and uses all memory with CONCURRENCY=auto
>
> I've published a new page on the OpenJDK jtreg pages that attempts to summarize the wisdom collected in this email thread.
>
> The page is http://openjdk.java.net/jtreg/concurrency.html
>
> The page is not yet linked in to other any pages, to give folk here a chance to give feedback before the page becomes more generally available.
>
> -- Jon
>
>
> On 05/21/2013 12:33 AM, Alan Bateman wrote:
>> On 20/05/2013 21:59, Tomasz Kowalczewski wrote:
>>> Hello,
>>>
>>> during a recent "Adopt OpenJDK" event held in Krakow I was using
>>> jtreg for running OpenJDK test cases. It was set up as per
>>> instructions published here:
>>> https://java.net/projects/adoptopenjdk/pages/InstallJtreg
>>>
>>> OpenJDK repository was cloned from http://hg.openjdk.java.net/jdk8/tl
>>> and jtreg version was jtreg-4.1-bin-b05_29_nov_2012.zip
>>>
>>> I have set 'CONCURRENCY=auto' environment variable.
>>>
>>> When running a test suite (.e.g.: 'cd jdk8_tl/test; make jdk_util')
>>> jtreg spawns a lot of java processes and it seems not to stop. Ever.
>>> It uses all 8 GB of RAM on my machine at which point Windows starts
>>> killing random processes.
>>>
>>> With 'CONCURRENCY=1' jtreg will spawn only a few java processes and
>>> will happily run all the test.
>>>
>>> What is the behavior for 'CONCURRENCY=auto'? Which value is used as
>>> default? Maybe it is not a bug its just that OpenJDK tests should be
>>> run on a big server machine with lots of RAM?
>>>
>>> Any suggestions or pointers to documentation will be very appreciated.
>>>
>>> --
>>> Regards,
>>> Tomasz Kowalczewski
>> If you are using -concurrency then I would also suggest adding
>> -vmoption:-Xmx512m so that you limit the heap size of the VMs. I think
>> Mike has a patch coming for the test Makefile to make this the default.
>>
>> On concurrency I personally find that -concurrency:auto works well for
>> the jdk tests, assuming you limit the heap size. The reason is that
>> the tests are a wild mix of work loads (some tests are I/O bound, some
>> are compute bound, some spend a lot of time sleeping). The exception
>> is Windows 32-bit where you can limit the heap but you might still run
>> into the virtual memory limits on the system.
>>
>> -Alan