build concurrency

[Erik Joelsson]

Hello,

When I implemented the heuristic to choose a suitable default 
concurrency, I only ever worried about the build. I think having tests 
use the same concurrency setting must be a new feature? In any case, it 
seems like there is a case for reducing concurrency when running tests.

Another note. It at least used to be quite tricky to get correct 
information about cores vs hyperthreading from the OS. I know today we 
aren't even consistent with this across platforms. Perhaps we should 
revisit this heuristic and take hyperthreading into consideration too.

The current implemenation uses 100% of number of virtual cpus when 1 to 
4 of them, then 90% at 5 to 16. After that it caps out at 16. (I might 
remember some detail wrong here)

/Erik

On 2015-09-14 04:10, Maurizio Cimadamore wrote:
> The information I posted was slightly incorrect, sorry - my machine 
> has 8 cores (and 16 virtual processors) - so you see why choosing 
> concurrency factor of 14 is particularly bad in this setup.
>
> Maurizio
>
> On 14/09/15 12:03, Maurizio Cimadamore wrote:
>> Hi,
>> I realized that the concurrency factor inferred by the JDK build 
>> might be too high; on a 16 core machine, concurrency is set to 14 - 
>> which then leads to absurd load averages (50-ish) when 
>> building/running tests. High load when building is not a big issue, 
>> but when running test this almost always turns into spurious failures 
>> due to timeouts. I know I can override the concurrency factor with 
>> --with-jobs - but I was curious as to why the default parameter is 
>> set to such aggressive value?
>>
>> Thanks
>> Maurizio
>