build concurrency

[Magnus Ihse Bursie]

On 2015-09-14 18:05, Erik Joelsson wrote:
> Hello,
>
> When I implemented the heuristic to choose a suitable default 
> concurrency, I only ever worried about the build. I think having tests 
> use the same concurrency setting must be a new feature? In any case, 
> it seems like there is a case for reducing concurrency when running 
> tests.

Just so we don't drop this part of the discussion. Maurizio, what do you 
think would be a good number for your machine when running tests?

/Magnus

>
> Another note. It at least used to be quite tricky to get correct 
> information about cores vs hyperthreading from the OS. I know today we 
> aren't even consistent with this across platforms. Perhaps we should 
> revisit this heuristic and take hyperthreading into consideration too.
>
> The current implemenation uses 100% of number of virtual cpus when 1 
> to 4 of them, then 90% at 5 to 16. After that it caps out at 16. (I 
> might remember some detail wrong here)
>
> /Erik
>
> On 2015-09-14 04:10, Maurizio Cimadamore wrote:
>> The information I posted was slightly incorrect, sorry - my machine 
>> has 8 cores (and 16 virtual processors) - so you see why choosing 
>> concurrency factor of 14 is particularly bad in this setup.
>>
>> Maurizio
>>
>> On 14/09/15 12:03, Maurizio Cimadamore wrote:
>>> Hi,
>>> I realized that the concurrency factor inferred by the JDK build 
>>> might be too high; on a 16 core machine, concurrency is set to 14 - 
>>> which then leads to absurd load averages (50-ish) when 
>>> building/running tests. High load when building is not a big issue, 
>>> but when running test this almost always turns into spurious 
>>> failures due to timeouts. I know I can override the concurrency 
>>> factor with --with-jobs - but I was curious as to why the default 
>>> parameter is set to such aggressive value?
>>>
>>> Thanks
>>> Maurizio
>>
>