Executors enhancement

Christian Schulte cs at schulte.it
Tue Aug 9 22:51:18 UTC 2016


Am 08/09/16 um 19:04 schrieb ecki at zusammenkunft.net:
> This is nice in theory but has a number of drawbacks. First of all the
> common pool is not very easy to configure. It takes the available CPUs
> and if you run more than one VM you overcommit. Secondly different
> strategies to deal with longrunning or blocked jobs, with priority
> inheritance, with finegrained concurrency (you don't want to block all
> threads with some ill behaved tasks and therefore block all paralllel
> streams.
> 
>  
> 
> So it is a good idea to find a good shared default, but an API quite
> seldomly can isolate itself and the users about those aspects. So always
> allow to provide an alternate Executor. This also helps with thread
> naming, thread context, interceptors and error handlers.
> 
>  
> 
> Also, Once you have file or network or database jobs your task
> parallelity is not related with system CPUs anymore (and the runtimes
> become so unpredictable that it especially hurts parallelStream and and
> forkjoin users

That are mainly the reasons why I was looking for something making all
of this the burden of the OS scheduler. That is the only entity with the
capabilities to prioritize threads and to also know about what a
specific thread is currently doing (wait states, blocked by IO, network,
etc.). Maintaining executors at library or application context it would
be needed to query the OS for capabilities (query resource limits,
ulimit, etc.) and setup things matching the current OS in its current
state (multiple VMs) and to re-configure things when something changes.
So maybe the common pool should auto-configure itself dynamically based
on more than just the number of available processors and make use of the
OS resource management facilities.

Regards,
-- 
Christian



More information about the core-libs-dev mailing list