JEP 103: Parallel Array Sorting - proposal, reaction to Mr. Harned post

[Andrew Thompson]

Both JEP 103 and JEP 108 are interesting to me because they remind me of something I've discussed a couple of times with one of the engineers working on the JDK at Apple. (I'll not name him solely because his input was a casual 'that sounds interesting' and I don't want to imply he endorses this idea.).

The topic was Apple's Grand Central Dispatch and its applicability or lack thereof to Java programs. When we originally discussed it we noted that obviously the old Java idiom of doing 'new Thread()' directly in code is not remotely abstract enough for a vendor like Apple to be able to 'plug in' Grand Central (aka libdispatch) to do the scheduling.

Of course, Executor/ExecutorService/Executors is a different story - since an ExecutorService can be a simple thread pool, a ForkJoinPool, or indeed, as Apple later created, a Grand Central backed pool:

http://developer.apple.com/library/mac/documentation/Java/Reference/JavaSE6_AppleExtensionsRef/api/com/apple/concurrent/Dispatch.html

But, what we concluded was Java 6 didn't offer an API or an SPI in this area. The creation of 'thread pools' is a bit more abstract than 'new Thread()' but really still fairly concrete

ExecutorService myPool = Executors.newFixedThreadPool(5); // pretty clear what this means, create a pool with 5 new dedicated threads

What I imagined would look something more like:

ExecutorServer myCoolPool = Executors.defaultPlatformExecutorService(); //which on OSX might mean Grand Central dispatch (com.apple.concurrent.Dispatch) and something else on other platforms.

Accompanying this would be an SPI allowing the 'default' thread pool to be changed.

Well, JEP 103 has a similar idea embedded in it:

"In addition to the actual sorting API this proposal adds a default ForkJoinPool to the platform. Consequently both the sorting API and access to the default pool is proposed to be provided by the ForkJoinUtils class:

public final class ForkJoinUtils {

public static ForkJoinPool defaultFJPool() { ... }
..."

So is there any convergence between these ideas? Should we be thinking about adding a default ForkJoinPool to the platform, or should we be thinking about adding a default ExecutorService to the platform, which may or may not be a ForkJoinPool based on some clever logic the platform vendor could run based on # of cpus, memory, etc?

I think the idea of a platform ExecutorService is interesting and may warrant its own separate JEP, which I'd be happy to write (I was thinking of doing so anyway, but I also thought it might make sense to prototype something first).


On Oct 4, 2011, at 12:25 PM, David Holmes wrote:

> Hi Janda,
> 
> Thanks for the comments.
> 
> On 4/10/2011 7:48 PM, Janda Martin wrote:
>> 
>> I hope that this is correct mailing list to comment JEP 103.
> 
> It certainly is.
> 
>> Proposal: provide static methods for creating sort tasks. This allows developers to have full control over ForkJoinPool.
> 
> I'd personally prefer to pass in the FJP to the sort method, rather than expose a SortTask type. Though I do cringe at the thought of all the additional methods (I wonder if Coin2 would be open to suggesting we add default arguments to Java ...)
> 
>> There can be problem with one shared defaultFJPool() in multi-module applications (like Netbeans platform) when two modules requests different FJPool settings.
> 
> Although the JEP proposes properties to allow the configuration of the default FJPool, the expectation is that:
> 
> a) the majority of the time you will use the default configuration (desired parallelism level = number of 'processors')
> 
> b) any change to the default config is done as a "system" setting, not by "local logic". By which I mean that it is not expected that different "modules" will attempt to do this configuration. Perhaps we need a way to enforce this.
> 
> Personally I think there is room to have multiple FJPools in a system, but that makes most sense if the pools use disjoint sets of processors - which isn't currently supported in Java.
> 
> But this is  precisely the kind of feedback we are looking for, so thanks again.
> 
> David Holmes
> ------------
> 
>> public final class ForkJoinUtils {
>> 
>> // should this be replaced?
>> public static ForkJoinPool defaultFJPool() { ... }
>> 
>> public static byte[] parallelSort(byte[] a) { ... }
>> public static byte[] parallelSort(byte[] a, int fromIndex, int toIndex)
>>   {...}
>> 
>> // with this?
>> public static SortTask<byte[]>  createParallelSortTask(byte[] a) {...}
>> public static SortTask<byte[]>  createParallelSortTask(byte[] a, int fromIndex, int toIndex) {...}
>> ...
>> 
>> }
>> 
>> Reaction to Mr. Harned post
>> 
>> I work as a Java SE software developer for several years. I think that Mr. Harned opinion to freeze Java in software history ('30' years ago) is not good idea.
>> JSR 166 Fork/Join is great contribution to make Java better (thank you very much Doug Lea). Today CPUs just adds new cores so there have to be better support for parallelism.
>> 
>> j.u.c package isn't only about sorting long arrays (huge overhead). Programs do a lot more.
>> 
>> Thread, ThreadPool, FJPool and j.u.c are just small pieces in real application. They are solid and can be extended/controlled to support specific needs.
>> 
>> I think that Mr. Harned should improve his own libraries to comply with his objections. I looked at his source code, demos and I prefer j.u.c to his work.
>> 
>> Thank you everybody who works on Java
>> 
>> Sincerely
>> Martin JANDA
>> 
>> PS sorry for my english

AndyT (lordpixel - the cat who walks through walls)
A little bigger on the inside

	(see you later space cowboy, you can't take the sky from me)


AndyT (lordpixel - the cat who walks through walls)
A little bigger on the inside

	(see you later space cowboy, you can't take the sky from me)