Moving GPU offload policy into Java sources

[Doug Simon]

On Mar 11, 2014, at 4:58 PM, S. Bharadwaj Yadavalli <bharadwaj.yadavalli at oracle.com> wrote:

> Doug,
> 
> On 03/10/2014 11:58 AM, Doug Simon wrote:
>> So, I think we agree on the worthy goal of automatic GPU offload. I just think this is best done within a compilation. Assuming you still think the required analysis is best done outside of compilation, can you describe how it can be done (efficiently) and what mechanisms it would use?
> 
> I do not yet have the full algorithm / technique chalked out. GPU/non-host offload decision making at runtime is an area that I have been trying to experiment with and have been trying to understand in the context of a JVM. Roughly speaking, the idea is to recognize parallel application of lambda methods and offload such application to GPU - when deemed beneficial. More concretely, I am currently looking at the possibility of recognizing the characteristics of a stream pipeline by the VM runtime (assuming current Streams.parallel() - the parallel streams pipeline - implementation can be rendered for SIMD execution). I would like to see if I can use information such as size of data, composability of functions being applied in the pipeline (may be others, I do not know, yet) can be used to make the offload decision.

Thanks for the explanation. So, if I understand correctly, the decision making you describe would be done:

1. in modified library code (i.e., current Sumatra approach),
2. during compilation, or
3. class file loading

One place it should not be done is when deciding whether or not to compile a method as it would be too slow and would involve duplicating machinery used by one of the above. As such, we should still proceed with removing the GPU offloading decision logic from the logic of deciding whether or not to compile a method.

-Doug