non-foreign-call tlab refill from hsail

[Doug Simon]

That is a very correct summary of the way it works!

On Nov 7, 2013, at 10:22 PM, Deneau, Tom <tom.deneau at amd.com> wrote:

> Doug --
> 
> Trying to see if I understand how these pieces fit together.
> 
> NewObjectSnippets.allocateInstance makes a call to
> NewInstanceStubCall.call if the current tlab does not have enough
> room.
> 
> NewInstanceStubCall.call looks up the ForeignCallLinkage and finds
> that it is not a simple foreign call to a specific foreign call
> address (its address is 0) but instead has a stub associated with it.
> I think this association came from the call to
> 
>        link(new NewInstanceStub(providers, target, registerStubCall(NEW_INSTANCE, REEXECUTABLE, NOT_LEAF, ANY_LOCATION)));
> 
> in HotSpotHostForeingCallsProvider.java
> 
> So when we try to finalizeAddress for the ForeignCallLinkage we end up
> compiling this stub.
> 
> The stub is a SnippetStub implemented with the snippet called
> "newInstance" in NewInstanceStub.java and tries to get a new tlab
> using CAS operations.  If this stub cannot get a new tlab it makes a
> "real" foreign call using
>        newInstanceC(NEW_INSTANCE_C, thread(), hub);
> 
> which ends up going to the graalRuntime::new_instance
> 
> 
> -----Original Message-----
> From: Doug Simon [mailto:doug.simon at oracle.com] 
> Sent: Tuesday, October 22, 2013 4:42 AM
> To: Deneau, Tom
> Cc: graal-dev at openjdk.java.net; sumatra-dev at openjdk.java.net
> Subject: Re: non-foreign-call tlab refill from hsail
> 
> 
> On Oct 22, 2013, at 12:18 AM, "Deneau, Tom" <tom.deneau at amd.com> wrote:
> 
>> We are experimenting with object (and array) allocation from an HSA device (using graal for
>> 
>> the HSAIL codegen).  Where we are now:
>> 
>> 
>> 
>>  * the hsa workitems are using TLABs from "donor threads" who exist
>> 
>>    just to supply TLABs and don't do any allocation themselves.
>> 
>> 
>> 
>>  * To reduce the number of donor threads required, a TLAB can be
>> 
>>    used by more than one workitem, in which case the workitems use
>> 
>>    HSAIL atomic_add instructions to bump the tlab top pointer.
>> 
>> 
>> 
>>     * the HSAIL backend has its own fastpath allocation snippets
>> 
>>       which generate the HSAIL atomic_add instructions which
>> 
>>       override the snippets in NewObjectSnippets.java
>> 
>> 
>> 
>> Some junit tests have been written and pass which allocate objects, or arrays of primitives, or arrays of objects.
>> 
>> 
>> 
>> All the above only works for the fastpath case, i.e., if there is indeed enough space in the donor TLAB.  We realize there are other cases:
>> 
>> 
>> 
>>  a) not enough space in current TLAB but ability to allocate a new TLAB.
>> 
>> 
>> 
>>  b) not able to allocate a new TLAB, GC required.
>> 
>> 
>> 
>> For only case a) above, we would like to experiment with grabbing the new TLAB from HSAIL without making a "foreign call" to the VM.  From the hotspot code, I assume the logic required is what one sees in
>> 
>>   mutableSpace::cas_allocate(size_t size) at least for the non-G1 case.
> 
> When the NewInstanceStub fails to allocate a new TLAB, it calls out to GraalRuntime::new_instance (in graalRuntime.cpp).
> 
>> Some of this non-foreign-call allocation logic appears to exist in the Snippet called NewInstanceStub.newInstance (as opposed to NewObjectSnippets.allocateInstance snippet which is what we are currently overriding).  This comments for this snippet say
>> 
>>  "Re-attempts allocation after an initial TLAB allocation failed or
>> 
>>  was skipped (e.g., due to * -XX:-UseTLAB)."
>> 
>> 
>> 
>> Is this NewInstanceStub.newInstance snippet actually used anywhere in the current graal framework.
> 
> Yes, you can see a call to NewInstanceStubCall in NewObjectSnippets.allocateInstance.
> 
>> Is this a starting point we could use to get a non-foreign-call TLAB refill working?
> 
> Yes. Note that this call *is* a foreign call (see the javadoc for ForeignCallDescriptor).
> 
> -Doug
>