non-foreign-call tlab refill from hsail

[Deneau, Tom]

Are snippets required to inline all their calls?  
Or alternatively is there no way to annotate that a method should not be inlined?

-- Tom


-----Original Message-----
From: Doug Simon [mailto:doug.simon at oracle.com] 
Sent: Thursday, November 07, 2013 4:15 PM
To: Deneau, Tom
Cc: graal-dev at openjdk.java.net; sumatra-dev at openjdk.java.net
Subject: Re: non-foreign-call tlab refill from hsail

Because it is slow (well, medium) path code that we don't want to inline at every allocation site.

On Nov 7, 2013, at 11:13 PM, Deneau, Tom <tom.deneau at amd.com> wrote:

> So I was trying to understand why the NewInstanceStub.newInstance Stub 
> code was not just included in the original NewObjectSnippet.allocateInstance snippet.
> 
> -- Tom
> 
> 
> -----Original Message-----
> From: Doug Simon [mailto:doug.simon at oracle.com]
> Sent: Thursday, November 07, 2013 4:04 PM
> To: Deneau, Tom
> Cc: graal-dev at openjdk.java.net; sumatra-dev at openjdk.java.net
> Subject: Re: non-foreign-call tlab refill from hsail
> 
> That is a very correct summary of the way it works!
> 
> On Nov 7, 2013, at 10:22 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
> 
>> Doug --
>> 
>> Trying to see if I understand how these pieces fit together.
>> 
>> NewObjectSnippets.allocateInstance makes a call to 
>> NewInstanceStubCall.call if the current tlab does not have enough 
>> room.
>> 
>> NewInstanceStubCall.call looks up the ForeignCallLinkage and finds 
>> that it is not a simple foreign call to a specific foreign call 
>> address (its address is 0) but instead has a stub associated with it.
>> I think this association came from the call to
>> 
>>       link(new NewInstanceStub(providers, target, 
>> registerStubCall(NEW_INSTANCE, REEXECUTABLE, NOT_LEAF, 
>> ANY_LOCATION)));
>> 
>> in HotSpotHostForeingCallsProvider.java
>> 
>> So when we try to finalizeAddress for the ForeignCallLinkage we end 
>> up compiling this stub.
>> 
>> The stub is a SnippetStub implemented with the snippet called 
>> "newInstance" in NewInstanceStub.java and tries to get a new tlab 
>> using CAS operations.  If this stub cannot get a new tlab it makes a 
>> "real" foreign call using
>>       newInstanceC(NEW_INSTANCE_C, thread(), hub);
>> 
>> which ends up going to the graalRuntime::new_instance
>> 
>> 
>> -----Original Message-----
>> From: Doug Simon [mailto:doug.simon at oracle.com]
>> Sent: Tuesday, October 22, 2013 4:42 AM
>> To: Deneau, Tom
>> Cc: graal-dev at openjdk.java.net; sumatra-dev at openjdk.java.net
>> Subject: Re: non-foreign-call tlab refill from hsail
>> 
>> 
>> On Oct 22, 2013, at 12:18 AM, "Deneau, Tom" <tom.deneau at amd.com> wrote:
>> 
>>> We are experimenting with object (and array) allocation from an HSA 
>>> device (using graal for
>>> 
>>> the HSAIL codegen).  Where we are now:
>>> 
>>> 
>>> 
>>> * the hsa workitems are using TLABs from "donor threads" who exist
>>> 
>>>   just to supply TLABs and don't do any allocation themselves.
>>> 
>>> 
>>> 
>>> * To reduce the number of donor threads required, a TLAB can be
>>> 
>>>   used by more than one workitem, in which case the workitems use
>>> 
>>>   HSAIL atomic_add instructions to bump the tlab top pointer.
>>> 
>>> 
>>> 
>>>    * the HSAIL backend has its own fastpath allocation snippets
>>> 
>>>      which generate the HSAIL atomic_add instructions which
>>> 
>>>      override the snippets in NewObjectSnippets.java
>>> 
>>> 
>>> 
>>> Some junit tests have been written and pass which allocate objects, or arrays of primitives, or arrays of objects.
>>> 
>>> 
>>> 
>>> All the above only works for the fastpath case, i.e., if there is indeed enough space in the donor TLAB.  We realize there are other cases:
>>> 
>>> 
>>> 
>>> a) not enough space in current TLAB but ability to allocate a new TLAB.
>>> 
>>> 
>>> 
>>> b) not able to allocate a new TLAB, GC required.
>>> 
>>> 
>>> 
>>> For only case a) above, we would like to experiment with grabbing 
>>> the new TLAB from HSAIL without making a "foreign call" to the VM.  
>>> From the hotspot code, I assume the logic required is what one sees 
>>> in
>>> 
>>>  mutableSpace::cas_allocate(size_t size) at least for the non-G1 case.
>> 
>> When the NewInstanceStub fails to allocate a new TLAB, it calls out to GraalRuntime::new_instance (in graalRuntime.cpp).
>> 
>>> Some of this non-foreign-call allocation logic appears to exist in 
>>> the Snippet called NewInstanceStub.newInstance (as opposed to 
>>> NewObjectSnippets.allocateInstance snippet which is what we are 
>>> currently overriding).  This comments for this snippet say
>>> 
>>> "Re-attempts allocation after an initial TLAB allocation failed or
>>> 
>>> was skipped (e.g., due to * -XX:-UseTLAB)."
>>> 
>>> 
>>> 
>>> Is this NewInstanceStub.newInstance snippet actually used anywhere in the current graal framework.
>> 
>> Yes, you can see a call to NewInstanceStubCall in NewObjectSnippets.allocateInstance.
>> 
>>> Is this a starting point we could use to get a non-foreign-call TLAB refill working?
>> 
>> Yes. Note that this call *is* a foreign call (see the javadoc for ForeignCallDescriptor).
>> 
>> -Doug
>> 
> 
> 
>