non-foreign-call tlab refill from hsail

[Doug Simon]

On Oct 22, 2013, at 12:18 AM, "Deneau, Tom" <tom.deneau at amd.com> wrote:

> We are experimenting with object (and array) allocation from an HSA device (using graal for
> 
> the HSAIL codegen).  Where we are now:
> 
> 
> 
>   * the hsa workitems are using TLABs from "donor threads" who exist
> 
>     just to supply TLABs and don't do any allocation themselves.
> 
> 
> 
>   * To reduce the number of donor threads required, a TLAB can be
> 
>     used by more than one workitem, in which case the workitems use
> 
>     HSAIL atomic_add instructions to bump the tlab top pointer.
> 
> 
> 
>      * the HSAIL backend has its own fastpath allocation snippets
> 
>        which generate the HSAIL atomic_add instructions which
> 
>        override the snippets in NewObjectSnippets.java
> 
> 
> 
> Some junit tests have been written and pass which allocate objects, or arrays of primitives, or arrays of objects.
> 
> 
> 
> All the above only works for the fastpath case, i.e., if there is indeed enough space in the donor TLAB.  We realize there are other cases:
> 
> 
> 
>   a) not enough space in current TLAB but ability to allocate a new TLAB.
> 
> 
> 
>   b) not able to allocate a new TLAB, GC required.
> 
> 
> 
> For only case a) above, we would like to experiment with grabbing the new TLAB from HSAIL without making a "foreign call" to the VM.  From the hotspot code, I assume the logic required is what one sees in
> 
>    mutableSpace::cas_allocate(size_t size) at least for the non-G1 case.

When the NewInstanceStub fails to allocate a new TLAB, it calls out to GraalRuntime::new_instance (in graalRuntime.cpp).

> Some of this non-foreign-call allocation logic appears to exist in the Snippet called NewInstanceStub.newInstance (as opposed to NewObjectSnippets.allocateInstance snippet which is what we are currently overriding).  This comments for this snippet say
> 
>   "Re-attempts allocation after an initial TLAB allocation failed or
> 
>   was skipped (e.g., due to * -XX:-UseTLAB)."
> 
> 
> 
> Is this NewInstanceStub.newInstance snippet actually used anywhere in the current graal framework.

Yes, you can see a call to NewInstanceStubCall in NewObjectSnippets.allocateInstance.

>  Is this a starting point we could use to get a non-foreign-call TLAB refill working?

Yes. Note that this call *is* a foreign call (see the javadoc for ForeignCallDescriptor).

-Doug