graalCodeInstaller questions, recording scopes, etc.

[Doug Simon]

On Jan 6, 2014, at 7:19 PM, Deneau, Tom <tom.deneau at amd.com> wrote:

> Doug --
> 
> I was using --vm server which we always use.

Add the —vmbuild debug switch to both the build and vm command to see if an assertion fails. If that doesn’t reveal anything and you’re still stuck, send me a patch and mx command (and the hs_err log of your crash).

> By setting up the function, I meant doing a skeleton implementation of installKernel0.
> Below is the code for that (pretty much copied from installCode0 in graalCompilerToVM)
> 
> Some background:
>  * Before you had set up the virtual get_scope_value, etc, I had patched graal CodeInstaller to
>    skip the iteration thru values->length() if the compilationREsult was an ExternalCompilationResult.
>    (This was so we could set up a skeleton ScopeDesc with bci, etc but without the values mapping).
>    This worked fine.  Thru installKernel0 below, we are basically still using that patched graal CodeInstaller,
>    so I suspect something in installKernel0.
> 
> I'll also keep the sending the patch offer in mind.
> 
> -- Tom
> 
> 
> C2V_VMENTRY(jint, installKernel0, (JNIEnv *env, jobject, jobject compiled_code, jobject installed_code))
>  if (gpu::is_available() == false || gpu::has_gpu_linkage() == false && gpu::is_initialized()) {
>    tty->print_cr("installKernel0 - not available / no linkage / not initialized");
>    return 1;
>  }
> 
>  tty->print_cr("in installKernel0");
> 
> 
>  ResourceMark rm;
>  HandleMark hm;
>  Handle compiled_code_handle = JNIHandles::resolve(compiled_code);
>  CodeBlob* cb = NULL;
>  Handle installed_code_handle = JNIHandles::resolve(installed_code);
>  Handle triggered_deoptimizations_handle = JNIHandles::resolve(NULL);
> 
>  CodeInstaller installer = gpu::get_code_installer();
>  GraalEnv::CodeInstallResult result = installer.install(compiled_code_handle, cb, installed_code_handle, triggered_deoptimizations_handle);
> 
> 
>  if (PrintCodeCacheOnCompilation) {
>    stringStream s;
>    // Dump code cache  into a buffer before locking the tty,
>    {
>      MutexLockerEx mu(CodeCache_lock, Mutex::_no_safepoint_check_flag);
>      CodeCache::print_summary(&s, false);
>    }
>    ttyLocker ttyl;
>    tty->print_cr(s.as_string());
>  }
> 
>  if (result != GraalEnv::ok) {
>    assert(cb == NULL, "should be");
>  } else {
>    if (!installed_code_handle.is_null()) {
>      assert(installed_code_handle->is_a(HotSpotInstalledCode::klass()), "wrong type");
>      HotSpotInstalledCode::set_codeBlob(installed_code_handle, (jlong) cb);
>      oop comp_result = HotSpotCompiledCode::comp(compiled_code_handle);
>      if (comp_result->is_a(ExternalCompilationResult::klass())) {

comp_result will always be an ExternalCompilationResult so this should be an assertion.

>        if (TraceGPUInteraction) {
>          tty->print_cr("installKernel0: ExternalCompilationResult");
>        }
>        HotSpotInstalledCode::set_codeStart(installed_code_handle, ExternalCompilationResult::entryPoint(comp_result));
>      } else {
>        HotSpotInstalledCode::set_size(installed_code_handle, cb->size());
>        HotSpotInstalledCode::set_codeStart(installed_code_handle, (jlong) cb->code_begin());
>        HotSpotInstalledCode::set_codeSize(installed_code_handle, cb->code_size());
>      }
>      nmethod* nm = cb->as_nmethod_or_null();
>      assert(nm == NULL || !installed_code_handle->is_scavengable() || nm->on_scavenge_root_list(), "nm should be scavengable if installed_code is scavengable");
>    }
>  }
> 
>  return result;
> C2V_END

-Doug

> -----Original Message-----
>> From: Doug Simon [mailto:doug.simon at oracle.com]
>> Sent: Monday, January 06, 2014 12:06 PM
>> To: Deneau, Tom
>> Cc: graal-dev at openjdk.java.net
>> Subject: Re: graalCodeInstaller questions, recording scopes, etc.
>> 
>> 
>> On Jan 6, 2014, at 6:48 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
>> 
>>> While setting up the installKernel0 function, I see some sort of
>> memory corruption happening in hotspot.
>> 
>> What do you mean by "setting up" the function?
>> 
>>> (for example sometimes a ci_metadata field in ciObjectFactory will be
>>> 0xf1f1f1f1f1f1f1f1)
>> 
>> That's very strange since GraalVM by-passes ci altogether.
>> 
>>> What command line flags are good for tracking this down?
>> 
>> Did you try with a debug build (i.e. -vmbuild debug)? With some luck, an
>> assertion will fire somewhere.
>> 
>>> I tried -XX:NativeMemoryTracking=detail (had never used this before)
>> but that didn't flag anything.
>> 
>> Feel free to send me a patch and a mx command line that reproduces the
>> problem (I have okra installed and working on my Linux box so you can
>> rely on that if it simplifies things).
>> 
>> -Doug
>> 
>>>> -----Original Message-----
>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>> Sent: Thursday, January 02, 2014 5:17 PM
>>>> To: Deneau, Tom
>>>> Cc: graal-dev at openjdk.java.net
>>>> Subject: Re: graalCodeInstaller questions, recording scopes, etc.
>>>> 
>>>> 
>>>> On Jan 2, 2014, at 10:40 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
>>>> 
>>>>> See below...
>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Doug Simon [mailto:doug.simon at oracle.com]
>>>>>> Sent: Thursday, January 02, 2014 12:02 PM
>>>>>> To: Deneau, Tom
>>>>>> Cc: graal-dev at openjdk.java.net
>>>>>> Subject: Re: graalCodeInstaller questions, recording scopes, etc.
>>>>>> 
>>>>>> 
>>>>>> On Dec 20, 2013, at 3:41 PM, Deneau, Tom <tom.deneau at amd.com>
>> wrote:
>>>>>> 
>>>>>>> Closing the loop again, for my original questions, adding answers
>>>>>>> below that were relayed in the Skype call on Thursday...
>>>>>>> 
>>>>>>>> -----Original Message-----
>>>>>>>> From: Deneau, Tom
>>>>>>>> Sent: Monday, December 16, 2013 3:58 PM
>>>>>>>> To: graal-dev at openjdk.java.net
>>>>>>>> Subject: graalCodeInstaller questions, recording scopes, etc.
>>>>>>>> 
>>>>>> 
>>>>>> <snip>
>>>>>> 
>>>>>>> * Our infopoints have byteCodePositions.  The ones that get
>>>>>>>>   "recorded" go thru record_scope which has all sorts of
>>>>>>>>   host-specific checking on the scope info.  For instance,
>>>>>>>>   get_hotspot_value will compare a register # with
>>>>>>>>   RegisterImpl::number_of_registers (which refers to the host)
>>>> and
>>>>>>>>   if it is, assert that the type is FLOAT or DOUBLE.  None of
>>>> this
>>>>>>>>   holds with HSAIL registers.  What is the best way to resolve
>>>>>>>>   this?
>>>>>>>> 
>>>>>>> 
>>>>>>> This issue will be handled by the virtualization of some of the
>>>>>>> calls in graalCodeInstaller that Doug will be implementing.
>>>>>> 
>>>>>> I made the CodeInstaller class subclassable and the ScopeValue
>>>>>> creation methods virtual[1]. At some point, I imagine
>>>>>> CompilerToGPU.generateKernel() will be expanded to incorporate (and
>>>>>> obsolete) HotSpotCodeCacheProvider.addExternalMethod(). This means
>>>>>> the code for generateKernel in graalCompilerToGPU.cpp can use a GPU
>>>>>> specific subclass of CodeInstaller to create GPU ScopeValues.
>>>>>> 
>>>>>> -Doug
>>>>>> 
>>>>>> [1] http://hg.openjdk.java.net/graal/graal/rev/03bb0ee05409
>>>>> 
>>>>> Doug --
>>>>> 
>>>>> Just to make sure I understand how we should fit all this
>> together...
>>>>> 
>>>>> * HSAILCompilationResult currently calls
>>>>> providers.getCodeCache()).addExternalMethod
>>>>> 
>>>>> * We already have a HSAILHotSpotCodeCacheProvider so we could
>>>> override addExternalMethod with
>>>>>  our own version.
>>>>> 
>>>>> * in our own version of addExternalMethod, we could call our own
>>>> version of vm.installCodeHSAIL
>>>>>  (after adding that to the CompilerToVM interface, this seems
>> messy.
>>>> Maybe there should be
>>>>>   a installCode in the CompilerToGPU interface)
>>>>> 
>>>>> * installCodeHSAIL would have a line such as
>>>>>    HSAILCodeInstaller installer(compiled_code_handle, result, cb,
>>>>> installed_code_handle, triggered_deoptimizations_handle);
>>>>> 
>>>>>    where HSAILCodeInstaller extends CodeInstaller and uses the new
>>>>> virtual methods
>>>>> 
>>>>> 
>>>>> Or alternatively (and maybe this is what you are suggesting)
>>>>> * HSAILCompilationResult calls toGPU.generateKernel before it calls
>>>>> providers.getCodeCache()).addExternalMethod
>>>>> 
>>>>> * the logic for using HSAILCodeInstaller and the rest of the logic
>>>>> in
>>>> graalCompilerToVM.installCode0
>>>>>  could be added to gpu::hsail::generateKernel
>>>>> 
>>>>> * we would still need some way to get the InstalledCode back
>>>> (currently returned by addExternalMethod)
>>>>>  so we could store it in the HSAILCompilationResult
>>>> 
>>>> Yes, the latter is what I was suggesting. In CompilerToGPU, we'd
>>>> replace:
>>>> 
>>>>   long generateKernel(byte[] code, String name) throws
>>>> InvalidInstalledCodeException;
>>>> 
>>>> with:
>>>> 
>>>>   CodeInstallResult installKernel(HotSpotCompiledCode compiledCode,
>>>> HotSpotInstalledCode code);
>>>> 
>>>> The implementation of this in CompilerToGPUImpl would closely
>>>> resemble HotSpotCodeCacheProvider.addExternalMethod (which would be
>> removed).
>>>> 
>>>> I've prototyped this change and have attached it as a patch (feel
>>>> free to create and upload a webrev from it). You'd just need to make
>>>> the changes in graalCompilerToGPU.cpp to implement
>>>> CompilerToGPUImpl.installKernel0. Note that generateKernel is still a
>>>> separate method so that a GPU kernel compilation can be performed
>>>> without installing the result as an nmethod.
>>>> 
>>>> -Doug
>>> 
>>> 
>> 
> 
>