[PATCH RFC 0/2] Add linux/ppc64 support for Hotspot serviceability agent to read core files

[Volker Simonis]

On Mon, Nov 17, 2014 at 6:59 PM, Maynard Johnson <maynardj at us.ibm.com> wrote:
> On 11/17/2014 10:20 AM, Volker Simonis wrote:
>> Hi Maynard,
>>
>> I'm currently looking at your changes. At first glance they look good.
>>
>> I could open a simple core file which contained both, interpreted and
>> compiled frames:
>>
>> $ jstack ./images/j2sdk-image/bin/java core.7034
>> ...
>> Thread 7035: (state = IN_VM)
>>  - sun.misc.Unsafe.putAddress(long, long) @bci=0 (Interpreted frame)
>>  - Crash.crashIt(sun.misc.Unsafe, int) @bci=10, line=8 (Interpreted frame)
>>  - Crash.doIt() @bci=45, line=23 (Compiled frame)
>>  - sun.reflect.NativeMethodAccessorImpl.invoke0(java.lang.reflect.Method,
>> java.lang.Object, java.lang.Object[]) @bci=0 (Interpreted frame)
>>  - sun.reflect.NativeMethodAccessorImpl.invoke(java.lang.Object,
>> java.lang.Object[]) @bci=100, line=62 (Interpreted frame)
>>  - sun.reflect.DelegatingMethodAccessorImpl.invoke(java.lang.Object,
>> java.lang.Object[]) @bci=6, line=43 (Interpreted frame)
>>  - java.lang.reflect.Method.invoke(java.lang.Object,
>> java.lang.Object[]) @bci=56, line=498 (Interpreted frame)
>>  - Crash.main(java.lang.String[]) @bci=32, line=31 (Interpreted frame)
>>
>> The one thing that doesn't currently work is "jstack -m" (i.e. "mixed
>> mode" for java and native frames). Are you aware of this?
> Hi, Volker,
> Yeah, I knew about this problem and forgot to mention it in my patch posting. I started
> looking at it this morning, and so far, I have at least fixed the UnmappedAddressException.
> But now I'm getting different results on little endian vs big endian ppc64 systems.
> On BE, I either get no symbol names (i.e., "?????") or wrong symbol names.  On LE,
> I seem to get correct symbol names for the first symbol (either __pthread_cond_wait
> or __pthread_cond_timedwait) and the last symbol (start_thread) of each stack, but
> everything in between is "?????".
>

Maybe this is related to the fact that we have function descriptors on
BE and simple function pointers on LE. You may have a look at the
elf-decoder for ppc64 to find some more information.

>>
>> Regarding your "test.java" example - how do you use it?
>>
>> If I just attach with jstack to the Java process which runs
>> "test.java" I get the correct stack trace of all threads. But I think
>> that's actual no SA-functionality but a VM-feature (the same that can
>> be triggered by sending kill -SIGQUIT to  java process).
>>
>> If I attach with "jstack -F" I see the problems you mentioned. First I
>> didn't saw any frame at all which confused me but then I also saw the
>> two cases mentioned by you. I'll need to have a closer look what
>> happens.
>
> I was just running the 'test' java app and, in another session, killing it with SIGSEGV.
> To be honest, I wasn't aware of the 'jstack -F' option.
>

Another possibility I've just found out is to create a core from gdb
with the 'generate-core-file' command. You can than still inspect the
original program in gdb while debugging how jstack is working on the
core file.

> -Maynard
>
>>
>> Regards,
>> Volker
>>
>>
>>
>> On Fri, Nov 14, 2014 at 7:09 PM, Maynard Johnson <maynardj at us.ibm.com> wrote:
>>> When Hotspot SA tools jmap, jstack, and jsadebugd are run against a core file, they fail with the following runtime exception:
>>>
>>>      OS/CPU combination linux/ppc64 not yet supported
>>>
>>> I will post a patch set that adds this support.  The patch set consists of the following patches:
>>>
>>> PATCH 1/2: Updates to non-Java files to support linux/ppc64 Hotspot SA with core files
>>>
>>> PATCH 2/2: New PPC64 class files (and updates to generic files) to support linux/ppc64 Hotspot SA with core files
>>>
>>> These two patches apply cleanly to a November 13 pull of the jdk9-dev upstream sources.
>>>
>>> ------------
>>> Open issues:
>>> ------------
>>>   1) The jstack tool does not print a stack entry for the 'main()' method of the Java
>>>      workload (attached) under test.  For example:
>>>
>>>      (Note:  Addresses and method signatures elided for brevity.)
>>>
>>>        Thread 24358: (state = IN_JAVA, current Java SP = null
>>>        )
>>>         - java.lang.String.getChars(...) @bci=58, line=814, pc=..., Method*=... (Compiled frame; ... imprecise)
>>>         - test.run_test() @bci=80, line=33, pc=..., Method*=... (Compiled frame)
>>>      ==> (Expect an entry for test.main() here)
>>>
>>>   2) The jstack tool sometimes prints what appears to be two complete stacks for the Java workload. For example:
>>>
>>>        Thread 24779: (state = IN_JAVA, current Java SP = null
>>>        )
>>>         - java.lang.String.getChars(...) @bci=58, line=814, pc=..., Method*=... (Compiled frame; ... imprecise)
>>>         - test.run_test() @bci=80, line=33, pc=..., Method*=... (Compiled frame)
>>>         - test.get_my_chars(...) @bci=39, line=15, pc=..., Method*=... (Compiled frame)
>>>         - test.run_test() @bci=92, line=34, pc=..., Method*=... (Compiled frame)
>>>
>>>        Again, the 'test.main' method is missing, but there's also the anomaly of the
>>>        test.run_test' method showing up twice in the stack, implying that it is called
>>>        by 'test.get_my_chars' at line 15.  But that that is not accurate. In fact, run_test
>>>        does call String.getChars at line 33 *and* it calls test.get_my_chars at line 34 --
>>>        but these are totally distinct call graphs.  Somehow, we are seeing these two distinct
>>>        stacks in the core file, which seems impossible.
>>>
>>> ---------
>>>
>>> Any help offered on these two open issues would be greatly appreciated.
>>>
>>> -Maynard
>>
>