RFR: 8020962: dump loaded java classes when vm crash

[Yumin Qi]

Ioi,

On 8/12/2013 4:14 PM, Ioi Lam wrote:
> |Yumin,||
> ||
> ||If I understand correctly, the reason of dumping the classes in a 
> separate process using SA is to avoid crashing (in case the system 
> dictionary is corrupt).||
> ||
> |
|Yes.|
> |||But you're already exec'ing SA as the very last step of writing to 
> hs_err. Maybe instead of exec SA, just dump the system dict using C 
> code within the crashing process (SystemDictionary::print()). ||
> ||
> ||If the sys dict is corrupt you will encounter a secondary crash 
> here, but that's OK, because ||hs_err_pid*.log already has ||all the 
> information that we used to have, so you are not making it any worse. 
> It may actually be helpful because now you know for sure that the sys 
> dict is corrupt. This is one piece of information that we didn't have.
>
> Also, if the sys dict is corrupt, I doubt SA can produce more 
> meaningful data than |||SystemDictionary::print()|. Instead, it may be 
> better to have a |
> |||||SystemDictionary::print_safely() function that makes more error 
> checking|| so it can better handle a partially corrupted sys dict.
>
> |
|SA will throw exception on encounter corrupted data in sys dictionary.
Let me think about your suggestion which I think is a good way to make 
it work.

Thanks
Yumin
|
> |- Ioi
> ||
> ||
> ||Since you're doing the dump of classes as the very last step of to 
> hs_err, instead of spawning another JVM ||
> ||On 08/12/2013 03:50 PM, Christian Tornqvist wrote:||
> |
>> |I agree with Coleen, this change should not be added to the JVM.
>>
>> Though I'd like to have the tool to create this archive from a core/mdmp. If this is not already present in any of the j*-tools we might want to look into adding it, please work with the serviceability team to determine where this would be best suited.
>>
>> Thanks,
>> Christian
>>
>> -----Original Message-----
>> From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com]
>> Sent: Monday, August 12, 2013 5:44 PM
>> To: Yumin Qi
>> Cc: Christian Tornqvist;serviceability-dev at openjdk.java.net  serviceability-dev at openjdk.java.net; 'hotspot compiler'; 'David Holmes';hotspot-runtime-dev at openjdk.java.net
>> Subject: Re: RFR: 8020962: dump loaded java classes when vm crash
>>
>>
>> Yumin,
>>
>> I don't think this change should be added to the JVM for the following reasons.
>>
>> 1. Error handling should only contain safe actions.   We have concerns
>> that the SA is not that stable and would prevent getting a real core
>> file in many error situations.   You couldn't have tested all error
>> situations because these are usually the things we don't know about.
>> You also mention that there are currently bugs preventing this feature from working in your first email.
>>
>> 2. I am not convinced that having a separate jar file with loaded classes (is it a list that SA generates?) would be collected by an error
>> reporter.   If it's collected, I don't see how helpful it would be.
>> Also a customer may not want their classes disclosed in error reporting.
>>
>> 3.  This doesn't seem important enough to include as a new feature in
>> jdk8 release, which is feature-complete.   I don't see a customer
>> request for it.
>>
>> Coleen
>>
>> On 08/12/2013 01:00 PM, Yumin Qi wrote:
>> |
>>> |Chris, David
>>>
>>>    Yes. This can be after crash with core file, but some time no core
>>> file generated (also if the error could not repeat, we will never got
>>> information again), so dumping  target process is also a choice. To
>>> avoid more confusion, this step was launched at the very last moment,
>>> just before raise abort to exit. At this moment, if client process
>>> could not attach to the target process it will exit right away.
>>>
>>> Answers to David:
>>>
>>> Note that the SA is only present in a full JDK, not a JRE (full or
>>> compact profile).
>>>    Yes, in code, if no sa-jdi.jar found, skip fork.
>>>
>>> - What mechanism will the SA try to use to query the VM?
>>>
>>>    After successfully attached to target process, SA will read via proc
>>> APIs and vmStructs (named TYPEDB) to iterate  memory of system
>>> dictionary read (only)  from target process to dump class information.
>>>
>>> - What if the state of the crashed VM stops the SA from being able to
>>> attach properly (ie both processes hang)?
>>>
>>>    That should be an attaching API problem. We haven't watched such
>>> case happened so far.
>>>
>>> - What if it the SA also crashes, will it launch a third VM then a
>>> fourth etc?
>>>
>>>    Definitely don't want to see this happened in a chain. The solution
>>> may use a property such as
>>> sun.jvm.hotspot.DumpLoadedClasses.dumpingInProcess=true to pass into
>>> SA process, at launching call, check if the property set, if set, do
>>> not fork. When SA process died, it will generate core file first, note
>>> the target process still waiting for its exit, so when target exit,
>>> the core file (if both use default core as name) will be override by
>>> target. The SA process will only leave a hs_err_pid*.log file. (? read
>>> such property in handler is possible?)
>>>
>>>   Also what is the nature of this dump? How big is it? Where will it go?
>>>   The jars includes *app.jar and *boot.jar, the later usually can be
>>> ignored (rt.jar etc system jars) unless it's rewritten by JVMTI agent.
>>> The app.jar will contain all classes by customer, so we can do
>>> whatever we can to the jar.
>>>
>>>
>>> Thanks
>>> Yumin
>>>
>>>
>>> On 8/12/2013 5:51 AM, Christian Tornqvist wrote:
>>> |
>>>> |Hi Yumin,
>>>>
>>>> The idea is to do as little as possible in the VM error handler,
>>>> since we've crashed for some reason we don't know what state the
>>>> process is in and we have to be extremely careful in when we're
>>>> gathering the information. This seems like a step that is risky for
>>>> all of the reasons David mentioned below.
>>>>
>>>> It's also information that can easily be extracted post-mortem from
>>>> the core/mdmp using the method you described for OSX, so gathering
>>>> this at the time of a crash seems like an unnecessary risk.
>>>>
>>>> Thanks,
>>>> Christian
>>>>
>>>> -----Original Message-----
>>>> From:hotspot-compiler-dev-bounces at openjdk.java.net
>>>> [mailto:hotspot-compiler-dev-bounces at openjdk.java.net] On Behalf Of
>>>> David Holmes
>>>> Sent: Monday, August 12, 2013 1:56 AM
>>>> To: Yumin Qi
>>>> Cc: hotspot compiler;hotspot-runtime-dev at openjdk.java.net
>>>> Subject: Re: RFR: 8020962: dump loaded java classes when vm crash
>>>>
>>>> Hi Yumin,
>>>>
>>>> Note that the SA is only present in a full JDK, not a JRE (full or
>>>> compact profile).
>>>>
>>>> I have quite a few concerns with this:
>>>>
>>>> We already do a lot of things that are not valid to be done from a
>>>> signal handling context but this really takes that to an extreme.
>>>> Doing fork-exec from a signal handler seems like a recipe for
>>>> disaster (Note the existing onError facility is typically used for
>>>> synchronous failures.)
>>>>
>>>> The idea of launching a second VM to try and query a VM that has
>>>> crashed also seems somewhat problematic:
>>>> - What mechanism will the SA try to use to query the VM?
>>>> - What if the state of the crashed VM stops the SA from being able to
>>>> attach properly (ie both processes hang)?
>>>> - What if it the SA also crashes, will it launch a third VM then a
>>>> fourth etc?
>>>>
>>>> Also what is the nature of this dump? How big is it? Where will it go?
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>> On 12/08/2013 9:36 AM, Yumin Qi wrote:
>>>> |
>>>>> |Hi, all
>>>>>
>>>>>      I would like to have your review for
>>>>>
>>>>> http://cr.openjdk.java.net/~minqi/8020962/webrev0/
>>>>> <http://cr.openjdk.java.net/%7Eminqi/8020962/webrev0/>
>>>>>
>>>>> Description: When JVM crashed, we also want to check the application
>>>>> class files especially we got core file from customers.  The
>>>>> aftermath analysis will benefit from all loaded java classes
>>>>> available. In this change, spawn another process running SA to do
>>>>> the job when JVM crashes, this way also avoid further messing up
>>>>> with the error report which already in signal handler.
>>>>>
>>>>> Note: The test has done with following two bugs worked around:
>>>>>      8022655: ClassDump ignored jarStream setting (This will be fixed
>>>>> and integrated by Kevin Walls soon)
>>>>>      8011888: sa.js: TypeError: [object JSAdapter] has no such
>>>>> function "__has__" (Not know when it will be integrated)
>>>>>      That is, without those two fixed, the jars of loaded classes
>>>>> will not be successfully dumped.
>>>>>      Also, on MacOS it requires security access permission to attach
>>>>> to another process, so omit doing so. To get loaded jar file s, with
>>>>> core file available (on all platforms), one can (only after this
>>>>> change) do
>>>>>
>>>>>      $JAVA_HOME/bin/java [-d64] -cp $JAVA_HOME/lib/sa-jdi.jar
>>>>> sun.jvm.hotspot.DumpLoadedClasses $JAVA_HOME/bin/java corefile
>>>>>
>>>>>
>>>>> Thanks
>>>>> Yumin
>>>>> |
>>> |
>>> |
>> |
>>
>> |
> |
> | 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/attachments/20130812/76e0c955/attachment-0001.html