ZGC: RFR: 8207843: HSDB cannot show Object Histogram when ZGC is working

[Per Liden]

Hi,

On 08/16/2018 10:39 AM, Yasumasa Suenaga wrote:
> Hi,
> 
> On 2018/08/16 15:55, Per Liden wrote:
>> Hi,
>>
>> On 08/16/2018 03:52 AM, Yasumasa Suenaga wrote:
>>> Hi,
>>>
>>> On 2018/08/15 2:36, Per Liden wrote:
>>>> Hi,
>>>>
>>>> On 2018-08-11 14:06, Yasumasa Suenaga wrote:
>>>>> Hi Per,
>>>>>
>>>>> SA needs memory regions to generate class histogram. So it is 
>>>>> difficult to use visitor model for it.
>>>>> I want to back to this issue when class unloading is implemented.
>>>>
>>>> If you want to pursue the path of walking the live set, then you can 
>>>> do that today, you don't need to wait for class unloading support to 
>>>> appear.
>>>
>>> I might be able to implement it. However ObjectHeap in SA needs 
>>> memory region, so it might be so complex code, and have large 
>>> different code from other GC algorithms.
>>
>> Yes, this would be a fairly big undertaking.
>>
>>> (Other GC algorithms can provide live spaces to ObjectHeap.)
>>>
>>> BTW I concern class unloading issue also occurs in heap dump.
>>> For example, if the user requests heap dump with -all option in ZGC 
>>> concurrent cycle, it might have invalid objects in it.
>>
>> This isn't an issue, because heap dumps are always generated by 
>> walking the live-set (see ZHeapIterator), so we never walk over 
>> unreachable objects.
> 
> Thank you for your advice.
> 
> I checked VM_HeapDumper::doit() and ZCollectedHeap::collect_as_vm_thread().
> ZCollectedHeap::collect_as_vm_thread() will not work, so `-all` option 
> in jmap / jcmd also will not work.

It works in the sense that you will get a heap dump, but it will not 
contain any unreachable objects. But the -all option does not guarantee 
that anyway, since the GC is free so compact at any time it wants, 
there's nothing guaranteeing that you will see unreachable objects in a 
heap dump when using the -all option.

But it's true that with ZGC, this is the common cases, where as with the 
other GCs it's the unusual case.

cheers,
Per

> 
> Is it listed somewhere? IMHO it should be documented.
> 
> 
> Cheers,
> 
> Yasumasa
> 
> 
>> cheers,
>> Per
>>
>>>
>>>
>>> cheers,
>>>
>>> Yasumasa
>>>
>>>
>>>> cheers,
>>>> Per
>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Yasumasa
>>>>>
>>>>>
>>>>> On 2018/08/08 20:45, Per Liden wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On 08/08/2018 10:34 AM, Yasumasa Suenaga wrote:
>>>>>>> On 2018/08/08 17:07, Per Liden wrote:
>>>>>>>> On 08/08/2018 10:02 AM, Per Liden wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> On 08/08/2018 10:00 AM, Yasumasa Suenaga wrote:
>>>>>>>>>> Hi Per,
>>>>>>>>>>
>>>>>>>>>> Will class unloading be occurred after relocation?
>>>>>>>>>
>>>>>>>>> No, before relocation.
>>>>>>>>
>>>>>>>> But it doesn't really matter when it happens, since there can 
>>>>>>>> still be unreachable objects with dead class pointers in ZPages 
>>>>>>>> that were not relocated.
>>>>>>>
>>>>>>> I think we can include all unreachable objects in the histogram.
>>>>>>> For example, ParallelScavengeHeap in SA collects all objects 
>>>>>>> between bottom() and top() in all spaces like `jcmd 
>>>>>>> GC.class_histogram` with -all option.
>>>>>>>
>>>>>>> So I think we can fix Object Histogram for ZGC if we resolve dead 
>>>>>>> class pointer.
>>>>>>> If SA can know ZGC phase, we might resolve it as following:
>>>>>>>
>>>>>>>
>>>>>>>    * GC is working:
>>>>>>>        Check colored pointer to collect oops.
>>>>>>>
>>>>>>>    * GC is NOT working:
>>>>>>>        Collect all oops in ZPage.
>>>>>>
>>>>>> Unfortunately that's not good enough. It doesn't matter if the GC 
>>>>>> is "working" or not, there might be unreachable objects from 
>>>>>> several GC cycles back which you can't touch. The only safe way to 
>>>>>> do this is to walk the live graph, which means starting from the 
>>>>>> roots. As you might have noticed, ZCollectedHeap::object_iterate() 
>>>>>> is implemented like this, for this exact reason.
>>>>>>
>>>>>> cheers,
>>>>>> Per
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Yasumasa
>>>>>>>
>>>>>>>
>>>>>>>> /Per