[G1] Why humongous object bytes allocated after a collection are not recorded in G1Policy::_bytes_allocated_in_old_since_last_gc?

[Thomas Schatzl]

Hi Ziyi,

On 25.05.20 18:16, Luo, Ziyi wrote:
> Hi Thomas,
> 
>> Hi,
>>
>> On 23.05.20 00:30, Luo, Ziyi wrote:
>>> Hi,
>>>
>>> I have a question about humongous object allocation result in g1CollectedHeap:
>>> http://hg.openjdk.java.net/jdk/jdk/file/6d7c3a8bfab6/src/hotspot/share/gc/g1/g1CollectedHeap.cpp#l874
>>> http://hg.openjdk.java.net/jdk/jdk/file/6d7c3a8bfab6/src/hotspot/share/gc/g1/g1CollectedHeap.cpp#l893
>>>
>>> L874 returns the result after a successful humongous allocation. L893 returns
>>> the result after a successful G1CollectForAllocation. In the former case, the
>>> allocated bytes are recorded in
>>> G1Policy::_bytes_allocated_in_old_since_last_gc in L873. But bytes in the
>>> latter case are not. Is this intentional or a mistake?
>>>
>>> Best,
>>> Ziyi
>>>
>>
>>    the collection itself has some code to add the allocated bytes. In
>> the first case since there is no collection, that value needs to be
>> incremented manually.
>>
>> But you probably already know that, you filed JDK-8245511 :)
> 
> Yes, this is about JDK-8245511. In IHOP, _bytes_allocated_in_old_since_last_gc
> is used to calculate the old gen allocation rate. In some jtreg tests (e.g., gc

Afaik the only consumer for the old gen allocation rate (i.e. 
_bytes_allocated_in_old_since_last_gc and the mutator phase time) is 
IHOP calculation, so whatever makes it "correct" should be stored in there.

As mentioned in my comment to JDK-8245511, imho 
_bytes_allocated_in_old_since_last_gc should contain the "surviving" 
bytes allocated in the old gen, i.e. bytes_at_end_of_gc_X_in_old_gen - 
bytes_at_end_of_gc_X_in_old_gen_minus_1.

However you somehow need/want to account for that during mutator time 
the actual old gen occupancy may be higher than at the end of gc. That's 
what basically the "young_gen_size" parameter of 
update_ihop_prediction() is for - some additional buffer space in the 
IHOP calculation to cover extra allocation not covered by 
allocation_rate * marking_time.

> /TestSoftReferencesBehaviorOnOOME), I observed the following gc log output:
> 
> GC(215) Pause Full (G1 Humongous Allocation)
> GC(215) Humongous regions: 0->0
> ...
> GC(216) Basic information (value update) ... recent allocation size: 0B,
> recent allocation duration: 173.09ms, recent old gen allocation rate: 0.00B/s
> GC(216) Humongous regions: 111->111
> 
> Correct me if I were wrong: The allocated bytes during G1CollectForAllocation
> are considered as "on gc allocation". Since they are allocated during a pause,
> they should not be used to calculate the old gen allocation rate, thus are not
> included in G1Policy::_bytes_allocated_in_old_since_last_gc.
> 
> If that's the case, for JDK-8245511, I need to adjust how the survived
> humongous regions in a GC is recorded.

The current calculation simply assumes that humongous object allocation 
is "rare", and so significant (or only) allocation of them is not 
particularly well handled as you noticed.

Thanks,
   Thomas