<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<pre style="background-color: rgb(255, 255, 255);"><div style="text-align: left;"><span style="letter-spacing: normal; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;" class="elementToProof">When an entry in the list is removed due to GC it should distribute its span to neighbors (and the priority updated).</span></div><div class="elementToProof" style="text-align: left;"><span style="letter-spacing: normal; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"><br></span></div><div class="elementToProof" style="text-align: left;"><span style="letter-spacing: normal; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">That way, the total in the queue should be total allocated since the beginning.</span></div><div class="elementToProof" style="text-align: left;"><span style="letter-spacing: normal; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"><br></span></div><div class="elementToProof" style="text-align: left;"><span style="letter-spacing: normal; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">(I'm not reading the code, but describing the intent)</span></div><div class="elementToProof" style="text-align: left;"><span style="letter-spacing: normal; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"><br></span></div><div class="elementToProof" style="text-align: left;"><span style="letter-spacing: normal; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">Erik</span></div></pre>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Galder Zamarreno <galder@redhat.com><br>
<b>Sent:</b> Wednesday, January 24, 2024 10:45 AM<br>
<b>To:</b> Thomas Stüfe <thomas.stuefe@gmail.com><br>
<b>Cc:</b> Erik Gahlin <erik.gahlin@oracle.com>; hotspot-jfr-dev@openjdk.org <hotspot-jfr-dev@openjdk.org><br>
<b>Subject:</b> [External] : Re: Priority redistribution in Old Object Sample event</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div dir="ltr">
<div class="x_gmail_default" style="font-family:arial,helvetica,sans-serif; font-size:small">
Erik,</div>
<div class="x_gmail_default" style="font-family:arial,helvetica,sans-serif; font-size:small">
<br>
</div>
<div class="x_gmail_default" style="font-family:arial,helvetica,sans-serif; font-size:small">
We never got an answer to Thomas' concerns that total_allocate only ever increases. Is that right? See below:</div>
</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr" class="x_gmail_attr">On Wed, Jun 14, 2023 at 9:56 AM Thomas Stüfe <<a href="mailto:thomas.stuefe@gmail.com">thomas.stuefe@gmail.com</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div><span class="x_gmail_default" style="font-family:arial,helvetica,sans-serif; font-size:small"></span>.<span class="x_gmail_default" style="font-family:arial,helvetica,sans-serif; font-size:small">..</span></div>
<div><br>
</div>
<div>- I am looking at the code that calculates the span in the first place:</div>
<div><a href="https://urldefense.com/v3/__https://github.com/tstuefe/jdk/blob/ba837b4bfa2dea85653d8a8fccd0817a569b4378/src/hotspot/share/jfr/leakprofiler/sampling/objectSampler.cpp*L229-L230__;Iw!!ACWV5N9M2RV99hQ!JgmmQAyJeNPRWOg8r8IUbg5HQlS3djiF8gehNTDM8FhGq4faiWXGnrmTJBOvt3ApJeSj8dDPVt7y-DwY$" target="_blank">https://github.com/tstuefe/jdk/blob/ba837b4bfa2dea85653d8a8fccd0817a569b4378/src/hotspot/share/jfr/leakprofiler/sampling/objectSampler.cpp#L229-L230</a></div>
<div><br>
</div>
<div>The span of a new sample is calculated as "_total_allocated (as of now) - sum-of-all-spans-of-samples-in-prio-queue". _total_allocated increases monotonously, whereas the prio queue only contains *retained* samples. This means that the new sample's span
is the "what had been allocated since the last sample was taken" + "all spans of samples that had been discarded". Is that intentional? Would this not mean that with prolonged sampling, the spans get ever larger?</div>
<div><br>
</div>
<div>I may be reading this code wrong.</div>
<div><br>
</div>
<div>Thank you!</div>
<div><br>
</div>
<div>..Thomas<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr" class="x_gmail_attr">On Wed, Jun 14, 2023 at 9:37 AM Erik Gahlin <<a href="mailto:erik.gahlin@oracle.com" target="_blank">erik.gahlin@oracle.com</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div>
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
The intent of the code is to remove the sample from both the time-based list and the priority queue as the object is no longer alive. <span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">The weight of the object, the
allocation span, in the time-based list, should be divided to its neighbour(s). </span><span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">The rationale for this is to keep samples evenly distributed over time, or
to be exact, the amount allocation has happened. It will be as if the object was never sampled (as if the TLAB size would be larger)</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)"><br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">The priority queue exists to find the sampled object with the lowest allocation span quickly when the list fills up. It needs to be replaced with a sampled object
with higher weight/span.</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)"><br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">This may, or may not, be what the code does.
</span><span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">It was written many years ago, and I need to re-read the code to check if that is really the case, but before digging deeper, I want to check if we are on
the same page when it comes to the intent</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)"><br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">Erik</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div id="x_m_-1869217787601454757m_7248280304523550059appendonsend"></div>
<hr style="display:inline-block; width:98%">
<div id="x_m_-1869217787601454757m_7248280304523550059divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> hotspot-jfr-dev <<a href="mailto:hotspot-jfr-dev-retn@openjdk.org" target="_blank">hotspot-jfr-dev-retn@openjdk.org</a>>
on behalf of Galder Zamarreno <<a href="mailto:galder@redhat.com" target="_blank">galder@redhat.com</a>><br>
<b>Sent:</b> Wednesday, June 14, 2023 6:30 AM<br>
<b>To:</b> <a href="mailto:hotspot-jfr-dev@openjdk.org" target="_blank">hotspot-jfr-dev@openjdk.org</a> <<a href="mailto:hotspot-jfr-dev@openjdk.org" target="_blank">hotspot-jfr-dev@openjdk.org</a>><br>
<b>Subject:</b> Priority redistribution in Old Object Sample event</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div style="font-family:arial,helvetica,sans-serif; font-size:small">Hi,</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">I've been porting over JFR Old Object Sample event to GraalVM and I had the following doubt about the code in HotSpot:</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">I'm trying to understand the code in [1]:</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">```</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"> // push span onto previous<br>
if (previous != NULL) {<br>
_priority_queue->remove(previous);<br>
previous->add_span(sample->span());<br>
_priority_queue->push(previous);<br>
}<br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">```</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">This code is related to old object sampling. When the queue is full, samples that have been GC'd are removed from the queue. To do so it takes its previous (from the list view) and adds the
span of the removed dead object onto it. Basically, I'm wondering why this is done this way.</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">My initial gut feeling when I first saw the code is that it does it so that the priority queue does not get too imbalanced (it uses a heap underneath) when removing objects, but that only
makes sense to me if the previous is not the previous object sampled based on time, but the previous object sampled based on the order of the priority queue, so that's not what seems to happen.<br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">I'm wondering now if the fact that the list view is ordered by insertion time is relevant here. Is it maybe trying to give more importance (by increasing its span) to surviving objects that
are near in time to those that have been garbage collected?<br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">Something that's probably related to this is how the span is initially calculated. This is done in [2], which you can find below:</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">```</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">_total_allocated += allocated;<br>
const size_t span = _total_allocated - _priority_queue->total();<br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">```</div>
<div>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">Again when I first looked at it, I thought: well that's just equivalent to:</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">```</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">const size_t span = allocated;<br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">```</div>
<br>
</div>
<div dir="ltr">
<div style="font-family:arial,helvetica,sans-serif; font-size:small">But clearly things are more nuanced than that.</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">The only documentation I've found so far is in Marcus' blog post [3]:</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">> If sampled objects are garbage collected, they are removed and the priority redistributed to the neighbours. The priority is called span, and is currently the size of the allocation, giving
more weight to larger (therefore more severe) leaks.</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">Unfortunately what is a neighbour is not explained, nor why the priority gets redistributed.</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">Thanks</div>
</div>
<div dir="ltr">Galder</div>
</div>
<div dir="ltr"><br>
</div>
<div dir="ltr">
<div style="font-family:arial,helvetica,sans-serif; font-size:small">[1] <a href="https://urldefense.com/v3/__https://github.com/openjdk/jdk17u-dev/blob/master/src/hotspot/share/jfr/leakprofiler/sampling/objectSampler.cpp*L264-L269__;Iw!!ACWV5N9M2RV99hQ!JgmmQAyJeNPRWOg8r8IUbg5HQlS3djiF8gehNTDM8FhGq4faiWXGnrmTJBOvt3ApJeSj8dDPVvH06sry$" target="_blank">
https://github.com/openjdk/jdk17u-dev/blob/master/src/hotspot/share/jfr/leakprofiler/sampling/objectSampler.cpp#L264-L269</a></div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">[2] <a href="https://urldefense.com/v3/__https://github.com/openjdk/jdk17u-dev/blob/ef86ea2842b1a204834291d9d6665bfcd7b75fbc/src/hotspot/share/jfr/leakprofiler/sampling/objectSampler.cpp*L212-L213__;Iw!!ACWV5N9M2RV99hQ!JgmmQAyJeNPRWOg8r8IUbg5HQlS3djiF8gehNTDM8FhGq4faiWXGnrmTJBOvt3ApJeSj8dDPVmGdkFVr$" target="_blank">https://github.com/openjdk/jdk17u-dev/blob/ef86ea2842b1a204834291d9d6665bfcd7b75fbc/src/hotspot/share/jfr/leakprofiler/sampling/objectSampler.cpp#L212-L213</a></div>
<div style="font-family:arial,helvetica,sans-serif; font-size:small">[3] <a href="https://urldefense.com/v3/__https://hirt.se/blog/?p=1055__;!!ACWV5N9M2RV99hQ!JgmmQAyJeNPRWOg8r8IUbg5HQlS3djiF8gehNTDM8FhGq4faiWXGnrmTJBOvt3ApJeSj8dDPVu1fE2fz$" target="_blank">
https://hirt.se/blog/?p=1055</a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
</body>
</html>