<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<br>
<br>
<div class="moz-cite-prefix">On 2023-12-01 13:16, Alen Vrečko wrote:<br>
</div>
<blockquote type="cite" cite="mid:CACOdjgTrO121Geau6dEc4VBf__CbF2E=+UEYtGV73oSTu8J7TQ@mail.gmail.com">
<div dir="ltr">
<div>Hi Stefan,</div>
<div><br>
</div>
I looked why JOL v0.10 had issues with generational. It failed
to attach Serviceability Agent to the process. In that case it
uses a fallback method to calculate various JVM parameters.
Among them it got object alignment totally wrong.
<div><br>
</div>
<div>It uses this method to guess object alignment: It allocates
objects, gets their memory address and does Math#GCD. I think
pretty clever.</div>
<div><br>
</div>
<div><a href="https://urldefense.com/v3/__https://github.com/openjdk/jol/blob/d9e890652b4cec9d155b28a8849dcdaa2706e058/jol-core/src/main/java/org/openjdk/jol/vm/HotspotUnsafe.java*L405__;Iw!!ACWV5N9M2RV99hQ!Pdh92iRfc7i53RQB_6amv4xSQ0vQjNjJXv6xBjou7074VgD4oxZRRrJwKGfnUCclidEcQZjGPjFsKj5OUN8sLNC5$" target="_blank" moz-do-not-send="true">https://github.com/openjdk/jol/blob/d9e890652b4cec9d155b28a8849dcdaa2706e058/jol-core/src/main/java/org/openjdk/jol/vm/HotspotUnsafe.java#L405</a><br>
</div>
<div><br>
</div>
<div>The JOL method guesses 65536 for object alignment when
using generational.</div>
</div>
</blockquote>
<br>
The problem seems to be JOL's implementation of addressOf:<br>
<br>
objectAddress = U.getLong(array, arrayObjectBase);<br>
<br>
It reads the object pointers as longs, which skips using load
barriers, and therefore doesn't shave of the ZGC colors:<br>
<br>
<a class="moz-txt-link-freetext" href="https://github.com/openjdk/jdk/blob/8f1d40b48bf145144ae90b1d147d418d3905661b/src/hotspot/share/gc/z/zAddress.hpp#L44">https://github.com/openjdk/jdk/blob/8f1d40b48bf145144ae90b1d147d418d3905661b/src/hotspot/share/gc/z/zAddress.hpp#L44</a><br>
<br>
// A zpointer is a combination of the address bits (heap base bit +
offset)<br>
// and two low-order metadata bytes, with the following layout:<br>
<br>
<blockquote type="cite" cite="mid:CACOdjgTrO121Geau6dEc4VBf__CbF2E=+UEYtGV73oSTu8J7TQ@mail.gmail.com">
<div dir="ltr">
<div> I tried my own version of this approach. I get 16 when
generational is enabled.</div>
<div><br>
</div>
<div>I am curious to know why are the results like this. Does
young gen have a different object alignment than the old gen?</div>
</div>
</blockquote>
<br>
I don't know why you got 16. The alignment is supposed to be 8 for
both old gen and young gen.<br>
<br>
Cheers,<br>
StefanK<br>
<br>
<blockquote type="cite" cite="mid:CACOdjgTrO121Geau6dEc4VBf__CbF2E=+UEYtGV73oSTu8J7TQ@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>Thanks</div>
<div>Alen</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">V V sre., 29. nov. 2023 ob
19:37 je oseba Alen Vrečko <<a href="mailto:alen.vrecko@gmail.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">alen.vrecko@gmail.com</a>>
napisala:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Hi Stefan,
<div><br>
</div>
<div>all good. Finally got around to it. My bad in both
cases.</div>
<div><br>
</div>
<div>o) adding System.gc() solved the problem. Indeed, not
a good idea to have expectations when working with
java.lang.ref.Cleaner. Preferably not use it at all.</div>
<div><br>
</div>
<div>o) for the corrupted byte[], got a chance to look
into it. Not just speculate on log output. The issue was
in Java Object Layout library (used v0.10). It returned
something like 500K for the size of an object if
Generational is enabled (should be in the range of <
100B). This caused a failure while processing byte[] and
why I assumed that the byte[] is corrupted. I updated
the jol library to 0.17 and it works fine now.
Interesting that it looks like JOL v0.10 works fine on
CentOS 7 with generational but not Alma 9.2 with
generational - same 21 jdk.</div>
<div><br>
</div>
<div>Time to fix some bad first impressions.</div>
<div><br>
</div>
<div>Thanks</div>
<div>Alen</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">V V pon., 13. nov. 2023
ob 22:21 je oseba Alen Vrečko <<a href="mailto:alen.vrecko@gmail.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">alen.vrecko@gmail.com</a>>
napisala:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Thanks for the fast reply Stefan.
<div><br>
</div>
<div>For the reference issue. Looks like I
misunderstood. Most probably issue with timing in
the toy program with major collections. For both G1
and ZGC (non generational) both counters for new
Foo() and Cleaner(foo)#clean match after a short
while. But not for generational ZGC. I'll add
System.gc() call in there and see what happens. Most
probably a non-issue then and a misunderstanding on
my part.</div>
<div><br>
</div>
<div>For the corrupted byte[]. Will see how much time
I have on my hands to look into it. Like mentioned
vanilla ZGC works fine, with generational ZGC seeing
funny stuff with byte[].</div>
<div><br>
</div>
<div>Alen</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">V V pon., 13. nov.
2023 ob 20:28 je oseba Stefan Karlsson <<a href="mailto:stefan.karlsson@oracle.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">stefan.karlsson@oracle.com</a>>
napisala:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div> Hi Alen,<br>
<br>
<div>On 2023-11-13 19:05, Alen Vrečko wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hello everyone,
<div><br>
</div>
<div>o) young gen reference processor</div>
<div><br>
</div>
<div>A bit puzzled by reading in a thread on
the list:</div>
<div><br>
</div>
<div>> <span style="color:rgb(0,0,0);white-space:pre-wrap">mentioning that we decided to not ship a young generation reference processor for 21</span><br>
</div>
<div><span style="color:rgb(0,0,0);white-space:pre-wrap">
</span></div>
<div><span style="color:rgb(0,0,0);white-space:pre-wrap">Unless you made changes to ByteBuffer#allocateDirect it uses reference processor to free native memory. If I am not mistaking just using standard library API such as Files.readAllBytes will in some cases do BB#allocateDirect in the internals.</span></div>
<div><font color="#000000"><span style="white-space:pre-wrap">
</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">Or maybe I am misunderstanding something? I made a toy program and indeed I could easily get a situation where 20% of reference handlers are not called like ever.</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">
</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">This will cause issues for code that is using reference handlers.</span></font></div>
</div>
</blockquote>
<br>
The reference processing will happen when the GC
performs a major collection, which collects both
the young and old generation. If you add a
System.gc() you should see that the reference
processor is kicking in for your program. Could
you share your toy program?<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><font color="#000000"><span style="white-space:pre-wrap">
</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">o) seeing weird byte[] corruption in production</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">
</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">On CentOS 7 Generational works fine. No issues observed. But on Alma Linux 9.2 either reading byte[] from file or sending byte[] over the network corrupts the byte[]. Didn't investigate at all. Just observed corruption in some cases for some byte[] arrays - not all - just some. On the same Alma Linux 9.2 without generational zgc no byte[] corruption is observed and everything works fine as before.</span></font></div>
</div>
</blockquote>
<br>
It's hard to say if this is a ZGC bug, compiler
bug, OS bug, etc. Here are some suggestions for
how to help pin-point the problem:<br>
1) Could you provide the output from 'java
-version'?<br>
2) Is it possible to reproduce this with a small
reproducer?<br>
3) What CPU is this running on?<br>
4) Does it happen with -XX:UseAVX=0<br>
5) Do you know the sizes of the corrupted byte[]s?
Do you know the offset to where it is corrupted?<br>
<br>
StefanK<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><font color="#000000"><span style="white-space:pre-wrap">
</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">To me Generational ZGC looks more like an experimental feature for now. I am a bit surprised it doesn't require the extra flag to unlock experimental features.</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">
</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">Thanks</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">Alen</span></font></div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</body>
</html>