<div dir="ltr">Thanks Stefan for looking into it as well.<div><br></div><div>For the address, hm, I don't get it. For the non generational case, method works without shaving off the colors in the address. But it doesn't work with generational.<br>I printed out some of the pointers.<br><br>non-generational pointers:<br><br>0000000000000000000100000000000000000111110001100000111001011000<br>0000000000000000000100000000000000000111110001100001001000001000<br>0000000000000000000100000000000000000111110001100001001000011000<br>0000000000000000000100000000000000000111110001100001010111001000</div><div><br></div><div>generational</div>0000000010000000000000001111001000101100000000000001010100010000<br>0000000010000000000000001111001000101100000000100001010100010000<br>0000000010000000000000001111001000101100011101100001010100010000<br>0000000010000000000000001111001000101100011110000001010100010000<div><br></div><div>So the 2 low order bytes are the ZGC meta, the barriers and colors right?</div><div>And the high order bytes are the heap base bit (part) + offset?</div><div><br></div><div>I tried just shaving off the 2 high order bytes but not seeing the correct results. Is there more to extracting the address than just shaving off the 2 low order bytes?</div><div><br></div><div>Thanks</div><div>Alen</div><div><div><p style="margin:0px;font-stretch:normal;font-size:22px;line-height:normal;font-family:Helvetica"><br></p></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">V V pet., 1. dec. 2023 ob 13:36 je oseba Stefan Karlsson <<a href="mailto:stefan.karlsson@oracle.com" target="_blank">stefan.karlsson@oracle.com</a>> napisala:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><u></u>
<div>
Hi Alen,<br>
<br>
I'm glad that you figured out what was happening. FWIW, I ran a
whole bunch of tests on Alma 9.2 and couldn't reproduce any issues.<br>
<br>
Cheers,<br>
StefanK<br>
<br>
<div>On 2023-11-29 19:37, Alen Vrečko wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Stefan,
<div><br>
</div>
<div>all good. Finally got around to it. My bad in both cases.</div>
<div><br>
</div>
<div>o) adding System.gc() solved the problem. Indeed, not a
good idea to have expectations when working with
java.lang.ref.Cleaner. Preferably not use it at all.</div>
<div><br>
</div>
<div>o) for the corrupted byte[], got a chance to look into it.
Not just speculate on log output. The issue was in Java Object
Layout library (used v0.10). It returned something like 500K
for the size of an object if Generational is enabled (should
be in the range of < 100B). This caused a failure while
processing byte[] and why I assumed that the byte[] is
corrupted. I updated the jol library to 0.17 and it works fine
now. Interesting that it looks like JOL v0.10 works fine on
CentOS 7 with generational but not Alma 9.2 with generational
- same 21 jdk.</div>
<div><br>
</div>
<div>Time to fix some bad first impressions.</div>
<div><br>
</div>
<div>Thanks</div>
<div>Alen</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">V V pon., 13. nov. 2023 ob
22:21 je oseba Alen Vrečko <<a href="mailto:alen.vrecko@gmail.com" target="_blank">alen.vrecko@gmail.com</a>>
napisala:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Thanks for the fast reply Stefan.
<div><br>
</div>
<div>For the reference issue. Looks like I misunderstood.
Most probably issue with timing in the toy program with
major collections. For both G1 and ZGC (non generational)
both counters for new Foo() and Cleaner(foo)#clean match
after a short while. But not for generational ZGC. I'll
add System.gc() call in there and see what happens. Most
probably a non-issue then and a misunderstanding on my
part.</div>
<div><br>
</div>
<div>For the corrupted byte[]. Will see how much time I have
on my hands to look into it. Like mentioned vanilla ZGC
works fine, with generational ZGC seeing funny stuff with
byte[].</div>
<div><br>
</div>
<div>Alen</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">V V pon., 13. nov. 2023 ob
20:28 je oseba Stefan Karlsson <<a href="mailto:stefan.karlsson@oracle.com" target="_blank">stefan.karlsson@oracle.com</a>>
napisala:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div> Hi Alen,<br>
<br>
<div>On 2023-11-13 19:05, Alen Vrečko wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hello everyone,
<div><br>
</div>
<div>o) young gen reference processor</div>
<div><br>
</div>
<div>A bit puzzled by reading in a thread on the
list:</div>
<div><br>
</div>
<div>> <span style="color:rgb(0,0,0);white-space:pre-wrap">mentioning that we decided to not ship a young generation reference processor for 21</span><br>
</div>
<div><span style="color:rgb(0,0,0);white-space:pre-wrap">
</span></div>
<div><span style="color:rgb(0,0,0);white-space:pre-wrap">Unless you made changes to ByteBuffer#allocateDirect it uses reference processor to free native memory. If I am not mistaking just using standard library API such as Files.readAllBytes will in some cases do BB#allocateDirect in the internals.</span></div>
<div><font color="#000000"><span style="white-space:pre-wrap">
</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">Or maybe I am misunderstanding something? I made a toy program and indeed I could easily get a situation where 20% of reference handlers are not called like ever.</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">
</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">This will cause issues for code that is using reference handlers.</span></font></div>
</div>
</blockquote>
<br>
The reference processing will happen when the GC
performs a major collection, which collects both the
young and old generation. If you add a System.gc() you
should see that the reference processor is kicking in
for your program. Could you share your toy program?<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><font color="#000000"><span style="white-space:pre-wrap">
</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">o) seeing weird byte[] corruption in production</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">
</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">On CentOS 7 Generational works fine. No issues observed. But on Alma Linux 9.2 either reading byte[] from file or sending byte[] over the network corrupts the byte[]. Didn't investigate at all. Just observed corruption in some cases for some byte[] arrays - not all - just some. On the same Alma Linux 9.2 without generational zgc no byte[] corruption is observed and everything works fine as before.</span></font></div>
</div>
</blockquote>
<br>
It's hard to say if this is a ZGC bug, compiler bug, OS
bug, etc. Here are some suggestions for how to help
pin-point the problem:<br>
1) Could you provide the output from 'java -version'?<br>
2) Is it possible to reproduce this with a small
reproducer?<br>
3) What CPU is this running on?<br>
4) Does it happen with -XX:UseAVX=0<br>
5) Do you know the sizes of the corrupted byte[]s? Do
you know the offset to where it is corrupted?<br>
<br>
StefanK<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><font color="#000000"><span style="white-space:pre-wrap">
</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">To me Generational ZGC looks more like an experimental feature for now. I am a bit surprised it doesn't require the extra flag to unlock experimental features.</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">
</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">Thanks</span></font></div>
<div><font color="#000000"><span style="white-space:pre-wrap">Alen</span></font></div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
<br>
</div>
</blockquote></div></div>