<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 15, 2022 at 11:16 AM David Holmes <<a href="mailto:david.holmes@oracle.com">david.holmes@oracle.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 15/08/2022 6:28 pm, Thomas Stüfe wrote:<br>
> On Mon, Aug 15, 2022 at 10:27 AM Thomas Stüfe <<a href="mailto:thomas.stuefe@gmail.com" target="_blank">thomas.stuefe@gmail.com</a> <br>
> <mailto:<a href="mailto:thomas.stuefe@gmail.com" target="_blank">thomas.stuefe@gmail.com</a>>> wrote:<br>
> <br>
> <br>
> On Mon, Aug 15, 2022 at 10:08 AM David Holmes<br>
> <<a href="mailto:david.holmes@oracle.com" target="_blank">david.holmes@oracle.com</a> <mailto:<a href="mailto:david.holmes@oracle.com" target="_blank">david.holmes@oracle.com</a>>> wrote:<br>
> <br>
> Hi Thomas,<br>
> <br>
> On 15/08/2022 3:42 pm, Thomas Stüfe wrote:<br>
> > ..The only way I could see this happening is if tty itself is<br>
> not valid.<br>
> > Is this very early during initialization?<br>
> <br>
> Quite the opposite, it is during VM termination.<br>
> <br>
> <br>
> Yes, that makes sense too. tty is defaultStream::instance, which we<br>
> delete in ostream_exit() in DestroyVM.<br>
> <br>
> Since this has bugging me for a while (more during<br>
> pre-initialization, but its the same problem) I opened<br>
> <a href="https://bugs.openjdk.org/browse/JDK-8292351" rel="noreferrer" target="_blank">https://bugs.openjdk.org/browse/JDK-8292351</a><br>
> <<a href="https://bugs.openjdk.org/browse/JDK-8292351" rel="noreferrer" target="_blank">https://bugs.openjdk.org/browse/JDK-8292351</a>> to track this. Should<br>
> be trivial to fix.<br>
<br>
Perhaps trivial to not crash, but it means we are attempting various <br>
"logging" actions after they can't be done - which is a pain. Afterall <br>
we do want to see the real error here.<br>
<br></blockquote><div><br></div><div>That can happen though. As in your case, NMT telling us that the C-heap is corrupted during VM cleanup. So it is a valid scenario.</div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> <br>
> As for your crash, if you comment out "delete defaultStream::instance;" <br>
> in ostream.cpp, you should be able to see the real issue.<br>
<br>
Thanks - needed to do a little more than just that but I got the idea. <br>
So that leads me to:<br>
<br>
# Internal Error <br>
(/scratch/users/daholme/jdk-dev3.git/open/src/hotspot/share/services/mallocHeader.inline.hpp:61), <br>
pid=2666, tid=2668<br>
# fatal error: NMT corruption: Block at 0x00007fba3f56ca90: header <br>
canary broken<br>
<br>
Cheers,<br>
David<br>
-----<br></blockquote><div><br></div><div>No hex dump?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> <br>
> Cheers,<br>
> David<br>
> <br>
> > On Mon, Aug 15, 2022 at 7:40 AM Thomas Stüfe<br>
> <<a href="mailto:thomas.stuefe@gmail.com" target="_blank">thomas.stuefe@gmail.com</a> <mailto:<a href="mailto:thomas.stuefe@gmail.com" target="_blank">thomas.stuefe@gmail.com</a>><br>
> > <mailto:<a href="mailto:thomas.stuefe@gmail.com" target="_blank">thomas.stuefe@gmail.com</a><br>
> <mailto:<a href="mailto:thomas.stuefe@gmail.com" target="_blank">thomas.stuefe@gmail.com</a>>>> wrote:<br>
> ><br>
> > Hi David,<br>
> ><br>
> > NMT detected a heap corruption or the pointer to be freed is<br>
> > invalid. And then the reporter itself crashed while<br>
> printing the<br>
> > report. That is strange, since the reporter should be<br>
> resilient<br>
> > against crashes. It uses os::print_hex_dump, which should use<br>
> > SafeFetch to check if the to-be-dumped info is accessible.<br>
> ><br>
> > Any more information?<br>
> ><br>
> > Cheers, Thomas<br>
> ><br>
> > On Mon, Aug 15, 2022 at 7:18 AM David Holmes<br>
> > <<a href="mailto:david.holmes@oracle.com" target="_blank">david.holmes@oracle.com</a> <mailto:<a href="mailto:david.holmes@oracle.com" target="_blank">david.holmes@oracle.com</a>><br>
> <mailto:<a href="mailto:david.holmes@oracle.com" target="_blank">david.holmes@oracle.com</a><br>
> <mailto:<a href="mailto:david.holmes@oracle.com" target="_blank">david.holmes@oracle.com</a>>>> wrote:<br>
> ><br>
> > I'm getting the below crash:<br>
> ><br>
> > #<br>
> > # A fatal error has been detected by the Java Runtime<br>
> Environment:<br>
> > #<br>
> > # SIGSEGV (0xb) at pc=0x00007f3e3e24b417, pid=4537,<br>
> tid=4538<br>
> > #<br>
> > # JRE version: Java(TM) SE Runtime Environment (20.0)<br>
> (fastdebug<br>
> > build<br>
> > 20-internal-2022-06-10-0001400.daholme...)<br>
> > # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug<br>
> > 20-internal-2022-06-10-0001400.daholme..., mixed<br>
> mode, sharing,<br>
> > tiered,<br>
> > compressed oops, compressed class ptrs, g1 gc,<br>
> linux-amd64)<br>
> > # Problematic frame:<br>
> > # V [libjvm.so+0x174b417] <br>
> outputStream::print_cr(char const*,<br>
> > ...)+0x67<br>
> ><br>
> > --------------- T H R E A D ---------------<br>
> ><br>
> > Current thread (0x00007f3e38029890): Thread "Unknown<br>
> thread"<br>
> > [stack:<br>
> > 0x00007f3e3f8a9000,0x00007f3e3f9aa000] [id=4538]<br>
> ><br>
> > Stack: [0x00007f3e3f8a9000,0x00007f3e3f9aa000],<br>
> > sp=0x00007f3e3f9a84b0,<br>
> > free space=1021k<br>
> > Native frames: (J=compiled Java code, j=interpreted,<br>
> Vv=VM code,<br>
> > C=native code)<br>
> > V [libjvm.so+0x174b417] outputStream::print_cr(char<br>
> const*,<br>
> > ...)+0x67<br>
> > V [libjvm.so+0x157d708]<br>
> > MallocHeader::print_block_on_error(outputStream*,<br>
> unsigned<br>
> > char*) const+0x38<br>
> > V [libjvm.so+0x157f2ef] <br>
> MallocHeader::assert_block_integrity()<br>
> > const+0x10f<br>
> > V [libjvm.so+0x157eb6b] <br>
> MallocTracker::record_free(void*)+0x3b<br>
> > V [libjvm.so+0x172d3ef] os::free(void*)+0xbf<br>
> > V [libjvm.so+0x1a6d77a] Thread::~Thread()+0x8a<br>
> > V [libjvm.so+0x1060292] JavaThread::~JavaThread()+0x12<br>
> > V [libjvm.so+0x1a8b174] Threads::destroy_vm()+0x264<br>
> ><br>
> > Any tips on trying to figure out what is going wrong?<br>
> ><br>
> > Thanks,<br>
> > David<br>
> ><br>
> <br>
</blockquote></div></div>