Crash in the MallocTracker::record_free

[Thomas Stüfe]

On Mon, Aug 15, 2022 at 11:16 AM David Holmes <david.holmes at oracle.com>
wrote:

> On 15/08/2022 6:28 pm, Thomas Stüfe wrote:
> > On Mon, Aug 15, 2022 at 10:27 AM Thomas Stüfe <thomas.stuefe at gmail.com
> > <mailto:thomas.stuefe at gmail.com>> wrote:
> >
> >
> >     On Mon, Aug 15, 2022 at 10:08 AM David Holmes
> >     <david.holmes at oracle.com <mailto:david.holmes at oracle.com>> wrote:
> >
> >         Hi Thomas,
> >
> >         On 15/08/2022 3:42 pm, Thomas Stüfe wrote:
> >          > ..The only way I could see this happening is if tty itself is
> >         not valid.
> >          > Is this very early during initialization?
> >
> >         Quite the opposite, it is during VM termination.
> >
> >
> >     Yes, that makes sense too. tty is defaultStream::instance, which we
> >     delete in ostream_exit() in DestroyVM.
> >
> >     Since this has bugging me for a while (more during
> >     pre-initialization, but its the same problem) I opened
> >     https://bugs.openjdk.org/browse/JDK-8292351
> >     <https://bugs.openjdk.org/browse/JDK-8292351> to track this. Should
> >     be trivial to fix.
>
> Perhaps trivial to not crash, but it means we are attempting various
> "logging" actions after they can't be done - which is a pain. Afterall
> we do want to see the real error here.
>
>
That can happen though. As in your case, NMT telling us that the C-heap is
corrupted during VM cleanup. So it is a valid scenario.


> >
> > As for your crash, if you comment out "delete defaultStream::instance;"
> > in ostream.cpp, you should be able to see the real issue.
>
> Thanks - needed to do a little more than just that but I got the idea.
> So that leads me to:
>
> #  Internal Error
> (/scratch/users/daholme/jdk-dev3.git/open/src/hotspot/share/services/mallocHeader.inline.hpp:61),
>
> pid=2666, tid=2668
> #  fatal error: NMT corruption: Block at 0x00007fba3f56ca90: header
> canary broken
>
> Cheers,
> David
> -----
>

No hex dump?


>
> >
> >         Cheers,
> >         David
> >
> >          > On Mon, Aug 15, 2022 at 7:40 AM Thomas Stüfe
> >         <thomas.stuefe at gmail.com <mailto:thomas.stuefe at gmail.com>
> >          > <mailto:thomas.stuefe at gmail.com
> >         <mailto:thomas.stuefe at gmail.com>>> wrote:
> >          >
> >          >     Hi David,
> >          >
> >          >     NMT detected a heap corruption or the pointer to be freed
> is
> >          >     invalid. And then the reporter itself crashed while
> >         printing the
> >          >     report. That is strange, since the reporter should be
> >         resilient
> >          >     against crashes. It uses os::print_hex_dump, which should
> use
> >          >     SafeFetch to check if the to-be-dumped info is accessible.
> >          >
> >          >     Any more information?
> >          >
> >          >     Cheers, Thomas
> >          >
> >          >     On Mon, Aug 15, 2022 at 7:18 AM David Holmes
> >          >     <david.holmes at oracle.com <mailto:david.holmes at oracle.com>
> >         <mailto:david.holmes at oracle.com
> >         <mailto:david.holmes at oracle.com>>> wrote:
> >          >
> >          >         I'm getting the below crash:
> >          >
> >          >         #
> >          >         # A fatal error has been detected by the Java Runtime
> >         Environment:
> >          >         #
> >          >         #  SIGSEGV (0xb) at pc=0x00007f3e3e24b417, pid=4537,
> >         tid=4538
> >          >         #
> >          >         # JRE version: Java(TM) SE Runtime Environment (20.0)
> >         (fastdebug
> >          >         build
> >          >         20-internal-2022-06-10-0001400.daholme...)
> >          >         # Java VM: Java HotSpot(TM) 64-Bit Server VM
> (fastdebug
> >          >         20-internal-2022-06-10-0001400.daholme..., mixed
> >         mode, sharing,
> >          >         tiered,
> >          >         compressed oops, compressed class ptrs, g1 gc,
> >         linux-amd64)
> >          >         # Problematic frame:
> >          >         # V  [libjvm.so+0x174b417]
> >         outputStream::print_cr(char const*,
> >          >         ...)+0x67
> >          >
> >          >         ---------------  T H R E A D  ---------------
> >          >
> >          >         Current thread (0x00007f3e38029890):  Thread "Unknown
> >         thread"
> >          >         [stack:
> >          >         0x00007f3e3f8a9000,0x00007f3e3f9aa000] [id=4538]
> >          >
> >          >         Stack: [0x00007f3e3f8a9000,0x00007f3e3f9aa000],
> >          >         sp=0x00007f3e3f9a84b0,
> >          >         free space=1021k
> >          >         Native frames: (J=compiled Java code, j=interpreted,
> >         Vv=VM code,
> >          >         C=native code)
> >          >         V  [libjvm.so+0x174b417]  outputStream::print_cr(char
> >         const*,
> >          >         ...)+0x67
> >          >         V  [libjvm.so+0x157d708]
> >          >         MallocHeader::print_block_on_error(outputStream*,
> >         unsigned
> >          >         char*) const+0x38
> >          >         V  [libjvm.so+0x157f2ef]
> >         MallocHeader::assert_block_integrity()
> >          >         const+0x10f
> >          >         V  [libjvm.so+0x157eb6b]
> >         MallocTracker::record_free(void*)+0x3b
> >          >         V  [libjvm.so+0x172d3ef]  os::free(void*)+0xbf
> >          >         V  [libjvm.so+0x1a6d77a]  Thread::~Thread()+0x8a
> >          >         V  [libjvm.so+0x1060292]
> JavaThread::~JavaThread()+0x12
> >          >         V  [libjvm.so+0x1a8b174]  Threads::destroy_vm()+0x264
> >          >
> >          >         Any tips on trying to figure out what is going wrong?
> >          >
> >          >         Thanks,
> >          >         David
> >          >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-runtime-dev/attachments/20220815/14c01603/attachment.htm>