Crash in the MallocTracker::record_free

[David Holmes]

On 15/08/2022 6:28 pm, Thomas Stüfe wrote:
> On Mon, Aug 15, 2022 at 10:27 AM Thomas Stüfe <thomas.stuefe at gmail.com 
> <mailto:thomas.stuefe at gmail.com>> wrote:
> 
> 
>     On Mon, Aug 15, 2022 at 10:08 AM David Holmes
>     <david.holmes at oracle.com <mailto:david.holmes at oracle.com>> wrote:
> 
>         Hi Thomas,
> 
>         On 15/08/2022 3:42 pm, Thomas Stüfe wrote:
>          > ..The only way I could see this happening is if tty itself is
>         not valid.
>          > Is this very early during initialization?
> 
>         Quite the opposite, it is during VM termination.
> 
> 
>     Yes, that makes sense too. tty is defaultStream::instance, which we
>     delete in ostream_exit() in DestroyVM.
> 
>     Since this has bugging me for a while (more during
>     pre-initialization, but its the same problem) I opened
>     https://bugs.openjdk.org/browse/JDK-8292351
>     <https://bugs.openjdk.org/browse/JDK-8292351> to track this. Should
>     be trivial to fix.

Perhaps trivial to not crash, but it means we are attempting various 
"logging" actions after they can't be done - which is a pain. Afterall 
we do want to see the real error here.

> 
> As for your crash, if you comment out "delete defaultStream::instance;" 
> in ostream.cpp, you should be able to see the real issue.

Thanks - needed to do a little more than just that but I got the idea. 
So that leads me to:

#  Internal Error 
(/scratch/users/daholme/jdk-dev3.git/open/src/hotspot/share/services/mallocHeader.inline.hpp:61), 
pid=2666, tid=2668
#  fatal error: NMT corruption: Block at 0x00007fba3f56ca90: header 
canary broken

Cheers,
David
-----

> 
>         Cheers,
>         David
> 
>          > On Mon, Aug 15, 2022 at 7:40 AM Thomas Stüfe
>         <thomas.stuefe at gmail.com <mailto:thomas.stuefe at gmail.com>
>          > <mailto:thomas.stuefe at gmail.com
>         <mailto:thomas.stuefe at gmail.com>>> wrote:
>          >
>          >     Hi David,
>          >
>          >     NMT detected a heap corruption or the pointer to be freed is
>          >     invalid. And then the reporter itself crashed while
>         printing the
>          >     report. That is strange, since the reporter should be
>         resilient
>          >     against crashes. It uses os::print_hex_dump, which should use
>          >     SafeFetch to check if the to-be-dumped info is accessible.
>          >
>          >     Any more information?
>          >
>          >     Cheers, Thomas
>          >
>          >     On Mon, Aug 15, 2022 at 7:18 AM David Holmes
>          >     <david.holmes at oracle.com <mailto:david.holmes at oracle.com>
>         <mailto:david.holmes at oracle.com
>         <mailto:david.holmes at oracle.com>>> wrote:
>          >
>          >         I'm getting the below crash:
>          >
>          >         #
>          >         # A fatal error has been detected by the Java Runtime
>         Environment:
>          >         #
>          >         #  SIGSEGV (0xb) at pc=0x00007f3e3e24b417, pid=4537,
>         tid=4538
>          >         #
>          >         # JRE version: Java(TM) SE Runtime Environment (20.0)
>         (fastdebug
>          >         build
>          >         20-internal-2022-06-10-0001400.daholme...)
>          >         # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug
>          >         20-internal-2022-06-10-0001400.daholme..., mixed
>         mode, sharing,
>          >         tiered,
>          >         compressed oops, compressed class ptrs, g1 gc,
>         linux-amd64)
>          >         # Problematic frame:
>          >         # V  [libjvm.so+0x174b417] 
>         outputStream::print_cr(char const*,
>          >         ...)+0x67
>          >
>          >         ---------------  T H R E A D  ---------------
>          >
>          >         Current thread (0x00007f3e38029890):  Thread "Unknown
>         thread"
>          >         [stack:
>          >         0x00007f3e3f8a9000,0x00007f3e3f9aa000] [id=4538]
>          >
>          >         Stack: [0x00007f3e3f8a9000,0x00007f3e3f9aa000],
>          >         sp=0x00007f3e3f9a84b0,
>          >         free space=1021k
>          >         Native frames: (J=compiled Java code, j=interpreted,
>         Vv=VM code,
>          >         C=native code)
>          >         V  [libjvm.so+0x174b417]  outputStream::print_cr(char
>         const*,
>          >         ...)+0x67
>          >         V  [libjvm.so+0x157d708]
>          >         MallocHeader::print_block_on_error(outputStream*,
>         unsigned
>          >         char*) const+0x38
>          >         V  [libjvm.so+0x157f2ef] 
>         MallocHeader::assert_block_integrity()
>          >         const+0x10f
>          >         V  [libjvm.so+0x157eb6b] 
>         MallocTracker::record_free(void*)+0x3b
>          >         V  [libjvm.so+0x172d3ef]  os::free(void*)+0xbf
>          >         V  [libjvm.so+0x1a6d77a]  Thread::~Thread()+0x8a
>          >         V  [libjvm.so+0x1060292]  JavaThread::~JavaThread()+0x12
>          >         V  [libjvm.so+0x1a8b174]  Threads::destroy_vm()+0x264
>          >
>          >         Any tips on trying to figure out what is going wrong?
>          >
>          >         Thanks,
>          >         David
>          >
>