RFR: JDK-8289477: Memory corruption with CPU_ALLOC, CPU_FREE on muslc

[Thomas Stuefe]

On Thu, 30 Jun 2022 04:42:05 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> In `os::Linux::active_processor_count()`, we use the CPU_xxx macros to manage sets of CPU information.
>> 
>> muslc defines those macros to call `calloc(3)` and `free(3)`:
>> 
>> 
>> #define CPU_ALLOC(n) ((cpu_set_t *)calloc(1,CPU_ALLOC_SIZE(n)))
>> #define CPU_FREE(set) free(set)
>> 
>> 
>> whereas glibc uses intermediate functions:
>> 
>> 
>> #define __CPU_ALLOC(count) __sched_cpualloc (count)
>> #define __CPU_FREE(cpuset) __sched_cpufree (cpuset)
>> 
>> 
>> which in the end also takes from C-heap, but those calls are not inlined.
>> 
>> So, on muslc we call `calloc()` and `free()`. Call happens inside the `os::Linux` namespace, `free()` resolves to `os::free()`. We have no wrapper in os for calloc though, so `calloc()` calls into muslc right away.
>> 
>> That means we have raw ::malloc() -> os::free(), which is unbalanced. Raw `::malloc()` does not write the header `os::free()` expects. If NMT is on, we assert now, because NMT does not find its header in os::free(). 
>> 
>> This can be very easily reproduced by starting an Alpine VM with NMT on (or, a debug VM) and ` -XX:+UnlockDiagnosticVMOptions -XX:+UseCpuAllocPath`. This causes a NMT fence alert right away:
>> 
>> 
>> NMT Block at 0x00007fc5a35db9f0, corruption at: 0x00007fc5a35db9f0: 
>> 0x00007fc5a35db970:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x00007fc5a35db980:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x00007fc5a35db990:   f8 a1 c8 1b ea 55 00 00 00 00 00 00 00 c0 00 00
>> 0x00007fc5a35db9a0:   00 a4 c8 1b ea 55 00 00 01 00 00 00 00 c0 00 00
>> 0x00007fc5a35db9b0:   d8 a3 c8 1b ea 55 00 00 1d 00 00 00 00 a0 00 00
>> 0x00007fc5a35db9c0:   2d 63 70 00 00 00 00 00 08 00 00 00 00 61 01 00
>> 0x00007fc5a35db9d0:   2d 76 65 72 73 69 6f 6e 00 00 00 00 00 82 02 00
>> 0x00007fc5a35db9e0:   2d 73 65 72 76 65 72 00 00 00 00 00 00 83 03 00
>> 0x00007fc5a35db9f0:   2d 63 6c 69 65 6e 74 00 00 00 00 00 00 84 04 00
>> 0x00007fc5a35dba00:   ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x00007fc5a35dba10:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x00007fc5a35dba20:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x00007fc5a35dba30:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x00007fc5a35dba40:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x00007fc5a35dba50:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x00007fc5a35dba60:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
>> # To suppress the following error report, specify this argument
>> # after -XX: or in .hotspotrc:  SuppressErrorAt=/mallocTracker.cpp:151
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  Internal Error (/home/ubuntu/client_home/workspace/build-user-branch-linux_alpine_x86_64/SapMachine/src/hotspot/share/services/mallocTracker.cpp:151), pid=219496, tid=219512
>> 
>> 
>> 
>> The position of the musl devs is that "calloc" and "free" are reserved words in C, and should not be used [1]. I think they are right. The way we reuse known C- and Posix symbol names in the os namespace has bitten me in the past in similar cases.
>> 
>> ------
>> 
>> The fix is minimally invasive for easy backporting. I just move the content of `os::Linux::active_processor_count()` into its own local function, outside the os namespace. That way, CPU_FREE cannot pick up `os::free()` accidentally, and the error is fixed.
>> 
>> I really would like a more thorough solution though, renaming all the potential conflict candidates, but leave that for a follow-up RFE.
>> 
>> [1] https://www.openwall.com/lists/musl/2022/06/29/3
>
> Okay so the "binding" of `free()` will happen before any potential inlining could occur.
> 
> It is a nice simple workaround.
> 
> Thanks.

Thanks @dholmes-ora and @RealCLanger.

-------------

PR: https://git.openjdk.org/jdk/pull/9328