RFR: 8306561: Possible out of bounds access in print_pointer_information

[Thomas Stuefe]

On Thu, 26 Oct 2023 16:11:00 GMT, Thomas Obermeier <duke at openjdk.org> wrote:

> MallocTracker::print_pointer_information in src/hotspot/share/services/mallocTracker.cpp is called to check the highest pointer address of the reserved region. To do so it aligns the test pointer down to the next 8 Byte boundary and casts this address to class MallocHeader in order to use this classes eye-catcher member _canary for validation. Method looks_valid() dereferences _canary's content. _canary has an offset of 14 bytes relative to the class. Therefore it resides outside the reserved region for the highest pointer address, which causes a segmentation violation.
> 
> We would expect the same error also for other platforms than AIX as memory is read, which is not allocated. Interestingly, Linux seems to allow this access for 5 times 4K above the reserved region.
> 
> As a solution, looks_valid() should check _canary's address as being invalid, and return false immediately.

I would fix it another way. The underlying assumption in MallocHeader::looks_valid() is that the header resides fully in readable memory. The caller must make sure of that.

The better way to fix this would be in print_pointer_information(). It must make sure, before calling MallocHeader::looks_valid(), that the header is contained fully in readable memory.

Something like:

-     if (!os::is_readable_pointer(here)) {
+    if (!os::is_readable_pointer(here) || !os::is_readable_pointer(here + sizeof(MallocHeader) {

or something similar.

-------------

Changes requested by stuefe (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/16381#pullrequestreview-1700078017