RFR: 8306561: Possible out of bounds access in print_pointer_information

[Joachim Kern]

On Fri, 27 Oct 2023 11:52:50 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>>> @tstuefe: I helped analyzing this problem by writing a plain c test pgm mmaping a page and trying to read beyound. On AIX as expected the very next byte after the requested region leads to a segmentation violation, but on linux (both flavours, linuxintel and linuxppc64) I was able to read exactly 20 KB beyond, before running into segmentation violation. This might be the reason, why the developer of print_pointer_information() was not aware that he creates code that could crash. Thomas, do you have an idea why linux (and maybe other platforms) map more memory as requested? It has nothing to do with memory pages. The additional memory does not end at the next memory page boundary, but exactly 20KB after the end of the requested region. Astonishing is that at the lower end of the region there is no extra memory accessible.
>> 
>> Plain bad luck and rare test execution.
>> 
>> Whether or not you can read over the end of an mmaped segment depends on whether there are VMAs mapped beyond that. Linux kernel clusters VMAs to keep VMA fragmentation down. So you may have adjacent mappings. This is subject to ASLR of course but I always see the VMAs pretty much clustered regardless of ASLR. And though this sounds like it should be random, you can have a pretty consistent order of VMA allocation across many VM runs, and therefore similarly looking process memory maps.
>> 
>> Just looking at /proc/pid/maps will be probably clearer to you than me explaining it :)
>
>> > @tstuefe: I helped analyzing this problem by writing a plain c test pgm mmaping a page and trying to read beyound. On AIX as expected the very next byte after the requested region leads to a segmentation violation, but on linux (both flavours, linuxintel and linuxppc64) I was able to read exactly 20 KB beyond, before running into segmentation violation. This might be the reason, why the developer of print_pointer_information() was not aware that he creates code that could crash. Thomas, do you have an idea why linux (and maybe other platforms) map more memory as requested? It has nothing to do with memory pages. The additional memory does not end at the next memory page boundary, but exactly 20KB after the end of the requested region. Astonishing is that at the lower end of the region there is no extra memory accessible.
>> 
>> Plain bad luck and rare test execution.
>> 
>> Whether or not you can read over the end of an mmaped segment depends on whether there are VMAs mapped beyond that. Linux kernel clusters VMAs to keep VMA fragmentation down. So you may have adjacent mappings. This is subject to ASLR of course but I always see the VMAs pretty much clustered regardless of ASLR. And though this sounds like it should be random, you can have a pretty consistent order of VMA allocation across many VM runs, and therefore similarly looking process memory maps.
>> 
>> Just looking at /proc/pid/maps will be probably clearer to you than me explaining it :)
> 
> Just read your answer and I see you wrote a little test program that shows that behavior, probably single threaded. The same explanation applies here though, unless there is a simple bug.
> 
> For instance, if - after the mmap - you do something that allocates C-heap, the libc may allocate a new arena with mmap and place it just beyond your mmaped region. A lot of libc functions need C-heap under the hood (e.g. calling C assert() will allocate C-heap to assemble the assert line).

@tstuefe: Meanwhile I had also asked Klaus Möser who gave me a similar explanation and we both together examined /proc/pid/maps to understand that a runtime library had allocated the memory beyond my mmap region.
Klaus also told me that AIX has an mmap alignment of 256GB, so it is very likely that the addresses beyond and below your mapped region are not used, explaining the segmentation violation.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16381#issuecomment-1782843697