MappedMemorySegments.isLoaded() can crash the JVM with large mapped files on Linux

[Maurizio Cimadamore]

Ugh. We'll file an issue in core-libs and take a look. As you notice, 
memory segment is just reusing mapped buffer implementation for some of 
the operation, and it seems some of those have issues.

We'll take a look:

https://bugs.openjdk.java.net/browse/JDK-8264827

Maurizio

On 06/04/2021 22:25, Marcel Käufler wrote:
> Hi,
>
> I've found a bug when calling `MappedMemorySegments::isLoaded`. With a 
> large segment size this causes malloc in the native isLoaded0 method 
> being called with an invalid size, crashing the JVM.
>
> Any size below 2GB - 4KB works fine. Above that it first gives an 
> OutOfMemoryError. Even larger sizes (e.g. 4GB) then crash the JVM.
> It's actually not only related to the foreign memory access api but 
> also applies to MappedByteBuffers in existing release builds. As the 
> ByteBuffers max. size is limited to Integer.MAX_VALUE it won't crash 
> the JVM and "only" give an OOM between 2G - 4K and 2G.
>
> It's caused by a miss-scoped cast in java.nio.Bits::pageCount:
>
>     static int pageCount(long size) {
>         return (int)(size + (long)pageSize() - 1L) / pageSize();
>     }
>
> There should be parentheses around the whole statement before the 
> (int) cast.
>
>
> To reproduce:
>
>     import jdk.incubator.foreign.MappedMemorySegments;
>     import jdk.incubator.foreign.MemorySegment;
>
>     import java.io.File;
>     import java.io.IOException;
>     import java.nio.channels.FileChannel;
>
>     public class Main {
>
>         public static final long SIZE_OK = 1024L * 1024 * 1024 * 2 - 
> 4096;
>         public static final long SIZE_OOM = SIZE_OK + 1;
>         public static final long SIZE_CRASH = SIZE_OK * 2;
>
>         public static void main(String[] args) throws IOException {
>             File f = new File("/tmp/is_loaded_test_file");
>             f.createNewFile();
>             MemorySegment memorySegment = 
> MemorySegment.mapFile(f.toPath(), 0,
>                     SIZE_CRASH, FileChannel.MapMode.READ_WRITE);
> System.out.println(MappedMemorySegments.isLoaded(memorySegment));
>         }
>     }
>
>
> Best Regards
> Marcel