RFR: 8324580: SIGFPE on THP initialization on kernels < 4.10 [v4]

Tue Feb 6 12:00:54 UTC 2024

On Wed, 31 Jan 2024 16:42:16 GMT, Zdenek Zambersky <zzambers at openjdk.org> wrote:

>> **Problem:**
>> When THP is enabled, JDK reads `/sys/kernel/mm/transparent_hugepage/hpage_pmd_size` file to [detect](https://github.com/openjdk/jdk/blob/96607df7f055a80d56ea4c19f3f4fcb32838b1f8/src/hotspot/os/linux/hugepages.cpp#L206) large page size. However this file only [appeared](https://github.com/torvalds/linux/commit/49920d28781dcced10cd30cb9a938e7d045a1c94) in kernel 4.10 and does not exist on kernel old kernels (such as 3.10 used by RHEL-7).
>> 
>> This results in detected large page size of 0B and crash, when `-XX:+UseTransparentHugePages` is used on old kernel.
>> 
>> gdb --args jdk-23+6/bin/java -XX:+UseTransparentHugePages -Xmx128m -Xlog:pagesize -version
>> ...
>> [0.005s][info][pagesize] Static hugepage support:
>> [0.005s][info][pagesize]   hugepage size: 2M
>> [0.005s][info][pagesize]   hugepage size: 1G
>> [0.005s][info][pagesize]   default hugepage size: 2M
>> [0.005s][info][pagesize] Transparent hugepage (THP) support:
>> [0.005s][info][pagesize]   THP mode: always
>> [0.005s][info][pagesize]   THP pagesize: 0B
>> [0.005s][info][pagesize] Shared memory transparent hugepage (THP) support:
>> [0.005s][info][pagesize]   Shared memory THP mode: unknown
>> [0.005s][info][pagesize] JVM will attempt to prevent THPs in thread stacks.
>> [0.005s][info][pagesize] UseLargePages=1, UseTransparentHugePages=1
>> [0.005s][info][pagesize] Large page support enabled. Usable page sizes: 4k. Default large page size: 0B.
>> 
>> Program received signal SIGFPE, Arithmetic exception.
>> [Switching to Thread 0x7ffff7fc2700 (LWP 31385)]
>> 0x00007ffff68eeb20 in lcm(unsigned long, unsigned long) () from /home/tester/jdk-23+6/lib/server/libjvm.so
>> (gdb) bt
>> #0  0x00007ffff68eeb20 in lcm(unsigned long, unsigned long) () from /home/tester/jdk-23+6/lib/server/libjvm.so
>> #1  0x00007ffff64a68f2 in Arguments::apply_ergo() () from /home/tester/jdk-23+6/lib/server/libjvm.so
>> #2  0x00007ffff700458f in Threads::create_vm(JavaVMInitArgs*, bool*) () from /home/tester/jdk-23+6/lib/server/libjvm.so
>> #3  0x00007ffff6a2373f in JNI_CreateJavaVM () from /home/tester/jdk-23+6/lib/server/libjvm.so
>> #4  0x00007ffff7fe704b in InitializeJVM (ifn=<synthetic pointer>, penv=0x7ffff7fc1ea8, pvm=0x7ffff7fc1ea0) at src/java.base/share/native/libjli/java.c:1550
>> #5  JavaMain (_args=<optimized out>) at src/java.base/share/native/libjli/java.c:491
>> #6  0x00007ffff7feb1c9 in ThreadJavaMain (args=<optimized out>) at src/java.base/unix/native/libjli/java_md.c:650
>> #7  0x00007ffff7bc6ea5 in start_thread (arg...
>
> Zdenek Zambersky has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Disable THP for too large page sizes

> I'm unsure about this. I originally wanted the stuff in huge pages.cpp/hpp to not "lie" - as in not making assumptions. It was supposed to truthfully reflect the system's settings without making assumptions. So, if information is missing, print 0. The assumption-making should happen in the code above, in os_linux.cpp.
> 
> That strict separation was born out of working with the code before, which had been a big ball of tangled yarn with respect to who calls what and who lies about what. But maybe I am too religious about it now. But whatever we decide, I would like the information that we _assume_ a THP size instead of just reading it from the OS to be reflected somewhere in the log and printed settings.
> 

I agree this is a good approach, and things have really improved in this area over the last few years. It still felt like this kind of logic was a better fit in `THPSupport::scan_os()` and reading the default and using that would still not violate the "truth aspect" too much. I do agree that doing the limiting and potentially using a fixed default is more towards "lying", and it would be possible to split that out to `os_linux.cpp`. Not sure if this is worth it, so making sure the log is clear would be my choice.

> > If we update `THPSupport::scan_os()` to call `scan_default_hugepagesize()` if reading `hpage_pmd_size` fails we should be good. We can add a check there to limit the "default size" to 16M (which sounds reasonable) and maybe log that we did this limiting. I'm not certain what we should do if reading the default hugepage size fail (returns 0), but I'm leaning towards having 2M as a general fallback. It will only be used if THP mode is configured to `madvide` or `always` and in those cases we likely have at least a 2M page size (and I'm not sure this should ever happen). We could also add code to `validate_thps_configured()` to return false if the thp page size is 0.
> 
> Sure, we can do that. Though I believe THPs are implemented atop of explicit huge pages, so I would assume them to be always available.
> 

I think so too, that's why I think having a "fixed default" if nothing can be read should be ok. If we manage to read that the setting is `always`or `madvise` we should try to use them even if we can't read a size of the pages. 

> > An alternative to how we log this is to add a tag to the "THP pagesize" line, something like:
> > ```
> > [0.005s][info][pagesize]   THP pagesize: 0B (configured/default/limited/fallback)
> > ```
> > What do you think?
> 
> See above. I can do both. If we move THP pagesize recognition, including assumptions, down into hugepages.cpp (that was @zzambers first attempt too), I would like the information that we made an assumption reflected in the printout.

So what would you prefer, some kind of tag like suggested above, or a messages saying what we do, like:

[0.005s][info][pagesize] THP page size not present in sysfs
[0.005s][info][pagesize] Using default huge page size for THP

Maybe those should be on debug level if going this route.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/17545#issuecomment-1929367962