RFR: 8311986: runtime/os/TestTracePageSizes.java fails for ShenandoahGC

Fri Jul 14 08:24:53 UTC 2023

On Fri, 14 Jul 2023 03:13:54 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> Hi,
>> 
>> runtime/os/TestTracePageSizes.java fails for ShenandoahGC on linux-x86_64:
>> 
>> 
>> ACTION: main -- Failed. Execution failed: `main' threw exception: java.lang.AssertionError: Page sizes mismatch: 4 != 2048
>> REASON: User specified action: run main/othervm -XX:+AlwaysPreTouch -Xmx128m -Xlog:pagesize:ps-%p.log -XX:-SegmentedCodeCache -XX:+UseTransparentHugePages TestTracePageSizes 
>> TIME:   0.353 seconds
>> messages:
>> command: main -XX:+AlwaysPreTouch -Xmx128m -Xlog:pagesize:ps-%p.log -XX:-SegmentedCodeCache -XX:+UseTransparentHugePages TestTracePageSizes
>> reason: User specified action: run main/othervm -XX:+AlwaysPreTouch -Xmx128m -Xlog:pagesize:ps-%p.log -XX:-SegmentedCodeCache -XX:+UseTransparentHugePages TestTracePageSizes 
>> started: Thu Jul 13 09:57:18 CST 2023
>> Mode: othervm [/othervm specified]
>> finished: Thu Jul 13 09:57:19 CST 2023
>> elapsed time (seconds): 0.353
>> configuration:
>> STDOUT:
>> STDERR:
>> java.lang.AssertionError: Page sizes mismatch: 4 != 2048
>> 	at TestTracePageSizes.main(TestTracePageSizes.java:294)
>> 	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
>> 	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
>> 	at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
>> 	at java.base/java.lang.Thread.run(Thread.java:1570)
>> 
>> JavaTest Message: Test threw exception: java.lang.AssertionError: Page sizes mismatch: 4 != 2048
>> JavaTest Message: shutting down test
>> 
>> STATUS:Failed.`main' threw exception: java.lang.AssertionError: Page sizes mismatch: 4 != 2048
>> rerun:
>> cd /mnt/repo/openjdk/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_runtime_os_TestTracePageSizes_java/scratch/0 && \
>> HOME=/home/loongson \
>> LANG=zh_CN.UTF-8 \
>> LC_ALL=C \
>> PATH=/bin:/usr/bin:/usr/sbin \
>> TEST_IMAGE_DIR=/mnt/repo/openjdk/jdk/build/linux-x86_64-server-fastdebug/images/test \
>> _JVM_DWARF_PATH=/mnt/repo/openjdk/jdk/build/linux-x86_64-server-fastdebug/images/symbols \
>> CLASSPATH=/mnt/repo/openjdk/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_runtime_os_TestTracePageSizes_java/classes/0/runtime/os/TestTracePageSizes_compiler-options.d:/mnt/repo/openjdk/jdk/test/hotspot/jtreg/runtime/os:/mnt/repo/openjdk/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_hotspot_jtreg_runtime_os_TestTracePageSizes_java/classes/0/test/lib:/mnt/repo/openjdk/jdk/test/lib:/mnt/download/jtreg/lib/j...
>
> It would be nice to understand why the test fails with ShenandoahGC (and ZGC for that matter).

Hi @dholmes-ora 

It might be `VmFlags` issue.

> "VmFlags" field deserves a separate description. This member represents the kernel flags associated with the particular virtual memory area in two letter encoded manner https://github.com/torvalds/linux/blob/master/Documentation/filesystems/proc.rst

[DEBUG](https://github.com/xiangzhai/jdk/commits/DEBUG-8311986) on purpose to add two bitmaps: `lzhai_bitmap` and `rzhai_bitmap` and `UseShenandoahGC` to rerun the testcase:

DEBUG: size: 4194304 preferred_page_size: 2097152
DEBUG: size: 4194304 preferred_page_size: 4096
DEBUG: size: 4194304 preferred_page_size: 2097152
DEBUG: size: 4194304 preferred_page_size: 2097152
DEBUG: size: 2097152 preferred_page_size: 2097152
DEBUG: smapsFileToParse: null
DEBUG: smapsCopy: smaps-copy-38346-0.txt
DEBUG: line: [0.001s][info][pagesize] Using the default large page size: 2M
DEBUG: line: [0.001s][info][pagesize] Usable page sizes: 4k, 2M
DEBUG: line: [0.002s][info][pagesize] CodeCache:  min=4M max=240M base=0x00007f1584800000 size=240M page_size=2M
DEBUG: pageSize: 2M
DEBUG: pageSizeFromSmaps: 4
DEBUG: pageSizeFromTrace: 2048
DEBUG: VmFlags: rd wr ex mr mw me ac sd hg 
DEBUG: line: [0.003s][info][pagesize] Heap: req_size=128M req_page_size=2M base=0x00000000f8000000 size=128M page_size=2M
DEBUG: pageSize: 2M
DEBUG: pageSizeFromSmaps: 4
DEBUG: pageSizeFromTrace: 2048
DEBUG: VmFlags: rd wr mr mw me ac sd hg 
DEBUG: line: [0.003s][info][pagesize] Mark Bitmap: req_size=4M req_page_size=2M base=0x00007f15a2a00000 size=4M page_size=2M
DEBUG: pageSize: 2M
DEBUG: pageSizeFromSmaps: 4
DEBUG: pageSizeFromTrace: 2048
DEBUG: VmFlags: rd wr mr mw me ac sd hg 
DEBUG: line: [0.004s][info][pagesize] DEBUG: Leslie Zhai Bitmap: req_size=4M req_page_size=4K base=0x00007f15a1ffa000 size=4M page_size=4K
DEBUG: pageSize: 4K
DEBUG: pageSizeFromSmaps: 4
DEBUG: pageSizeFromTrace: 4
=> DEBUG: line: [0.004s][info][pagesize] DEBUG: Ray Zhai Bitmap: req_size=4M req_page_size=2M base=0x00007f15a1a00000 size=4M page_size=2M
DEBUG: pageSize: 2M
DEBUG: pageSizeFromSmaps: 4
DEBUG: pageSizeFromTrace: 2048
=> DEBUG: VmFlags: mr mw me nr sd 
java.lang.AssertionError: Page sizes mismatch: 4 != 2048
	at TestTracePageSizes.main(TestTracePageSizes.java:301)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
	at java.base/java.lang.Thread.run(Thread.java:1570)

JavaTest Message: Test threw exception: java.lang.AssertionError: Page sizes mismatch: 4 != 2048
JavaTest Message: shutting down test

STATUS:Failed.`main' threw exception: java.lang.AssertionError: Page sizes mismatch: 4 != 2048

[smaps-copy-38346-0.txt](https://github.com/openjdk/jdk/files/12047810/smaps-copy-38346-0.txt)

The `VmFlags` of virtual memory area `Ray Zhai Bitmap` is lack of `hg`. So *not* hit the condition `range.isTransparentHuge()`.

Compared with Generational ZGC `UseZGC` and `ZGenerational` to rerun the testcase:

DEBUG: smapsFileToParse: null
DEBUG: smapsCopy: smaps-copy-38376-0.txt
DEBUG: line: [0.001s][info][pagesize] Using the default large page size: 2M
DEBUG: line: [0.001s][info][pagesize] Usable page sizes: 4k, 2M
=> DEBUG: line: [0.002s][info][pagesize] CodeCache:  min=4M max=240M base=0x00007f024c800000 size=240M page_size=2M
DEBUG: pageSize: 2M
DEBUG: pageSizeFromSmaps: 4
DEBUG: pageSizeFromTrace: 2048
=> DEBUG: VmFlags: rd wr ex mr mw me ac sd hg 
STATUS:Passed.

There is `hg` in the `VmFlags` of vma `CodeCache` even if the `os::vm_page_size` (4KB for linux-x86_64) is not equals to `bitmap_page_size` when `UseLargePages`.

linux-x86_64 and linux-loongarch64 confirmed enabled THP:

$ cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never

Thanks,
Leslie Zhai

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14858#issuecomment-1635488777