RFR: 8239129: Use DAX in ZGC

[Yasumasa Suenaga]

On 2020/02/17 19:06, Per Liden wrote:
> 
> 
> On 2/17/20 8:58 AM, Yasumasa Suenaga wrote:
>> Hi Per,
>>
>> On 2020/02/17 15:50, Per Liden wrote:
>>> Hi,
>>>
>>> On 2/17/20 5:05 AM, Yasumasa Suenaga wrote:
>>>> Hi,
>>>>
>>>> I filed this enhancement to JBS:
>>>>
>>>>    JBS: https://bugs.openjdk.java.net/browse/JDK-8239129
>>>>    CSR: https://bugs.openjdk.java.net/browse/JDK-8239130
>>>
>>> We will not introduce a new option like this, so please withdraw the CSR (you also don't need a CSR for adding an experimental options).
>>
>> I withdrew it.
>>
>>
>>>>    webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8239129/webrev.00/
>>>
>>> Before this patch can go forward, you need to get to the bottom of how to get that ioctl command to work. If it's not possible, you need to explain why and propose alternatives that we can discuss.
>>
>> I guess it is caused by Linux kernel.
>> In case of ext4, `ext4_iflags_to_xflags()` would set filesystem flags to `struct FS_IOC_FSGETXATTR`.
>> However `FS_XFLAG_DAX` is not handled in it.
> 
> Did a bit of googleing and it seems the DAX flag is in a bit of flux at the moment. I guess this will be fixed down the road, when DAX in the kernel becomes a non-experimental feature.
> 
> How about we just do like this for now:
> 
> http://cr.openjdk.java.net/~pliden/8239129/webrev.0

I thought ZGC requires tmpfs or hugetlbfs due to performance reason.
So I introduced new -XX option to make users aware of it.

If not so, I agree with your change.


Yasumasa


> /Per
> 
>>
>> https://urldefense.com/v3/__https://github.com/torvalds/linux/blob/master/fs/ext4/ioctl.c*L525__;Iw!!GqivPVa7Brio!KN3UJKZwdbjq6abJnSXLf78BAUX9742P2PJFHS6kO5_cAgG6kxQEBBBez7uFixk$
>>
>> Cheers,
>>
>> Yasumasa
>>
>>
>>> cheers,
>>> Per
>>>
>>>>
>>>> Could you review this change and CSR?
>>>> It passed tests on submit repo (mach5-one-ysuenaga-JDK-8239129-20200217-0213-8777205).
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Yasumasa
>>>>
>>>>
>>>> On 2020/02/15 2:08, Per Liden wrote:
>>>>> Hi,
>>>>>
>>>>> On 2/14/20 3:23 PM, Yasumasa Suenaga wrote:
>>>>>> On 2020/02/14 23:08, Per Liden wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> On 2/14/20 2:31 PM, Yasumasa Suenaga wrote:
>>>>>>>> Hi Per,
>>>>>>>>
>>>>>>>> On 2020/02/14 20:52, Per Liden wrote:
>>>>>>>>> Hi Yasumasa,
>>>>>>>>>
>>>>>>>>> On 2/14/20 10:07 AM, Yasumasa Suenaga wrote:
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I tried to allocate heap to DAX on Linux with -XX:AllocateHeapAt, but it couldn't.
>>>>>>>>>> It seems to allow when filesystem is hugetlbfs or tmpfs.
>>>>>>>>>>
>>>>>>>>>> According to kernel document [1], DAX is supported in ext2, ext4, and xfs.
>>>>>>>>>> Also we need to mount it with "-o dax".
>>>>>>>>>>
>>>>>>>>>> I want to use ZGC on DAX, so I want to introduce new option -XX:ZAllowHeapOnFileSystem to allow to use all filesystem as backing storage.
>>>>>>>>>> What do you think this change?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> +  experimental(bool, ZAllowHeapOnFileSystem, false,     \
>>>>>>>>> +          "Allow to use filesystem as Java heap backing storage "     \
>>>>>>>>> +          "specified by -XX:AllocateHeapAt")     \
>>>>>>>>> +     \
>>>>>>>>>
>>>>>>>>> Instead of adding a new option it would be preferable to automatically detect that it's a dax mounted filesystem. But I haven't has a chance to look into the best way of doing that.
>>>>>>>>
>>>>>>>> I thought so, but I guess it is difficult.
>>>>>>>> PMDK also does not check it automatically.
>>>>>>>>
>>>>>>>> https://urldefense.com/v3/__https://github.com/pmem/pmdk/blob/master/src/libpmem2/pmem2_utils_linux.c*L18__;Iw!!GqivPVa7Brio!PlQs19bQVBJF7PDA9RLZ9JLbXOQ2KYocNW6DJH-eOUqXZcYwl-cSvSjpfC316y0$
>>>>>>>> In addition, we don't seem to be able to get mount option ("-o dax") via syscall.
>>>>>>>> I strace'ed `mount -o dax ...`, I saw "-o dax" was passed to 5th argument (const void *data). It would be handled in each filesystem, so I could not get it.
>>>>>>>>
>>>>>>>> Another solution, we can use /proc/mounts, but it might be complex.
>>>>>>>
>>>>>>> I was maybe hoping you could get this information through some ioctl() command on the file descriptor?
>>>>>>
>>>>>> I tried to FS_IOC_FSGETXATTR ioctl (FS_XFLAG_DAX might be set in fsx_xflags), but I couldn't get.
>>>>>> (I use ext4 with "-o dax")
>>>>>
>>>>>
>>>>> Ok. It would be good to get to the bottom of why it's not set.
>>>>>
>>>>> cheers,
>>>>> Per