RFR: 8239129: Use DAX in ZGC

[Per Liden]

On 2/17/20 8:58 AM, Yasumasa Suenaga wrote:
> Hi Per,
> 
> On 2020/02/17 15:50, Per Liden wrote:
>> Hi,
>>
>> On 2/17/20 5:05 AM, Yasumasa Suenaga wrote:
>>> Hi,
>>>
>>> I filed this enhancement to JBS:
>>>
>>>    JBS: https://bugs.openjdk.java.net/browse/JDK-8239129
>>>    CSR: https://bugs.openjdk.java.net/browse/JDK-8239130
>>
>> We will not introduce a new option like this, so please withdraw the 
>> CSR (you also don't need a CSR for adding an experimental options).
> 
> I withdrew it.
> 
> 
>>>    webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8239129/webrev.00/
>>
>> Before this patch can go forward, you need to get to the bottom of how 
>> to get that ioctl command to work. If it's not possible, you need to 
>> explain why and propose alternatives that we can discuss.
> 
> I guess it is caused by Linux kernel.
> In case of ext4, `ext4_iflags_to_xflags()` would set filesystem flags to 
> `struct FS_IOC_FSGETXATTR`.
> However `FS_XFLAG_DAX` is not handled in it.

Did a bit of googleing and it seems the DAX flag is in a bit of flux at 
the moment. I guess this will be fixed down the road, when DAX in the 
kernel becomes a non-experimental feature.

How about we just do like this for now:

http://cr.openjdk.java.net/~pliden/8239129/webrev.0

/Per

> 
>    
> https://urldefense.com/v3/__https://github.com/torvalds/linux/blob/master/fs/ext4/ioctl.c*L525__;Iw!!GqivPVa7Brio!KN3UJKZwdbjq6abJnSXLf78BAUX9742P2PJFHS6kO5_cAgG6kxQEBBBez7uFixk$ 
> 
> 
> Cheers,
> 
> Yasumasa
> 
> 
>> cheers,
>> Per
>>
>>>
>>> Could you review this change and CSR?
>>> It passed tests on submit repo 
>>> (mach5-one-ysuenaga-JDK-8239129-20200217-0213-8777205).
>>>
>>>
>>> Thanks,
>>>
>>> Yasumasa
>>>
>>>
>>> On 2020/02/15 2:08, Per Liden wrote:
>>>> Hi,
>>>>
>>>> On 2/14/20 3:23 PM, Yasumasa Suenaga wrote:
>>>>> On 2020/02/14 23:08, Per Liden wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On 2/14/20 2:31 PM, Yasumasa Suenaga wrote:
>>>>>>> Hi Per,
>>>>>>>
>>>>>>> On 2020/02/14 20:52, Per Liden wrote:
>>>>>>>> Hi Yasumasa,
>>>>>>>>
>>>>>>>> On 2/14/20 10:07 AM, Yasumasa Suenaga wrote:
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I tried to allocate heap to DAX on Linux with 
>>>>>>>>> -XX:AllocateHeapAt, but it couldn't.
>>>>>>>>> It seems to allow when filesystem is hugetlbfs or tmpfs.
>>>>>>>>>
>>>>>>>>> According to kernel document [1], DAX is supported in ext2, 
>>>>>>>>> ext4, and xfs.
>>>>>>>>> Also we need to mount it with "-o dax".
>>>>>>>>>
>>>>>>>>> I want to use ZGC on DAX, so I want to introduce new option 
>>>>>>>>> -XX:ZAllowHeapOnFileSystem to allow to use all filesystem as 
>>>>>>>>> backing storage.
>>>>>>>>> What do you think this change?
>>>>>>>>
>>>>>>>>
>>>>>>>> +  experimental(bool, ZAllowHeapOnFileSystem, false,     \
>>>>>>>> +          "Allow to use filesystem as Java heap backing storage 
>>>>>>>> "     \
>>>>>>>> +          "specified by -XX:AllocateHeapAt")     \
>>>>>>>> +     \
>>>>>>>>
>>>>>>>> Instead of adding a new option it would be preferable to 
>>>>>>>> automatically detect that it's a dax mounted filesystem. But I 
>>>>>>>> haven't has a chance to look into the best way of doing that.
>>>>>>>
>>>>>>> I thought so, but I guess it is difficult.
>>>>>>> PMDK also does not check it automatically.
>>>>>>>
>>>>>>> https://urldefense.com/v3/__https://github.com/pmem/pmdk/blob/master/src/libpmem2/pmem2_utils_linux.c*L18__;Iw!!GqivPVa7Brio!PlQs19bQVBJF7PDA9RLZ9JLbXOQ2KYocNW6DJH-eOUqXZcYwl-cSvSjpfC316y0$ 
>>>>>>>
>>>>>>> In addition, we don't seem to be able to get mount option ("-o 
>>>>>>> dax") via syscall.
>>>>>>> I strace'ed `mount -o dax ...`, I saw "-o dax" was passed to 5th 
>>>>>>> argument (const void *data). It would be handled in each 
>>>>>>> filesystem, so I could not get it.
>>>>>>>
>>>>>>> Another solution, we can use /proc/mounts, but it might be complex.
>>>>>>
>>>>>> I was maybe hoping you could get this information through some 
>>>>>> ioctl() command on the file descriptor?
>>>>>
>>>>> I tried to FS_IOC_FSGETXATTR ioctl (FS_XFLAG_DAX might be set in 
>>>>> fsx_xflags), but I couldn't get.
>>>>> (I use ext4 with "-o dax")
>>>>
>>>>
>>>> Ok. It would be good to get to the bottom of why it's not set.
>>>>
>>>> cheers,
>>>> Per