Java heap file on daxfs should be more secure

[Yasumasa Suenaga]

Hi all,

HotSpot allocates Java heap on daxfs if we pass -XX:AllocateHeapAt.
It performs open(2) and unlink(2) on daxfs, and it is used via mmap'ed address.

mmap(2) would be called with MAP_SHARED, and it is not atomically between open(2) and unlink(2). If malicious user open Java heap file before unlink(2), it might be exposed.

So I think we can use open(2) with O_TMPFILE instead of mkstemp(3) as below.

   http://cr.openjdk.java.net/~ysuenaga/dax/

O_TMPFILE would create inode on filesystem, and it cannot be accessed from out-of-process.
However it cannot be provided in older Linux kernel. So I keep current code as fall back.

   http://man7.org/linux/man-pages/man2/open.2.html

What do you think about it? or someone is working for it?
If it is ok, I will file it to JBS and will send review request.


Thanks,

Yasumasa


P.S.
   I tried to use MAP_PRIVATE for it, but it was slower than MAP_SHARED.