<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>Hi Cleber,<br>
This is more a general question on FileChannel, so I'm adding
nio-dev in CC, as someone there might be able to help more.</p>
<p>Cheers<br>
Maurizio<br>
</p>
<div class="moz-cite-prefix">On 13/03/2024 23:09, Cleber Muramoto
wrote:<br>
</div>
<blockquote type="cite" cite="mid:CAECsnDSd3ZmSHVeoK7YfiDkfSzSq0gRWXrYh5X2j=tMXC1gdKQ@mail.gmail.com">
<div dir="ltr">Are there plans to encapsulate offset/page boundary
computation in FileChannel.map for files under a hugetlbfs?
<div><br>
</div>
<div>The problem is that, unlike files in "regular" file systems
with 4K pages, <b>ftruncate</b> can only be called with
multiples of page size, otherwise it will fail and set errno
to EINVAL, which is translated in the JNI land into an
IOException with a not very informative message: "Invalid
argument".</div>
<div><br>
</div>
<div>With hugetlbfs, we also have to manually compute page
positions, since FileDispatcher::allocationGranularity() will
"always" report 4K and we have to call <b>mmap</b> with, e.g.
2M page alignment:</div>
<div><br>
</div>
<div>----------</div>
<div>import static java.nio.file.StandardOpenOption.*;<br>
<br>
import java.io.*;<br>
import java.lang.foreign.*;<br>
import java.nio.channels.*;<br>
import java.nio.channels.FileChannel.*;<br>
import java.nio.file.*;<br>
<br>
public class TestHugeTLBFS {<br>
<br>
static final long REGULAR_PS = 4096L;<br>
static final long HUGE_PS = 1024 * 1024 * 2L;<br>
static final StandardOpenOption[] OPTS = { CREATE, WRITE,
READ };<br>
<br>
public static void main(String[] args) throws IOException {<br>
var base =
Paths.get("/var/lib/hugetlbfs/global/pagesize-2MB");<br>
<br>
var tmp = Files.createTempFile(base, "test", ".bin");<br>
<br>
var a = Arena.ofShared();<br>
<br>
MemorySegment ms = null;<br>
<br>
try (var fc = FileChannel.open(tmp, OPTS)) {<br>
// fails in truncate (length is not a multiple of
HUGE_PS)<br>
ms = fc.map(MapMode.READ_WRITE, 0, HUGE_PS - 1, a);<br>
} catch (IOException e) {<br>
e.printStackTrace();<br>
}<br>
<br>
assert ms == null : "Worked?!";<br>
<br>
try (var fc = FileChannel.open(tmp, OPTS)) {<br>
// fails in mmap (computed offset is 4K aligned, but
it's not 2MB aligned)<br>
ms = fc.map(MapMode.READ_WRITE, HUGE_PS + 3 *
REGULAR_PS, HUGE_PS - 3 * REGULAR_PS, a);<br>
} catch (IOException e) {<br>
e.printStackTrace();<br>
}<br>
<br>
assert ms == null : "Worked?!";<br>
<br>
try (var fc = FileChannel.open(tmp, OPTS)) {</div>
<div> // This works, because the aligned offset ends up 2MB
aligned</div>
<div> ms = fc.map(MapMode.READ_WRITE, HUGE_PS + 19, HUGE_PS
- 19, a);<br>
} catch (IOException e) {<br>
throw new UncheckedIOException(e);<br>
}</div>
<div><br>
ms.set(ValueLayout.JAVA_INT_UNALIGNED, 0, 42);<br>
a.close();<br>
<br>
try (var fc = FileChannel.open(tmp, OPTS)) {</div>
<div> ms = fc.map(MapMode.READ_WRITE, HUGE_PS + 19 ,
HUGE_PS - 19, a = Arena.ofShared());</div>
<div> } catch (IOException e) {<br>
throw new UncheckedIOException(e);<br>
}<br>
<br>
assert ms.get(ValueLayout.JAVA_INT_UNALIGNED, 0) == 42;<br>
a.close();<br>
}<br>
}<br>
</div>
<div>----------<br>
</div>
<div><br>
</div>
<div>To satisfy the alignment constraints for both offset and
length, the offset has to be rounded down to the beginning of
a page boundary and the length must be compensated taking into
account the aligned start offset, more or less like:</div>
<div><br>
</div>
<div>MemorySegment map(Path path, long offset, long length, long
pageSize) {<br>
var start = offset;<br>
var len = length;<br>
var truncLen = start + len;<br>
<br>
if (truncLen % pageSize != 0) {<br>
// round down to start of a page offset -> offset -
(offset % pageSize)<br>
start = alignDown(offset, pageSize);<br>
var end = offset + length;<br>
len = end - start;<br>
truncLen = start + len;<br>
<br>
if (truncLen % pageSize != 0) {<br>
// round up to a multiple of pageSize: length ->
pageSize * (length / pageSize + ((length % pageSize == 0) ? 0
: 1))<br>
len = alignUp(len, pageSize);<br>
truncLen = start + len;<br>
<br>
assert truncLen % pageSize == 0 : "Sanity";<br>
}<br>
}<br>
<br>
try (var fc = FileChannel.open(path,...)) {</div>
<div> var segment = fc.map(mode, path, start, len,
Arena.ofSomething());<br>
<br>
if (start != offset || len != length) {<br>
segment = segment.asSlice(offset - start, length);<br>
}<br>
<br>
return segment;<br>
}<br>
}<br>
</div>
<div><br>
</div>
<div>It would be nice to have this somehow handled by
FileChannel. </div>
<div><br>
</div>
<div>An overload with a user-defined page size might not be
ideal, but it might be the cheapest way to bypass the
(incorrect) FileDispatcher::allocationGranularity() value to
enforce correct alignment constraints.</div>
<div><br>
</div>
<div>Regards</div>
</div>
</blockquote>
</body>
</html>