<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<br>
<br>
<div class="moz-cite-prefix">On 12/09/2023 18:05,
<a class="moz-txt-link-abbreviated" href="mailto:github@paul.kungfoocoder.org">github@paul.kungfoocoder.org</a> wrote:<br>
</div>
<blockquote type="cite" cite="mid:D3EFDB1F-13D8-4633-9831-65A998554FF8@paul.kungfoocoder.org">
<div style="display: block;">Recently we came across a very
surprising situation.</div>
<div style="display: block;"><br>
</div>
<div style="display: block;">In our code we were doing something
similar to:</div>
<div style="display: block;"><br>
</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding:
0px;">
<div style="display: block;">echo 'new
java.io.ByteArrayInputStream(new
byte[256*1024*1024]).transferTo(new
java.io.FileOutputStream("/tmp/aaa"));' | jshell
-R-XX:MaxDirectMemorySize=25M</div>
</blockquote>
<div style="display: block;"><br>
</div>
<div style="display: block;">Which works perfectly well. For
various reasons some of the code was adapted to use Paths,
instead of Files, and this was changed to (effectively):</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding:
0px;">
<div style="display: block;"><br>
</div>
<div style="display: block;">echo 'java.nio.file.Files.copy(new
java.io.ByteArrayInputStream(new byte[256*1024*1024]),
java.nio.file.Path.of("/tmp/aaa"),
java.nio.file.StandardCopyOption.REPLACE_EXISTING);' | jshell
-R-XX:MaxDirectMemorySize=25M</div>
</blockquote>
<div style="display: block;"><br>
</div>
<div style="display: block;">Whereupon it started to fail with a
message similar to:</div>
<div style="display: block;">
<pre class="notranslate" style="box-sizing: border-box; font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 11.9px; margin-top: 0px; margin-bottom: 16px; overflow-wrap: normal; padding: 16px; overflow: auto; line-height: 1.45; color: rgb(31, 35, 40); border-radius: 6px; caret-color: rgb(31, 35, 40);"><code class="notranslate" style="box-sizing: border-box; font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 11.9px; padding: 0px; margin: 0px; background: none; border-radius: 6px; word-break: normal; border: 0px; display: inline; overflow: visible; line-height: inherit; overflow-wrap: normal;">Exception java.lang.OutOfMemoryError: Cannot reserve 268435456 bytes of direct buffer memory (allocated: 127, limit: 26214400)</code></pre>
</div>
<div style="display: block;"><span style="caret-color: rgb(31, 35,
40); color: rgb(31, 35, 40); font-family: -apple-system,
BlinkMacSystemFont, "Segoe UI", "Noto
Sans", Helvetica, Arial, sans-serif, "Apple Color
Emoji", "Segoe UI Emoji"; font-size: 14px;
background-color: rgb(255, 255, 255);">Note that this isn't
strictly a problem of </span><code class="notranslate" style="box-sizing: border-box; font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 11.9px; padding: 0.2em 0.4em; margin: 0px; white-space: break-spaces; border-radius: 6px; caret-color: rgb(31, 35, 40); color: rgb(31, 35, 40);">Files.copy</code><span style="caret-color: rgb(31, 35, 40); color: rgb(31, 35, 40);
font-family: -apple-system, BlinkMacSystemFont, "Segoe
UI", "Noto Sans", Helvetica, Arial, sans-serif,
"Apple Color Emoji", "Segoe UI Emoji";
font-size: 14px; background-color: rgb(255, 255, 255);">, but
rather of the output stream returned by</span><code class="notranslate" style="box-sizing: border-box; font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 11.9px; padding: 0.2em 0.4em; margin: 0px; white-space: break-spaces; border-radius: 6px; caret-color: rgb(31, 35, 40); color: rgb(31, 35, 40);">Files.newOutputStream</code><span style="caret-color: rgb(31, 35, 40); color: rgb(31, 35, 40);
font-family: -apple-system, BlinkMacSystemFont, "Segoe
UI", "Noto Sans", Helvetica, Arial, sans-serif,
"Apple Color Emoji", "Segoe UI Emoji";
font-size: 14px; background-color: rgb(255, 255, 255);">, as
evidenced by </span><code class="notranslate" style="box-sizing: border-box; font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 11.9px; padding: 0.2em 0.4em; margin: 0px; white-space: break-spaces; border-radius: 6px; caret-color: rgb(31, 35, 40); color: rgb(31, 35, 40);">echo 'new java.io.ByteArrayInputStream(new byte[256*1024*1024]).transferTo(java.nio.file.Files.newOutputStream(java.nio.file.Path.of("/tmp/aaa")));' | jshell -R-XX:MaxDirectMemorySize=25M</code><span style="caret-color: rgb(31, 35, 40); color: rgb(31, 35, 40); font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 11.9px; white-space: break-spaces;"> </span></div>
<br>
</blockquote>
Right, it's ChannelOutputStream.writeFully that doesn't limit the
size of the writes to the underlying FileChannel. Improving that
would reduce direct buffer usage for several operations (not just
copy).<br>
<br>
-Alan<br>
</body>
</html>