<div dir="ltr"><div>At the risk of repeating my <a href="https://mail.openjdk.org/pipermail/core-libs-dev/2025-March/141871.html">previous comment</a>, I agree with Chen.</div><div><br></div><div>That is to say, there is a separate, more fundamental unsolved problem lurking underneath this discussion, and the two problem "layers" are perhaps better addressed separately.</div><div><br></div><div>Once the lower layer problem is properly framed and resolved, it becomes reusable, and wrapping it to solve various higher-layer problems is easy.</div><div><div><br></div><div>An internal class would be a reasonable and conservative way to start. There could even be a suite of such classes, built from templates a la X-Buffer.java.template.</div><div><br></div></div><div>These could be used all over the place (e.g., refactor StringBuilder). For example, I wonder how much the performance of e.g. ArrayList could be improved in scenarios where you are building (or removing elements from) large lists?</div><br><div>Just thinking out loud (apologies)... Define a "segmented array allocator" as an in-memory byte[] array builder that "chunks" the data into individual segments of size at most N.</div><div><br></div><div>We can think of the current ByteArrayOutputStream as such a thing with N = 2³² that is, there's only ever one "chunk".</div><div><br></div><div>The assertion is that N = 2³² is not the most efficient value. And obviously neither is N = 1.</div><div><br></div><div>So somewhere in the middle there is an optimal value for N, which presumably could be discovered via experimentation. It may be different for different architectures.</div><div><br></div><div>Another parameter would be: What is the size M ≤ N of a new chunk? E.g. you could start with M = 16 and then the chunk grows exponentially until it reaches N, at which point you start a new chunk. The optimal value for M could also be performance tested (it may already have been).</div><div><br></div><div>Of course, for performance optimization we'd need some distribution of array sizes that models "typical" use, etc.</div><div><br></div><div>-Archie</div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Wed, Apr 9, 2025 at 6:19 PM Chen Liang <<a href="mailto:liangchenblue@gmail.com">liangchenblue@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi John Engebretson,<div>I still wonder if we can make the byte array allocator a utility to the JDK, at least an internal one. I find that besides replacing BAOS uses, it can also optimize users like InputStream.readNBytes, BufWriterImpl of classfile, and maybe many more usages. Such an internal addition may be accepted to the JDK immediately because it has no compatibility impact and does not need to undergo CSR review.</div><div><br></div><div>Chen Liang</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Apr 9, 2025 at 11:35 AM Engebretson, John <<a href="mailto:jengebr@amazon.com" target="_blank">jengebr@amazon.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>
<div lang="EN-US">
<div>
<p class="MsoNormal" style="margin-left:0.5in">I think there is agreement that ByteArrayOutputStream.unsynchronized(int cap) (or some such method) would be useful. It would not require using a contiguous byte[] as the backing array so there is scope to experiment
with implementations that don't need to resize like the base BAOS does.<br>
<br>
Extending BOAS means the size is limited to an int. To go beyond would be a different API and there is lots of scope to do experiments. I think too early to create a CSR for MOS, assuming this is what you mean.<br>
<br>
<u></u><u></u></p>
<p class="MsoNormal"> Thank you Alan! PR [1] is updated with the factory method as suggested; MOS is a package-private subclass of BAOS and the Javadoc in BAOS makes no promises about the implementation returned. I allowed MOS to grow beyond 2GB and throw
an intelligent error if it cannot service a particular call (toByteArray, size, etc.). Performance is unchanged. Feedback appreciated.<u></u><u></u></p>
<p class="MsoNormal"> Thanks! <span style="font-family:"Segoe UI Emoji",sans-serif">
😊</span><u></u><u></u></p>
<p class="MsoNormal"> John<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">[1] <a href="https://github.com/openjdk/jdk/pull/24232" target="_blank">https://github.com/openjdk/jdk/pull/24232</a><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div></blockquote></div>
</blockquote></div><div><br clear="all"></div><br><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature">Archie L. Cobbs<br></div>