<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 10/11/2025 13:05, Liam Miller-Cushon
wrote:<br>
</div>
<blockquote type="cite" cite="mid:CAL4Qsgv4cjgYAasuz+jO4dbJ1GbXUFqubn_OSUk4=AbdPtZAcA@mail.gmail.com">
<div dir="ltr">Thanks Maurizio,
<div><br>
</div>
<div>I think that looks great. It addresses the use-cases I'm
aware of for reading and writing strings of known length,
without null terminators.</div>
<div><br>
I have a few small questions and comments.</div>
<div><br>
</div>
> The number of bytes associated with chars is
platform-dependent: it's 10 bytes on Windows, but 20 bytes on
Linux. By having MemorySegment::getString work on units the only
thing we need to worry about is that we specify the correct
charset when reading the string -- e.g. UTF16 (on Windows) or
UTF32 (on Linux).
<div><br>
</div>
<div>I want to double-check my understanding of that example. Is
the idea that you would read a code unit length from the
struct Foo (5 in the example) and then depending on whether
the API expected bytes or code units, you would have to do one
of the following?</div>
<div><br>
// with a code unit length<br>
getString(0, /* code units */ 5, windows ? UTF16 : UTF32)<br>
<br>
// vs. with a byte length<br>
getString(0, /* bytes */ 5 * (windows ? 2 : 4), windows ?
UTF16 : UTF32)</div>
</div>
</blockquote>
Yes, note that you can do `Linker.canonicalLayouts().get("wchar_t")`
to discover the platform dependent size of `wchar_t` and select the
appropriate charset.<br>
<blockquote type="cite" cite="mid:CAL4Qsgv4cjgYAasuz+jO4dbJ1GbXUFqubn_OSUk4=AbdPtZAcA@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>> Liam said protobuf uses byte lengths. Native strings
probably use unit lengths</div>
</div>
</blockquote>
I hope this comment was not in my doc?<br>
<blockquote type="cite" cite="mid:CAL4Qsgv4cjgYAasuz+jO4dbJ1GbXUFqubn_OSUk4=AbdPtZAcA@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>One of my colleagues working on protobuf pointed out I had
described that imprecisely and offered the following
clarification:<br>
<br>
Its really that Protobuf just solely ever represents strings
as UTF8 in native heap memory (even on windows where the OS
APIs will expect utf16, if someone ever wanted to take use
C++Protobuf strings to/from windows paths, they would actually
have to convert utf8<>utf16 themselves); it means that
byte and unit lengths are always the same for us when stored
in native heap</div>
</div>
</blockquote>
Makes sense -- so units seems a good choice there (and always use
UTF8 on protobuf side).<br>
<blockquote type="cite" cite="mid:CAL4Qsgv4cjgYAasuz+jO4dbJ1GbXUFqubn_OSUk4=AbdPtZAcA@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>Another small clarification - the doc mentions 'compressed
strings', should that be 'compact strings'? (JEP 254 mentions
there was an older 'compressed strings' feature in JDK 6?)</div>
</div>
</blockquote>
Whoops - yes, confused with compressed oops :-)<br>
<blockquote type="cite" cite="mid:CAL4Qsgv4cjgYAasuz+jO4dbJ1GbXUFqubn_OSUk4=AbdPtZAcA@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>And one more thing - the doc discusses string lengths. In
addition to the number of UTF-16 code units in the UTF-16
encoding of the string (String.length), and the number of
unicode characters (codePointCount), it is sometimes desirable
to know the number of code units in the encoded string. It's
possible to do this with getBytes(charset).length, but similar
to the other performance discussions here, it's expensive to
create the array and throw it away, and an implementation
outside the JDK can't benefit from the JDK's fast paths in
StringSupport. In theory there could be a method like
String#getEncodedLength(Charset) that returned the same value
as getBytes(charset).length with better performance. Is that
something you think could be interesting to consider, either
as part of this proposal, or separately?</div>
</div>
</blockquote>
<p>You mean the _byte size_ of the encoded string (rather than
number of code units?)</p>
<p>Something like this might be interesting. That said... if the
charset matches, then creating the segment view, then obtaining
its byte size is O(1) (e.g. no decoding). And if the charset
doesn't match, you'll need to decode anyway -- at which point I'm
not sure the array creation is really the bottleneck?</p>
<p>Maurizio<br>
</p>
<blockquote type="cite" cite="mid:CAL4Qsgv4cjgYAasuz+jO4dbJ1GbXUFqubn_OSUk4=AbdPtZAcA@mail.gmail.com"><br>
<div class="gmail_quote gmail_quote_container">
<div dir="ltr" class="gmail_attr">On Fri, Nov 7, 2025 at 5:40 PM
Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hi,<br>
here's a document that outlines a more general strategy to
read and write strings from/to memory segments:</p>
<p><a href="https://cr.openjdk.org/~mcimadamore/panama/strings_ffm.html" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://cr.openjdk.org/~mcimadamore/panama/strings_ffm.html</a></p>
<p>As outlined in this thread, the enabling primitive for
the writing side is a factory that allows to view a Java
string as a read-only heap memory segment.</p>
<p>We believe the proposed strategy should provide better
usability, while at the same time leaving room for the
implementation to "cut corners", to make performance as
good as possible.</p>
<p>Let us know what you think.</p>
<p>Cheers<br>
Maurizio<br>
</p>
<div>On 05/11/2025 12:45, Liam Miller-Cushon wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Thanks Maurizio, considering all of this
holistically makes sense. I look forward to hearing your
thoughts on it once you've had time to consider the
options :)</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Nov 4, 2025 at
11:21 PM Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p><br>
</p>
<div>On 04/11/2025 15:23, Jorn Vernee wrote:<br>
</div>
<blockquote type="cite">
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div>Do you have thoughts on the best way to
proceed here? Do you think it makes sense
to do incrementally, or would you prefer
to see all of these related changes happen
together under a single issue?<br>
<br>
</div>
</div>
</div>
</blockquote>
<p>I don't have a preference. Since you've already
started a PR for enhancing getString, maybe you
can focus on that for now, and we'll file
followup issues for the others. Splitting things
up might be nice since there's probably some
benchmarking work involved for each. I think the
copy and allocateFrom overload can be done in
one patch though.</p>
</blockquote>
<p>I think I sort of do have a preference :-)</p>
<p>My feeling is that we're dealing with 2-3
different APIs that are all tightly
interconnected. We have some ideas on how to solve
some (I think getString with explicit length seems
the most settled), but we're still playing with
other ideas for the others (like copy, or memory
segment views). And we also have to think about
relationship with SegmentAllocator.</p>
<p>For this reason, I'd prefer to think about it
more holistically and think about all the related
APIs at once.</p>
<p>Once we get a consensus on how to proceed we can
decide whether to pursue them all in a single PR,
or split them into separate PRs.</p>
<p>But it would be unfortunate, I think, if the
first PR would later reveal to be a dead end for
the other use cases.</p>
<p>I plan to think about this some more (but I need
some more time).</p>
<p>Cheers<br>
Maurizio<br>
</p>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</body>
</html>