<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 28/12/2022 15:57, Gavin Ray wrote:<br>
</div>
<blockquote type="cite" cite="mid:CAFtvWZPwfsr2nys8=5f3VSY8e8Upzsde9o7qwWpG9W7ic3Eo9g@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">Ah, I was afraid the answer might be something
like that
<div><br>
</div>
<div>My interest was in trying to model the types of Postgres
Wire Protocol as MemoryLayout types</div>
<div>Unfortunately, several of the types have variable-length
strength fields in them</div>
<div><br>
</div>
<div>If you ctrl+f here, there are some ~30-ish fields in
total which are "String" null-terminated C-string types =/</div>
<div><a href="https://urldefense.com/v3/__https://www.postgresql.org/docs/current/protocol-message-formats.html__;!!ACWV5N9M2RV99hQ!LrohIHwbfxnFmvqcqXjpDsATdrYOHZgc2uhIPKbc1MdBIEkccH2oKCjc8_YD2snFYrcAX0hF-THduftUN1UWTe840Oq3$" moz-do-not-send="true">PostgreSQL: Documentation: 15:
55.7. Message Formats</a><br>
</div>
</div>
</div>
</blockquote>
<p>Yeah - this problem has been explored during some early
explorations.</p>
<p>If you have a message like this:</p>
<p>source : String<br>
destination : String</p>
<p>And both strings are variable-length - how big is the size of the
entire message?</p>
<p>You'll quickly discover that layouts are not that helpful when
you start working with things like these.</p>
<p>Let's say we add another field:</p>
<p>source : String<br>
destination : String<br>
timestamp : Long<br>
</p>
<p>What is the offset of the "timestamp" field? Well, it depends on
the length of the first string *and* on the length of the second
string.</p>
<p>Both lengths are buried _somewhere_ inside the string. If you are
lucky, as in ProtoBuf, the length is encoded as part of the
string. If you are unlucky, as it seems the case for Postgre, you
have to scan both strings to figure out what the offset is!</p>
<p>Needless to say, this kind of operation is already very far from
what a VarHandle can do. Assuming we'd even implement such a
monstruosity, how would we go about providing atomic access, etc?</p>
<p>The only solution that's kind of sensible here is to admit that
the string layout has a size hole. So, to access "timestamp" you'd
have to fill (dynamically) two holes (for the sizes of the two
strings).</p>
<p>While this is more doable, it adds complexity to the layout API.
While complexity is not bad per se, using a dummy layout with a
size hole seems a bit of roundabout way to do things. That is, the
crux of this issue is that you don't know the layout, or better,
that the layout depends on the data you want to pass (dependent
layouts). My feelings, when looking at this kind of things is that
they always seem to fall outside the sweet spot of the problems
the layout API wants to solve.<br>
</p>
<p></p>
Maurizio<br>
<blockquote type="cite" cite="mid:CAFtvWZPwfsr2nys8=5f3VSY8e8Upzsde9o7qwWpG9W7ic3Eo9g@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">
<div>Of course for these specific fields/types exceptions can
be made but it triggers my OCD, ha</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Dec 28, 2022 at 8:40
AM Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi
Gavin,<br>
while something like that is possible, that would not be the
best way to <br>
use memory layouts. I believe the sweet spot for memory layout
is to <br>
capture data types whose size can be determined statically. <br>
Variable-length data structures don't fall in this case
(another <br>
relatively common case is C struct which end with a
variable-length <br>
array). While it would be possible, with some heroics, to
enhance var <br>
handles to take extra dynamic access coordinates at runtimes,
the <br>
condition under which this would be possible are very limited.<br>
<br>
Perhaps it would be better to understand what you are trying
to achieve <br>
by modelling a C string with a memory layout? Often, passing
strings as <br>
opaque pointer just works. Note also that the FFM API provides
some <br>
helper functions to allocate and dereference NULL terminated
strings - <br>
see MemorySegment::get/setUtf8String, as well as <br>
SegmentAllocator::allocateUtf8String.<br>
<br>
Cheers<br>
Maurizio<br>
<br>
<br>
<br>
On 25/12/2022 19:24, Gavin Ray wrote:<br>
> Is it possible to model C-Strings with MemoryLayouts?<br>
><br>
> I was thinking of using "MemoryLayout.sequenceLayout(0,
C_CHAR)" and <br>
> computing the<br>
> length at runtime, plus manually adding the
null-terminator.<br>
><br>
> Would this work alright, or is there some better way to
do this?<br>
</blockquote>
</div>
</blockquote>
</body>
</html>