<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 11/10/2024 14:46, Владимир Козелков
wrote:<br>
</div>
<blockquote type="cite" cite="mid:CAGF0iuSgp7ZraitHyboZdDkeAnBc3AL8VbpW6EmHV7+txpHagQ@mail.gmail.com">
<div dir="ltr">Thanks for the answer.<br>
<div><br>
</div>
<div>At the moment, support on 64-bit architectures, their
32-bit variants are very difficult, and I see several problems
with this.<br>
</div>
<div><br>
</div>
<div>It seems to me that the main problem is in considering
addresses outside of Linker and the existence of the
ValueLayout.ADDRESS constant. All ValueLayout.JAVA_* constants
have the same size, alignment and byte order on all platforms
- this is determined by the Java platform itself. All native
layouts are inside Linker.canonicalLayouts(), except for
addresses (which are <b>always </b>platform-dependent). Why?<br>
</div>
</div>
</blockquote>
<p>Well, I see what you say, but there is such a thing as "the
natural address layout in a given platform". This thing pops up
frequently enough (what is the size of a pointer?) which makes
sense to give it a more direct exposure.</p>
<p>That said, the linker also exposes an ABI-dependent canonical
layout for `void*`. So, if we added support for multiple linkers
in same platform, you would need to get the correct canonical
linker for "void*" for the particular ABI used.</p>
<p>In that sense, `ValueLayout.ADDRESS` can be thought of/rectonned
as `Linker.nativeLinker(defaultABI()).canonicalLayout("void*")`.</p>
<p>I don't see a lot of issues with this approach.<br>
</p>
<blockquote type="cite" cite="mid:CAGF0iuSgp7ZraitHyboZdDkeAnBc3AL8VbpW6EmHV7+txpHagQ@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>If we really want to support multiple calling conventions
for ABIs with different bit depths (and this is the most
common case of different ABIs on the same platform), we will
also need to add support for AddressLayouts not only of
different alignments, but also of different sizes, which will
require non-trivial handling in VarHandles and some other
places. In this case, we also need to say that
ValueLayout.ADDRESS refers to Linker.nativeLayout(), but there
could be others...<br>
</div>
</div>
</blockquote>
See above. The reality will be that in 99% of cases,
`ValueLayout.ADDRESS` will be fine (and what the user really mean).
If you need more (e.g. interop) then use a canonical layout, not
`ADDRESS`.<br>
<blockquote type="cite" cite="mid:CAGF0iuSgp7ZraitHyboZdDkeAnBc3AL8VbpW6EmHV7+txpHagQ@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>Unfortunately, there are problems not only with layouts,
but also with memory segments. 32-bit ABIs only support 32-bit
addresses, as funny as it may sound. So standard memory
segments are unlikely to be used with 32-bit ABIs - you need a
linker-dependent way to allocate memory, for example only in
the first four gigabytes of process memory (I know for sure
that Linux supports this)<br>
</div>
</div>
</blockquote>
This is an interesting point - e.g. memory allocated for a 32-bit
application needs to be put in a certain part of the addressing
space. That said, while this might not be supported out of the box,
it might be fairly easy to just wrap an OS-specific allocation
library using the Linker, and then wrap an Arena around that.<br>
<blockquote type="cite" cite="mid:CAGF0iuSgp7ZraitHyboZdDkeAnBc3AL8VbpW6EmHV7+txpHagQ@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>All of this needs to be carefully thought out and reflected
in the documentation, which can require a lot of work. This
seems like a pretty big and radical change, but it is
possible.<br>
</div>
</div>
</blockquote>
<p>I don't see major API roadblocks to get there (and, indeed, we
have worked through these details in the past, to make sure that
was the case). I agree it's a lot of work, and that is the main
reason (coupled with the fact that, at least for now, the return
on investment doesn't seem super high) why it was left out in the
initial release.</p>
<p>With my project management hat on (is that a Panama hat? :-) ),
there are several interesting problems competing for our
attention. Some stuff in our radar:<br>
</p>
<p>* better access to structured data (e.g. reading a struct into a
record);<br>
* have more arena options - for instance an `Arena` that supports
structured confinement (a la `StructuredTaskScope`). This would be
an ideal middle ground between `ofConfined` and `ofShared`;<br>
* support for more efficient allocation strategies (e.g.
allocation pools etc.);<br>
* having a better story to distribute Java libraries that depend
on native libraries.</p>
<p>For now, supporting alternative ABIs on the same platform doesn't
strike me as having quite the same impact as some of the items in
the above list (some of which are more widely applicable than
"just" FFI). But, as I said, of course we'll keep monitoring this
space, and bump priorities as appropriate.<br>
</p>
<p>Finally, note that supporting alternate ABIs is not just about
API design: a lot also depends on how much the community is
willing to take up the effort to actually write and maintain such
cross-platform linker implementations. It is simply not fair nor
realistic to expect that Oracle will provide (and support) all
these niche linkers forever.</p>
<p>Cheers<br>
Maurizio<br>
</p>
<blockquote type="cite" cite="mid:CAGF0iuSgp7ZraitHyboZdDkeAnBc3AL8VbpW6EmHV7+txpHagQ@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>Cheers<br>
</div>
<div>Vladimir</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">пт, 11 окт. 2024 г. в 16:46,
Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hi,<br>
Having the ability to select different calling conventions
(or, more accurately, completely different ABIs) is a
powerful trick. It comes in especially handy in cases that
I'd call foreign^2 - that is, when you want to talk to
some native function that adopts calling conventions that
are not first-class on that particular system. I view x86
on x64 and x64 on arm64 as largely similar in spirit.</p>
<p>That is in contrast, IMHO with the situation we had with
x86 - where multiple competing calling conventions often
existed within the same system (sometimes with the intent
of providing better performances in certain contexts).
Windows x86 supports six (!!) calling conventions [1]. By
contrast, on Windows x64 there's only two (__vectorcall is
apparently still around, although I don't know how widely
used). Other platforms followed a similar evolution.</p>
<p>The cross-architecture-compatibility use case you mention
is an emerging important one, so we will keep an eye in
this space for sure.</p>
<p>Maurizio</p>
<p>[1] -
<a href="https://urldefense.com/v3/__https://learn.microsoft.com/en-us/cpp/cpp/argument-passing-and-naming-conventions?view=msvc-170__;!!ACWV5N9M2RV99hQ!KTTYOdNHrZQKf2LQl0EkQ5uc-pLqe2rRwjbtWubWx3AT_PRWk-AxSBGTNDk_IzLAFtxor15S_rICwo90JYNPbvPzKrQEUw$" target="_blank" moz-do-not-send="true">https://learn.microsoft.com/en-us/cpp/cpp/argument-passing-and-naming-conventions?view=msvc-170</a></p>
<div>On 11/10/2024 01:48, Владимир Козелков wrote:<br>
</div>
<blockquote type="cite">
<p dir="ltr">I think the main use of alternative linkers
is to reflect the existing ability of systems to run
binaries from other platforms. </p>
<p dir="ltr">In my example, it was possible to use old
binaries for 32-bin systems on 64-bit systems. But
platforms are not limited to this. You were wrong when
you said about the unified calling convention on new
architectures - just look at the ARM64EC calling
convention - it allows an application to have both
aarch64 and x86_64 binaries in the process!</p>
<p dir="ltr">Also... I'm confused by the existence of Wine
on Linux - it provides a platform for running binaries
of the same architecture, but of a different operating
system (Windows). Unfortunately, I don't know if it has
the ability to have a process with mixed binaries and
how this relates to Java, but this is also an
interesting example. </p>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">пт, 11 окт. 2024 г.,
4:04 Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
as you noticed, while the Linker javadoc alludes at
the fact that there <br>
might be other calling conventions supported in the
future, at the <br>
moment there's no API to expose this. What we had in
mind the last time <br>
we discussed this was not too dissimilar to what you
propose here - <br>
basically just keep calling convention open, by using
strings, and then <br>
allow the "nativeLinker" factory to accept a calling
convention string.<br>
<br>
Another possibility would be to use linker options -
e.g. have a <br>
CallingConvention linker option that can be passed to
<br>
downcallHandle/upcallStub. This would allow to keep a
single linker, but <br>
to support downcalls with different calling
conventions. Both approaches <br>
are equally expressive, at least in terms of allowing
to call functions <br>
using different argument shuffling. That said, on some
platforms, like <br>
PowerPC support for instance different kind of
endianness. So perhaps it <br>
would be good to have a way to ask for the "big
endian" Linker, whose <br>
canonical layouts will be... big endian. That is, a
Linker is about <br>
functions as much as it is about the definition of
fundamental data <br>
types. So, perhaps when adding support for different
Linker "flavors" it <br>
would be good to keep this in mind.<br>
<br>
The reason we left this out in 22 was that we wanted
to learn more use <br>
cases where this was useful. For instance, while it's
true that x86 <br>
supported several calling conventions, modern systems
seems to have <br>
evolved a bit, so that each major platform tend to
gravitate towards one <br>
main set of calling convention, typically specified in
that platform's <br>
ABI (e.g. SysV for Linux). It seems to me that even in
your case, the <br>
main driver for selecting an alternate calling
convention is x86 really. <br>
So I'm still not 100% sure that this is something
worth pursuing. I <br>
would feel more at ease if we had more cases where
this was useful.<br>
<br>
Cheers<br>
Maurizio<br>
<br>
<br>
On 10/10/2024 20:14, Владимир Козелков wrote:<br>
> Greetings,<br>
><br>
> The documentation for the Linker.nativeLinker()
method says: "It is <br>
> not currently possible to obtain a linker for a
different combination <br>
> of OS and processor."<br>
><br>
> This is indeed true for hotspot, but what if
another implementation <br>
> could provide the ability to create a linker for
a different calling <br>
> convention? Even if the implementation wanted to
do this, it would <br>
> fail because the API does not provide any points
through which this <br>
> could be done.<br>
><br>
> As an example - android allows us to use binaries
for arm in aarch64 <br>
> and for x86 in x86_64 with JNI. In the current
implementation, I have <br>
> to filter the output of
SymbolLookup.loaderLookup() so that the user <br>
> does not get symbols with a different calling
convention, although the <br>
> platform really allows to use them.<br>
><br>
> Additionally, I would like to note that the x86
and x86_64 platforms <br>
> have several "native" calling conventions, such
as cdecl (which is <br>
> actually used now), fastcall, vectorcall,
etc. Even if a hotspot does <br>
> not allow these calling conventions, it would be
useful to have at <br>
> least the potential to implement them.<br>
><br>
> I can suggest a not very good and naive method
for solving the problem <br>
> - it is inspired by target-triple from LLVM:<br>
><br>
> interface Linker ... {<br>
> static List<String>
supportedConventions() {return ... ;}<br>
> static String defaultConvention() {return ...
;}<br>
> static boolean isSupportedConvention(String
convention) {return ... ;}<br>
> static Linker linkerForConvention(String
convention) {return ... ;}<br>
> static Linker nativeLinker() {<br>
> return
linkerForConvention(defaultConvention());<br>
> }<br>
> }<br>
><br>
> For android aarch64 defaultConvention() will
return something like <br>
> "aarch64-android-cdecl"<br>
><br>
> Thanks for reading<br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</body>
</html>