Provide API points for implementing linkers with non-standard calling conventions
Jorn Vernee
jorn.vernee at oracle.com
Fri Oct 11 14:40:53 UTC 2024
As far as I'm aware, there are no implementations that allow you to load
a 32-bit library into a 64-bit process (and vice versa of course).
Though, it is possible to run 32-bit processes on 64-bit machines
through emulation (e.g. Wow64 on Windows). Do you have an example of
loading 32-bit libraries into 64-bit processes?
Jorn
On 11-10-2024 15:46, Владимир Козелков wrote:
> Thanks for the answer.
>
> At the moment, support on 64-bit architectures, their 32-bit variants
> are very difficult, and I see several problems with this.
>
> It seems to me that the main problem is in considering addresses
> outside of Linker and the existence of the ValueLayout.ADDRESS
> constant. All ValueLayout.JAVA_* constants have the same size,
> alignment and byte order on all platforms - this is determined by the
> Java platform itself. All native layouts are inside
> Linker.canonicalLayouts(), except for addresses (which are *always
> *platform-dependent). Why?
>
> If we really want to support multiple calling conventions for ABIs
> with different bit depths (and this is the most common case of
> different ABIs on the same platform), we will also need to add support
> for AddressLayouts not only of different alignments, but also of
> different sizes, which will require non-trivial handling in VarHandles
> and some other places. In this case, we also need to say that
> ValueLayout.ADDRESS refers to Linker.nativeLayout(), but there could
> be others...
>
> Unfortunately, there are problems not only with layouts, but also with
> memory segments. 32-bit ABIs only support 32-bit addresses, as funny
> as it may sound. So standard memory segments are unlikely to be used
> with 32-bit ABIs - you need a linker-dependent way to allocate memory,
> for example only in the first four gigabytes of process memory (I know
> for sure that Linux supports this)
>
> All of this needs to be carefully thought out and reflected in the
> documentation, which can require a lot of work. This seems like a
> pretty big and radical change, but it is possible.
>
> Cheers
> Vladimir
>
>
> пт, 11 окт. 2024 г. в 16:46, Maurizio Cimadamore
> <maurizio.cimadamore at oracle.com>:
>
> Hi,
> Having the ability to select different calling conventions (or,
> more accurately, completely different ABIs) is a powerful trick.
> It comes in especially handy in cases that I'd call foreign^2 -
> that is, when you want to talk to some native function that adopts
> calling conventions that are not first-class on that particular
> system. I view x86 on x64 and x64 on arm64 as largely similar in
> spirit.
>
> That is in contrast, IMHO with the situation we had with x86 -
> where multiple competing calling conventions often existed within
> the same system (sometimes with the intent of providing better
> performances in certain contexts). Windows x86 supports six (!!)
> calling conventions [1]. By contrast, on Windows x64 there's only
> two (__vectorcall is apparently still around, although I don't
> know how widely used). Other platforms followed a similar evolution.
>
> The cross-architecture-compatibility use case you mention is an
> emerging important one, so we will keep an eye in this space for sure.
>
> Maurizio
>
> [1] -
> https://learn.microsoft.com/en-us/cpp/cpp/argument-passing-and-naming-conventions?view=msvc-170
>
> On 11/10/2024 01:48, Владимир Козелков wrote:
>>
>> I think the main use of alternative linkers is to reflect the
>> existing ability of systems to run binaries from other platforms.
>>
>> In my example, it was possible to use old binaries for 32-bin
>> systems on 64-bit systems. But platforms are not limited to this.
>> You were wrong when you said about the unified calling convention
>> on new architectures - just look at the ARM64EC calling
>> convention - it allows an application to have both aarch64 and
>> x86_64 binaries in the process!
>>
>> Also... I'm confused by the existence of Wine on Linux - it
>> provides a platform for running binaries of the same
>> architecture, but of a different operating system (Windows).
>> Unfortunately, I don't know if it has the ability to have a
>> process with mixed binaries and how this relates to Java, but
>> this is also an interesting example.
>>
>>
>> пт, 11 окт. 2024 г., 4:04 Maurizio Cimadamore
>> <maurizio.cimadamore at oracle.com>:
>>
>> Hi,
>> as you noticed, while the Linker javadoc alludes at the fact
>> that there
>> might be other calling conventions supported in the future,
>> at the
>> moment there's no API to expose this. What we had in mind the
>> last time
>> we discussed this was not too dissimilar to what you propose
>> here -
>> basically just keep calling convention open, by using
>> strings, and then
>> allow the "nativeLinker" factory to accept a calling
>> convention string.
>>
>> Another possibility would be to use linker options - e.g. have a
>> CallingConvention linker option that can be passed to
>> downcallHandle/upcallStub. This would allow to keep a single
>> linker, but
>> to support downcalls with different calling conventions. Both
>> approaches
>> are equally expressive, at least in terms of allowing to call
>> functions
>> using different argument shuffling. That said, on some
>> platforms, like
>> PowerPC support for instance different kind of endianness. So
>> perhaps it
>> would be good to have a way to ask for the "big endian"
>> Linker, whose
>> canonical layouts will be... big endian. That is, a Linker is
>> about
>> functions as much as it is about the definition of
>> fundamental data
>> types. So, perhaps when adding support for different Linker
>> "flavors" it
>> would be good to keep this in mind.
>>
>> The reason we left this out in 22 was that we wanted to learn
>> more use
>> cases where this was useful. For instance, while it's true
>> that x86
>> supported several calling conventions, modern systems seems
>> to have
>> evolved a bit, so that each major platform tend to gravitate
>> towards one
>> main set of calling convention, typically specified in that
>> platform's
>> ABI (e.g. SysV for Linux). It seems to me that even in your
>> case, the
>> main driver for selecting an alternate calling convention is
>> x86 really.
>> So I'm still not 100% sure that this is something worth
>> pursuing. I
>> would feel more at ease if we had more cases where this was
>> useful.
>>
>> Cheers
>> Maurizio
>>
>>
>> On 10/10/2024 20:14, Владимир Козелков wrote:
>> > Greetings,
>> >
>> > The documentation for the Linker.nativeLinker() method
>> says: "It is
>> > not currently possible to obtain a linker for a different
>> combination
>> > of OS and processor."
>> >
>> > This is indeed true for hotspot, but what if another
>> implementation
>> > could provide the ability to create a linker for a
>> different calling
>> > convention? Even if the implementation wanted to do this,
>> it would
>> > fail because the API does not provide any points through
>> which this
>> > could be done.
>> >
>> > As an example - android allows us to use binaries for arm
>> in aarch64
>> > and for x86 in x86_64 with JNI. In the current
>> implementation, I have
>> > to filter the output of SymbolLookup.loaderLookup() so that
>> the user
>> > does not get symbols with a different calling convention,
>> although the
>> > platform really allows to use them.
>> >
>> > Additionally, I would like to note that the x86 and x86_64
>> platforms
>> > have several "native" calling conventions, such as cdecl
>> (which is
>> > actually used now), fastcall, vectorcall, etc. Even if a
>> hotspot does
>> > not allow these calling conventions, it would be useful to
>> have at
>> > least the potential to implement them.
>> >
>> > I can suggest a not very good and naive method for solving
>> the problem
>> > - it is inspired by target-triple from LLVM:
>> >
>> > interface Linker ... {
>> > static List<String> supportedConventions() {return ... ;}
>> > static String defaultConvention() {return ... ;}
>> > static boolean isSupportedConvention(String convention)
>> {return ... ;}
>> > static Linker linkerForConvention(String convention)
>> {return ... ;}
>> > static Linker nativeLinker() {
>> > return linkerForConvention(defaultConvention());
>> > }
>> > }
>> >
>> > For android aarch64 defaultConvention() will return
>> something like
>> > "aarch64-android-cdecl"
>> >
>> > Thanks for reading
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20241011/db303409/attachment-0001.htm>
More information about the panama-dev
mailing list