<div dir="auto">Hi!<div dir="auto">The least scary hack I've seen so far is libdlbind [1] which depends on /run/shm instead of /tmp. I am not sure how portable is /run/shm.</div><div dir="auto">I suppose a similar trick may be possible in Windows with shared memory and the extended path notation.</div><div dir="auto">[1] <a href="https://github.com/stephenrkell/libdlbind">https://github.com/stephenrkell/libdlbind</a></div><div dir="auto">Atte.</div><div dir="auto">Pedro.</div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">Em qui., 16 de jan. de 2025, 19:33, Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com">maurizio.cimadamore@oracle.com</a>> escreveu:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u>
<div>
<p>Yes, I was wondering about creating a new descriptor with
memfd_create, copy your library in there, and then load that (with
plain dlopen). Seems a bit hacky, and I'm sure there will be rough
edges - I'm mostly curious if anyone has gave that a try - if that
sort of works, I believe at least we could share some possible
code on how you would build a custom SymbolLookup using that
trick.<br>
<br>
Maurizio<br>
</p>
<div>On 16/01/2025 22:22, Pedro Lamarão
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">Hi Maurizio!
<div>Unfortunately, AFAIK, there is no fdlopen in Linux land;
dlopen requires a file path.</div>
<div>Windows idem: there is only LoadLibrary which requires a
file path.</div>
<div>Atte.</div>
<div>Pedro.<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">Em qui., 16 de jan. de 2025
às 19:13, Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" rel="noreferrer">maurizio.cimadamore@oracle.com</a>>
escreveu:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hi Chris,<br>
Pedro is correct -- library loading is based on what
capability is provided by the OS (typically, some form
of `dlopen`). In most OS, the argument to dlopen (or
equivalent) must be a file descriptor. This is why
existing workarounds need to dump the packaged native
libraries in /tmp, so that the OS would be able to load
them.</p>
<p>I agree this practice is brittle, and in some cases (as
you point out) access to /tmp is not even available. If
you live in a modular world and you can build your own
JDK image (with jlink), then there's an elegant solution
to this: create a jmod for the Java library. The jmod
file can also contain native libraries that will, once
linked in a modular image, be placed inside the "lib"
folder of the image, so that they will be discoverable
at runtime. This is the way in which, for instance, we
package jextract.</p>
<p>But this is, sadly, not the world the majority of the
world lives in. Build tools like Maven and Gradle don't
understand jlink images and jmod files (nor do IDEs). I
do hope that the status quo will change, and that
adoption of custom images, especially in the case of
projects with native library becomes more popular (but
to do so, popular tools and framework would need to lead
the way).</p>
<p>Failing that, our best bet is that better library
loading facilities become more widely available (like
fdlopen). I know linux has something called memfd_create
[1], which allows to create a file descriptor from
shared memory. I have not tried to see if it can be used
to load libraries directly from memory, but we'd be
curious to know if anybody did, and what kind of
limitations would derive from doing things that way.</p>
<p>Cheers<br>
Maurizio<br>
<br>
[1] - <a href="https://urldefense.com/v3/__https://man7.org/linux/man-pages/man2/memfd_create.2.html__;!!ACWV5N9M2RV99hQ!Orwl--LGacndIiYWLY7o6IDVtYaHQej62Z7LIjXDGhN_pVICLeqZkVi3eJ1LwNzoC2YlYLebXK2wfXGbLcG8HyvBOZTqXbSAGps$" target="_blank" rel="noreferrer">https://man7.org/linux/man-pages/man2/memfd_create.2.html</a><br>
</p>
<br>
<div>On 16/01/2025 18:54, Pedro Lamarão wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">Hi Chris!
<div>as a way forward, see for example
FreeBSD's fdlopen:</div>
<div><a href="https://urldefense.com/v3/__https://man.freebsd.org/cgi/man.cgi?fdlopen(3)__;!!ACWV5N9M2RV99hQ!Orwl--LGacndIiYWLY7o6IDVtYaHQej62Z7LIjXDGhN_pVICLeqZkVi3eJ1LwNzoC2YlYLebXK2wfXGbLcG8HyvBOZTqc3nYkJI$" target="_blank" rel="noreferrer">https://man.freebsd.org/cgi/man.cgi?fdlopen(3)</a></div>
<div>Generally, we need the dynamic loader to allow
loading from alternative storage.</div>
<div>Atte.</div>
<div>Pedro.</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">Em qui., 16 de
jan. de 2025 às 15:48, Chris Vest <<a href="mailto:mr.chrisvest@gmail.com" target="_blank" rel="noreferrer">mr.chrisvest@gmail.com</a>>
escreveu:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Hi,
<div><br>
</div>
<div>SymbolLookup and the older
System.load/loadLibrary all make the
assumption that any library we wish to load is
located in a file on the, er, "native" file
system.</div>
<div><br>
</div>
<div>I guess this is a limitation inherited from
dlopen and LoadLibrary.</div>
<div><br>
</div>
<div>Many Java projects that include native
code, then tend to package their native
libraries as part of their jar files, and
extract them to /tmp, or the like, before
loading them.</div>
<div><br>
</div>
<div>This old practice is starting to become
problematic, since /tmp is now often mounted
with noexec for security.</div>
<div><br>
</div>
<div>So, is there some possible path forward
that'll allow us to load libraries from e.g. a
memory segment instead of a file?</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Chris</div>
</div>
</blockquote>
</div>
<div><br>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
<div><br>
</div>
</div>
</blockquote>
</div>
</blockquote></div>