<div dir="auto">Hi!<div dir="auto">The least scary hack I've seen so far is libdlbind [1] which depends on /run/shm instead of /tmp. I am not sure how portable is /run/shm.</div><div dir="auto">I suppose a similar trick may be possible in Windows with shared memory and the extended path notation.</div><div dir="auto">[1] <a href="https://github.com/stephenrkell/libdlbind">https://github.com/stephenrkell/libdlbind</a></div><div dir="auto">Atte.</div><div dir="auto">Pedro.</div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">Em qui., 16 de jan. de 2025, 19:33, Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com">maurizio.cimadamore@oracle.com</a>> escreveu:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u>

  
  <div>
    <p>Yes, I was wondering about creating a new descriptor with
      memfd_create, copy your library in there, and then load that (with
      plain dlopen). Seems a bit hacky, and I'm sure there will be rough
      edges - I'm mostly curious if anyone has gave that a try - if that
      sort of works, I believe at least we could share some possible
      code on how you would build a custom SymbolLookup using that
      trick.<br>
      <br>
      Maurizio<br>
    </p>
    <div>On 16/01/2025 22:22, Pedro Lamarão
      wrote:<br>
    </div>
    <blockquote type="cite">
      
      <div dir="ltr">
        <div dir="ltr">Hi Maurizio!
          <div>Unfortunately, AFAIK, there is no fdlopen in Linux land;
            dlopen requires a file path.</div>
          <div>Windows idem: there is only LoadLibrary which requires a
            file path.</div>
          <div>Atte.</div>
          <div>Pedro.<br>
          </div>
        </div>
        <br>
        <div class="gmail_quote">
          <div dir="ltr" class="gmail_attr">Em qui., 16 de jan. de 2025
            às 19:13, Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" rel="noreferrer">maurizio.cimadamore@oracle.com</a>>
            escreveu:<br>
          </div>
          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
            <div>
              <p>Hi Chris,<br>
                Pedro is correct -- library loading is based on what
                capability is provided by the OS (typically, some form
                of `dlopen`). In most OS, the argument to dlopen (or
                equivalent) must be a file descriptor. This is why
                existing workarounds need to dump the packaged native
                libraries in /tmp, so that the OS would be able to load
                them.</p>
              <p>I agree this practice is brittle, and in some cases (as
                you point out) access to /tmp is not even available. If
                you live in a modular world and you can build your own
                JDK image (with jlink), then there's an elegant solution
                to this: create a jmod for the Java library. The jmod
                file can also contain native libraries that will, once
                linked in a modular image, be placed inside the "lib"
                folder of the image, so that they will be discoverable
                at runtime. This is the way in which, for instance, we
                package jextract.</p>
              <p>But this is, sadly, not the world the majority of the
                world lives in. Build tools like Maven and Gradle don't
                understand jlink images and jmod files (nor do IDEs). I
                do hope that the status quo will change, and that
                adoption of custom images, especially in the case of
                projects with native library becomes more popular (but
                to do so, popular tools and framework would need to lead
                the way).</p>
              <p>Failing that, our best bet is that better library
                loading facilities become more widely available (like
                fdlopen). I know linux has something called memfd_create
                [1], which allows to create a file descriptor from
                shared memory. I have not tried to see if it can be used
                to load libraries directly from memory, but we'd be
                curious to know if anybody did, and what kind of
                limitations would derive from doing things that way.</p>
              <p>Cheers<br>
                Maurizio<br>
                <br>
                [1] - <a href="https://urldefense.com/v3/__https://man7.org/linux/man-pages/man2/memfd_create.2.html__;!!ACWV5N9M2RV99hQ!Orwl--LGacndIiYWLY7o6IDVtYaHQej62Z7LIjXDGhN_pVICLeqZkVi3eJ1LwNzoC2YlYLebXK2wfXGbLcG8HyvBOZTqXbSAGps$" target="_blank" rel="noreferrer">https://man7.org/linux/man-pages/man2/memfd_create.2.html</a><br>
              </p>
              <br>
              <div>On 16/01/2025 18:54, Pedro Lamarão wrote:<br>
              </div>
              <blockquote type="cite">
                <div dir="ltr">
                  <div dir="ltr">Hi Chris!
                    <div>as a way forward, see for example
                      FreeBSD's fdlopen:</div>
                    <div><a href="https://urldefense.com/v3/__https://man.freebsd.org/cgi/man.cgi?fdlopen(3)__;!!ACWV5N9M2RV99hQ!Orwl--LGacndIiYWLY7o6IDVtYaHQej62Z7LIjXDGhN_pVICLeqZkVi3eJ1LwNzoC2YlYLebXK2wfXGbLcG8HyvBOZTqc3nYkJI$" target="_blank" rel="noreferrer">https://man.freebsd.org/cgi/man.cgi?fdlopen(3)</a></div>
                    <div>Generally, we need the dynamic loader to allow
                      loading from alternative storage.</div>
                    <div>Atte.</div>
                    <div>Pedro.</div>
                    <div><br>
                    </div>
                    <div><br>
                    </div>
                  </div>
                  <br>
                  <div class="gmail_quote">
                    <div dir="ltr" class="gmail_attr">Em qui., 16 de
                      jan. de 2025 às 15:48, Chris Vest <<a href="mailto:mr.chrisvest@gmail.com" target="_blank" rel="noreferrer">mr.chrisvest@gmail.com</a>>
                      escreveu:<br>
                    </div>
                    <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                      <div dir="ltr">Hi,
                        <div><br>
                        </div>
                        <div>SymbolLookup and the older
                          System.load/loadLibrary all make the
                          assumption that any library we wish to load is
                          located in a file on the, er, "native" file
                          system.</div>
                        <div><br>
                        </div>
                        <div>I guess this is a limitation inherited from
                          dlopen and LoadLibrary.</div>
                        <div><br>
                        </div>
                        <div>Many Java projects that include native
                          code, then tend to package their native
                          libraries as part of their jar files, and
                          extract them to /tmp, or the like, before
                          loading them.</div>
                        <div><br>
                        </div>
                        <div>This old practice is starting to become
                          problematic, since /tmp is now often mounted
                          with noexec for security.</div>
                        <div><br>
                        </div>
                        <div>So, is there some possible path forward
                          that'll allow us to load libraries from e.g. a
                          memory segment instead of a file?</div>
                        <div><br>
                        </div>
                        <div>Thanks,</div>
                        <div>Chris</div>
                      </div>
                    </blockquote>
                  </div>
                  <div><br>
                  </div>
                </div>
              </blockquote>
            </div>
          </blockquote>
        </div>
        <div><br>
        </div>
      </div>
    </blockquote>
  </div>

</blockquote></div>