Should I free the pointer returned by native method manually?

[Maurizio Cimadamore]

Hi,
yes, the API has changed considerably when it comes to managing the 
lifetime of pointers that have been allocated extrenally (we sometimes 
refer to this use case as "external allocation").

This is covered in this document in more details:

https://cr.openjdk.org/~mcimadamore/panama/why_lifetimes.html

And there's few slides on this topic in my latest JVMLS talk:

https://youtu.be/kUFysMkMS00?feature=shared&t=1718

The main idea is that pointers returned by native libraries are just 
shallow memory segments that have no size and a global scope (we call 
such segments "zero-length memory segments" [1]). In order to use these 
segments correctly, one must give them a size (otherwise dereference 
operations will fail). Then, there are typically two cases:

* the library (callee) is responsible for deallocation - in such case, 
no other operation (other than the resize) is needed;
* the client (caller) is responsible for deallocation - in such case, 
when resizing the segment it is typically useful to also provide a 
suitable Arena (which determines _when_ the segment will be deallocated) 
and a cleanup actiojn (which determines _how_ the segment will be 
deallocated).

In both these case, the unsafe MemorySegment::reinterpret method has to 
be used, which means that the "--enable-native-access" flag must be 
passed on the command line (to avoid the warnings). As a result of these 
changes, we have now consolidated other operations such as turning a 
long address into a memory segment (`MemorySegment::ofAddress`, which 
now simply returns a zero-length memory segment [2]).

Maurizio

[1] - 
https://download.java.net/java/early_access/jdk22/docs/api/java.base/java/lang/foreign/MemorySegment.html#wrapping-addresses
[2] - 
https://download.java.net/java/early_access/jdk22/docs/api/java.base/java/lang/foreign/MemorySegment.html#ofAddress(long)

On 23/01/2024 01:11, Red IO wrote:
> Thanks Maurizio for the clear explanation.
> Since last time I experimented with Panama the api changed a lot.
> I will need to look more into what jextract generates to understand 
> the current api better.
>
> Great regards
> RedIODev
>
> On Mon, Jan 22, 2024, 22:42 Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com> wrote:
>
>
>     On 22/01/2024 20:01, Red IO wrote:
>>     I know that the original question is answered, but how would you
>>     free memory in case the libraries documentation says to use free.
>>     Would you just create a method handle to libc free and call it
>>     with the pointer or is there an api to free native memory?
>
>     If the API says to use free, then you should have a downcall
>     method handle for "free" and call that (using the same technique
>     with MemorySegment::reinterpret I showed in the last email).
>
>     (Note: when using jextract, chances are that "free" will come
>     along for the ride in the extracted classes).
>
>     Maurizio
>
>>     Great regards
>>     RedIODev
>>
>>     On Mon, Jan 22, 2024, 19:36 Maurizio Cimadamore
>>     <maurizio.cimadamore at oracle.com> wrote:
>>
>>         Hi tison,
>>         in this case, the function is returning a pointer. What
>>         happens to that
>>         pointer is 100% library-dependent. In this case, doing `man
>>         getpwnam`
>>         gives us some clues:
>>
>>         > The return value may point to a static area, and may be
>>         overwritten  by
>>         >        subsequent  calls  to  getpwent(3), getpwnam(), or
>>         getpwuid().
>>         > (Do not
>>         >        pass the returned pointer to free(3).)
>>
>>         Basically, the returned pointer points to an area that is
>>         managed by the
>>         library, so you don't have to free.
>>
>>         This means the code you have seems correct to me.
>>
>>         Other libraries will behave differently, for instance they
>>         might have a
>>         pair of functions such as "createFoo" and "destroyFoo", the
>>         former
>>         allocating region of memory, and the latter destroying such
>>         region. In
>>         such cases it might be helpful to associate the return memory
>>         segment to
>>         an existing arena *and* also attach a cleanup function (via
>>         MemorySegment::reinterpret), so that, when the arena is closed,
>>         "destroyFoo" is called on the pointer.
>>
>>         Hope this helps
>>
>>         Maurizio
>>
>>
>>         On 22/01/2024 18:28, tison wrote:
>>         > Here is the code snippet:
>>         >
>>         >      public static void main(String[] args) throws Throwable {
>>         >          final Linker linker = Linker.nativeLinker();
>>         >          final SymbolLookup libc = linker.defaultLookup();
>>         >          final MethodHandle handle =
>>         > linker.downcallHandle(libc.find("getpwnam").orElseThrow(),
>>         > FunctionDescriptor.of(ValueLayout.ADDRESS,
>>         ValueLayout.ADDRESS));
>>         >          try (Arena arena = Arena.ofConfined()) {
>>         >              final MemorySegment passwd = (MemorySegment)
>>         > handle.invoke(arena.allocateUtf8String("tison"));
>>         >              System.out.println("passwd=" + passwd);
>>         >              System.out.println("pw_name=" +
>>         > passwd.reinterpret(Long.MAX_VALUE).get(ValueLayout.ADDRESS,
>>         > 48).reinterpret(Long.MAX_VALUE).getUtf8String(0));
>>         >          }
>>         >      }
>>         >
>>         > I wonder if I should explicitly free passwd that is
>>         returned from a
>>         > native method. If so, how? If not, how the Arena tracks it
>>         and free on
>>         > closed?
>>         >
>>         > Best,
>>         > tison.
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20240123/2900997c/attachment-0001.htm>