<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>Another thing I'd like to add: while "turn a Java object into a
pointer" sounds simple, in reality if you look at the
implementation of JNI global handles, it is not that much
different from the hash map approach, in the sense that objects
are stored into some (global) data structure [1]. Of course,
resolving objects in this storage is simpler, since a handle is a
tagged pointer directly into this data structure. But, this global
data structure is also a very general-purpose storage mechanism.
Depending on the use case, another kind of data structure might be
more suited.<br>
</p>
<p>So, I'd say that using a hash map + integer keys is not as much
of a workaround as you think it is. Since Java objects are managed
by a GC, in reality we'll always need some kind of map/store that
the GC knows about, since it can not track every arbitrary pointer
value floating around in memory or CPU registers.<br>
</p>
<p>Jorn<br>
</p>
<p>[1] :
<a class="moz-txt-link-freetext" href="https://github.com/openjdk/jdk/blob/0c9658446d111ec944f06b7a8a4e3ae7bf53ee8d/src/hotspot/share/gc/shared/oopStorage.cpp#L389-L436">https://github.com/openjdk/jdk/blob/0c9658446d111ec944f06b7a8a4e3ae7bf53ee8d/src/hotspot/share/gc/shared/oopStorage.cpp#L389-L436</a><br>
</p>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 15/02/2023 15:03, Maurizio
Cimadamore wrote:<br>
</div>
<blockquote type="cite" cite="mid:a60fed27-a597-6351-5c03-e6edd47f22fb@oracle.com">
<p><br>
</p>
<div class="moz-cite-prefix">On 15/02/2023 07:34, Manuel
Bleichenbacher wrote:<br>
</div>
<blockquote type="cite" cite="mid:CAA7F5jJN91yEzMn3Lu4gNVYk9NazKHL1+qMoYxcv9StSv+SSFA@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">
<div>Hi Maurizio</div>
<div><br>
</div>
<div>Thanks for the prompt answer.</div>
<div><br>
</div>
<div>I can understand that you are reluctant to introduce a
new concept for a use cases that isn't particularly
frequent.</div>
<div><br>
</div>
<div>From what I read about ScopedValues, I doubt they will
be helpful. In many of the use cases I'm dealing with, the
void* parameter is used in a cross-thread context and has
a lifetime, which is not aligned with a function call or a
similar scope. A thread-local variable is not helpful for
my use cases. Doesn't this imply that ScopedValues won't
help either?</div>
</div>
</div>
</blockquote>
<p>ScopedValues can be inherited across threads, but only if the
threads are constructed/managed by a StructuredTaskScope:</p>
<p><a class="moz-txt-link-freetext" href="https://openjdk.org/jeps/437" moz-do-not-send="true">https://openjdk.org/jeps/437</a></p>
<p>Which might or might not be flexible enough for the particular
use cases you have in mind. But I wanted to make clear that some
well-behaved form of muli-threading is supported by
ScopedValues.<br>
</p>
<blockquote type="cite" cite="mid:CAA7F5jJN91yEzMn3Lu4gNVYk9NazKHL1+qMoYxcv9StSv+SSFA@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">
<div><br>
</div>
<div>Regarding JNI handles and resurrection I don't have
sufficient insight into the implementation to comment on
this. Since it is already possible to bind an object to a
method handle, create an upcall stub for it and passed it
around in native code, it would seem to me that part of
the challenges have already been solved.</div>
</div>
</div>
</blockquote>
<p>The main difference is that when you call an "upcall" there is
always a known class that acts as an execution context for the
code that runs in the upcall. If an upcall binds one or more
objects, these objects will belong to the same context as the
resulting upcall method handle itself, so no problem arises.<br>
</p>
<p>Allowing ordinary Java objects to be turned into JNI handles
which can be stored inside native libraries opens up a scenario
where an object created in one context might "leak" into another
context (this is possible if two clients interact with the same
underlying native library). This situation is new, and would
have to be taken care of.</p>
<p>Cheers<br>
Maurizio<br>
</p>
<p><br>
</p>
<p><br>
</p>
<blockquote type="cite" cite="mid:CAA7F5jJN91yEzMn3Lu4gNVYk9NazKHL1+qMoYxcv9StSv+SSFA@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">
<div><br>
</div>
<div>Regards</div>
<div>Manuel<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">Am Mi., 15. Feb. 2023 um
00:21 Uhr schrieb Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">Hi Manuel,<br>
in principle, it would be doable to expose an API which
creates a JNI <br>
handle out of a Java object and exposes it as a
MemorySegment so that it <br>
can be passed to native functions. One thing to consider
is that if we <br>
started passing Java objects around to native code, then
we'd also have <br>
to start worrying about the same objects being
"resurrected" in <br>
unexpected places (e.g. created by one classloader,
accessed by <br>
another). Some of these issues are the very reasons behind
the <br>
classloader restrictions behind System::loadLibrary [1].
Perhaps here, <br>
since the API doing the access is a Java API, it might be
possible to <br>
insert some kind of class loader check.<br>
<br>
That said, one of the main reason we did not add any
special support for <br>
JNI handles is that, in addition to the workarounds you
mention, a new <br>
and more powerful workaround is/will be possible from Java
20: <br>
ScopedValues [2]. That is, you can set up a scoped value,
and "bind" it <br>
before the upcall runs (e.g. before you call the
corresponding <br>
downcall). The bound value would then be in scope (as a
sort of implicit <br>
parameter) inside the upcall code as well. Retrieving a
scoped value is <br>
also quite fast (compared e.g. to a ThreadLocal).<br>
<br>
For this reason we'd like to see how using ScopedValues
goes before <br>
thinking about adding more ad-hoc machinery to expose JNI
handles in Panama.<br>
<br>
Cheers<br>
Maurizio<br>
<br>
[1] - <br>
<a href="https://docs.oracle.com/javase/7/docs/technotes/guides/jni/jni-12.html#libmanage" rel="noreferrer" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://docs.oracle.com/javase/7/docs/technotes/guides/jni/jni-12.html#libmanage</a><br>
[2] - <a href="https://openjdk.org/jeps/429" rel="noreferrer" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://openjdk.org/jeps/429</a><br>
<br>
<br>
On 14/02/2023 18:39, Manuel Bleichenbacher wrote:<br>
> Hi all<br>
><br>
> Many native SDKs allow the caller to pass a void* or
long as an <br>
> additional parameter. This parameter is then later
provided as a <br>
> reference to the original call, either in the context
of a callback <br>
> function call or as an attribute of an operating
system object. The <br>
> native SDK will not interpret the parameter in any
way, neither as a <br>
> number nor as a pointer. It's up to the caller to
know what it represents.<br>
><br>
> Examples are:<br>
><br>
> lParam in Windows' SendMessage() - <br>
> <a href="https://urldefense.com/v3/__https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendmessage__;!!ACWV5N9M2RV99hQ!ObuzTPS17OHPuAWdkoUdZ15H6ZrH5uj07MP5jV1Zccc4rS1XvpdILx3cG-IefKuy_8uY3YyZwU19dNZ7GxNigifnyTkGC6mZlA$" rel="noreferrer" target="_blank" moz-do-not-send="true">https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendmessage</a><br>
><br>
> dwNewLong in Windows' SetClassLongPtr() - <br>
> <a href="https://urldefense.com/v3/__https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setclasslongptrw__;!!ACWV5N9M2RV99hQ!ObuzTPS17OHPuAWdkoUdZ15H6ZrH5uj07MP5jV1Zccc4rS1XvpdILx3cG-IefKuy_8uY3YyZwU19dNZ7GxNigifnyTmGQRUeqw$" rel="noreferrer" target="_blank" moz-do-not-send="true">https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setclasslongptrw</a><br>
><br>
> arg in POSIX's pthread_create() - <br>
> <a href="https://urldefense.com/v3/__https://pubs.opengroup.org/onlinepubs/000095399/functions/pthread_create.html__;!!ACWV5N9M2RV99hQ!ObuzTPS17OHPuAWdkoUdZ15H6ZrH5uj07MP5jV1Zccc4rS1XvpdILx3cG-IefKuy_8uY3YyZwU19dNZ7GxNigifnyTm2gkc1kA$" rel="noreferrer" target="_blank" moz-do-not-send="true">https://pubs.opengroup.org/onlinepubs/000095399/functions/pthread_create.html</a><br>
><br>
> refcon in macOS' WritePipeAsync() - <br>
> <a href="https://urldefense.com/v3/__https://developer.apple.com/documentation/iokit/iousbinterfaceinterface800/1639539-writepipeasync__;!!ACWV5N9M2RV99hQ!ObuzTPS17OHPuAWdkoUdZ15H6ZrH5uj07MP5jV1Zccc4rS1XvpdILx3cG-IefKuy_8uY3YyZwU19dNZ7GxNigifnyTnv2uJlmA$" rel="noreferrer" target="_blank" moz-do-not-send="true">https://developer.apple.com/documentation/iokit/iousbinterfaceinterface800/1639539-writepipeasync</a><br>
><br>
> Is there a way to use this from Java for Java
objects, i.e. to pass a <br>
> Java object as a void* and later convert the void*
back to the Java <br>
> object?<br>
><br>
> Since the FFM API exposes void* as MemorySegments, it
would probably <br>
> mean that a MemorySegment could be created
encapsulating a reference <br>
> to a Java object, and that the Java object can later
be retrieved from <br>
> this MemorySegment.<br>
><br>
> I am aware of two workarounds:<br>
><br>
> - In the context of a callback function, the Java
callback function <br>
> can be extended with an additional parameter and then
the Java object <br>
> can be bound to it. This function is then used to
create the up call <br>
> stub. Since the callback function already contains
the reference, the <br>
> void* parameter is no longer needed and can be passed
<br>
> MemorySegment.NULL. In my context (I/O processing
with several <br>
> thousand functions calls per second), it likely is
rather inefficient <br>
> as several thousand upcall stubs will need to be
generated by second.<br>
><br>
> - Instead of the Java object, a generated integer key
could be passed <br>
> as the void*. The link between the key and the Java
object would need <br>
> to be managed in a hash map. Possible, but more of a
workaround.<br>
><br>
> Regards<br>
> Manuel<br>
><br>
><br>
><br>
><br>
</blockquote>
</div>
</div>
</blockquote>
</blockquote>
</body>
</html>