can I expect a lock functionality on a MemorySegment similar to mlock?

[Maurizio Cimadamore]

On 12/10/2020 11:45, kant kodali wrote:
> Thanks for the response. I will check out the Linker API and that does 
> look better than JNI however, it still looks very clunky but I 
> understand introducing something like mlock would break the "write 
> once run anywhere" principle. I personally don't see harm in 
> supporting something that is OS specific as long as the user is well 
> aware. For example, say if you use package X there is no guarantee of 
> "write once run anywhere" I would be totally fine with that. Currently 
> as you were saying there is some way to call mlock like using the 
> Linker API so why not a cleaner API like this 
> https://golang.org/pkg/syscall/ 
> <https://urldefense.com/v3/__https://golang.org/pkg/syscall/__;!!GqivPVa7Brio!LdF8xYjuFb09L7U2EZPea81Mk-FmSIalOoNISokPl0BGjdUPRoRvjQzG1GxFI6gUbUMTI-w$> 
> ? It is purely in Golang and looks so simple to use for any low level 
> application.

Without diving too much into the specifics, reading from the docs, it 
doesn't seem to me as if `syscall` supports "write once and run 
anywhere" - this document:

https://docs.google.com/document/d/1QXzI9I1pOfZPujQzxhyRy6EeHYTQitKKjHfpq0zpxZs/edit

Seems to point out at the need to separate out the functionalities in 
different packages, windows, linux, etc - which is more or less what I'd 
expect for such a low level set of functionalities.

Note that the foreign linker method handles are a "tool" - not a final 
API. You can write a class which exports some kind of `mlock` Java 
method which, internally might use method handle X on Linux and method 
handle Y on Windows (or some other code). So, while the _implementation_ 
of the `mlock` method will have to (necessarily) distinguish between 
platforms, clients of this method won't have to.

Maurizio


>
>
>
> On Mon, Oct 12, 2020 at 2:47 AM Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com 
> <mailto:maurizio.cimadamore at oracle.com>> wrote:
>
>     We have no plans to add such a functionality directly, in the memory
>     segment API. However, with the addition of the Foreign Linker API,
>     things are not so bad, you can define your mlock MethodHandle once
>     and
>     for all:
>
>     ```
>     //int mlock(const void *addr, size_t len);
>     MethodHandle mlock = CLinker.getInstance().downcallHandle(
>              LibraryLookup.ofDefault().lookup("mlock").get(),
>              MethodType.methodType(int.class, MemoryAddress.class,
>     long.class),
>              FunctionDescriptor.of(C_INT, C_POINTER, C_LONG)
>     );
>     ```
>
>     And then you can call it on an existing segment, as desired:
>
>     ```
>     MemorySegment segment = ...
>     mlock.invokeExact(segment.address(), 100L);
>     ```
>
>     No JNI code is required to do this; so clients might define all the
>     helper method handles they need to interact with mapped memory
>     e.g. in a
>     separate "util" class.
>
>     Full disclosure: I'm not a super fan of having such low level
>     functionality API points; learning from our experience with existing
>     similar low-level functionalities, such as mapped buffer/segment
>     force/load/unload, what I've learned is that an approach that
>     tries to
>     expose a single API, regardless of the platform is not a very sharp
>     tool, and you end up with things like (see Windows implementation of
>     MappedMemoryUtils.c):
>
>     ```
>     NIEXPORT void JNICALL
>     Java_java_nio_MappedMemoryUtils_load0(JNIEnv *env, jobject obj, jlong
>     address,
>                                           jlong len)
>     {
>          // no madvise available
>     }
>     ```
>
>     I believe a demand for such capabilities, in the past, has come
>     from the
>     fact that JNI is the only tool which allows you to peek and poke at
>     mapped memory; that put increased demand for such functions in the
>     public ByteBuffer API. But as you can clearly see, you can't expect
>     these function to behave correctly on all systems - and even within a
>     "supported" system, different clients and workload, might need
>     different
>     tuning as to how the memory mapping is implemented.
>
>     As a result, my feeling is that we could spend years trying to add a
>     rich set of memory mapping functions, without really improving
>     much over
>     the status quo. If you are a client where the use of "mlock" is
>     crucial,
>     I think you should just tap into that - and I think the linker API
>     makes
>     it easy enough for you to do that w/o resorting to any magic JNI
>     incantation.
>
>     Cheers
>     Maurizio
>
>
>     On 11/10/2020 11:43, kant kodali wrote:
>     > Hi All,
>     >
>     > I am implementing a buffer pool for my experimental database and
>     ideally I
>     > want to be able to pin a page using Java(without using JNI/JNA
>     myself) and
>     > by pinning a page I mean prevent a page from being swapped out
>     or in. some
>     > java version of mlock system call would work so I was thinking
>     why not a
>     > mlock like functionality on MemorySegment? If I have to resort
>     to JNI/JNA
>     > all the time then it makes me think Java shouldn't be the right
>     language to
>     > build a database like application. please let me know what you
>     think?
>     >
>     > Thanks,
>     > Kant
>