Embedded JDK how to replace the default allocator

[Guy Korland]

> you are still not very forthcoming with information, so we don't know
what your goal is. Total isolation of all allocations? Or just monitoring?
Is partial isolation okay?

Sorry for not being clear, we are trying to build an extension to Redis
that will run an embedded JVM. In order to allow Redis monitor all the
memory allocated by the JVM we need to make sure all the allocation are
going through Redis allocation commands.
We already managed to do it in the past with other VMs like CPython, WSAM &
Lua, but didn't find an easy way to set it in OpenJDK.


Regards,
Guy Korland


On Tue, Dec 15, 2020 at 8:54 AM Thomas Stüfe <thomas.stuefe at gmail.com>
wrote:

> Hi,
>
> you are still not very forthcoming with information, so we don't know what
> your goal is. Total isolation of all allocations? Or just monitoring? Is
> partial isolation okay?
>
> Since I do not know what you need, here are some hints:
>
> - majority of allocations of a VM are not done via C-Heap but via virtual
> memory APIs like mmap() or SystemV shm().
>
> - C Heap allocations from the hotspot itself could be redirected, since
> they go through the os::malloc... layer. I believe this also takes care of
> C++ allocations
>
> - However, that is only a small part of all allocations, since any library
> loaded via JNI (including both the JDK libraries themselves as well as any
> third party JNI libraries we don't have control over) rolls their own
> memory management by calling malloc or C++ new or mmap or... directly.
>
> - Also note that there may be places in the hotspot where C heap allocated
> pointers are kept in a hashmap, and radically changing how the pointers
> look like (different allocator) may mess with the efficiency of those hash
> functions.
>
> So if you need total isolation, the only realistic way is as David
> described: use LD_PRELOAD and provide redirections for all system level
> APIs which allocate memory.
>
> The problem with redirecting C-Heap allocations at that level however is
> that if you override free() you need to have perfect coverage of all
> allocating APIs. These are not limited to malloc/calloc/free etc but there
> are a number of other APIs which return their results in C-Heap allocated
> buffers. And I have no idea honestly how to redirect C++ allocations via
> LD_PRELOAD.
>
> Cheers, Thomas
>
>
> On Tue, Dec 15, 2020 at 12:08 AM Guy Korland <gkorland at gmail.com> wrote:
>
>>
>> I'm talking about replacing the malloc implementation, we're running on
>> Linux.
>> But, the tricky part is that my C application malloc should not be
>> changed, only the libjvm.so when loaded embedded in my application.
>>
>> Regards,
>> Guy Korland
>>
>>
>> On Sun, Dec 13, 2020 at 12:15 PM Thomas Stüfe <thomas.stuefe at gmail.com>
>> wrote:
>>
>>> Hi Guy,
>>>
>>> What do you mean by default allocator? C-Heap? Mmap/shmat? And what OS
>>> are you targeting?
>>>
>>> Cheers, Thomas
>>>
>>> On Fri, Dec 11, 2020 at 9:49 PM Guy Korland <gkorland at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm sorry if that is not the right mailing list for such questions,
>>>> please
>>>> point me to the right one if it's not.
>>>>
>>>> We're embedding the OpenJDK in a C application, we already managed to
>>>> make
>>>> it work and run Java bytecode, but we are still missing one last piece,
>>>> we
>>>> need a way to replace the default memory allocator to use our allocator.
>>>>
>>>> Is there a way to do it?
>>>>
>>>> Thanks,
>>>> Guy Korland
>>>>
>>>