MemorySegment JVM memory leak

[Maurizio Cimadamore]

Thinking more about this, I think I stand by the existing API.

Short-term is going to be a little inconvenient, yes, because to 
map/unmap files you basically need to go native (or to call close() on 
the original segment and accept some thread dances)

But not so long-term there's gonna be the next chapter in the story - 
the ABI integration - which will let you create native method handles; 
that is method handles targeting native library functions. Once this 
capability comes in (and you can already try it out in the Panama repo), 
it is relatively easy to map/unmap a file directly, w/o using JNI code 
or any kind of external dll/so lib.

For instance you could create a truly unsafe segment w/o even using the 
standard MemorySegment::mapFromFile; below is a simple Gist which shows 
how you can do something like that using the ABI support:

https://gist.github.com/mcimadamore/128ee904157bb6c729a10596e69edffd

(it is less than 100 LoC - half of it is the test logic).

I think that, for power-users like you, this way of doing things is 
probably more direct than bending MappedByteBuffer, or 
MappedMemorySegment exactly the way you want it. At some point you are 
gonna need some extra customization (you mentioned about MADV_DONTNEED, 
other people mentioned MADV_REMOVE) which might make difference in your 
case; while in some cases some additional API points will be added to 
the JDK, we can't expect the JDK to support all possible ways in which a 
client might wish to interact with memory mapped files. But with custom 
memory segments + ABI support you don't need to wait on the JDK to give 
you the knobs you want - you can just reach for them directly.

I think that's a much saner way to get things done - in a way, the whole 
mappedXYZ business is a big workaround for the fact that we have no 
other ways in Java to reason about memory mapped files (which are 
useful!) but their behavior is ultimately platform-dependent, hence some 
of the APIs in MappedByteBuffer and MappedMemorySegment are "best 
effort" or simply do nothing on certain platforms (e.g. Windows).

So, maybe what you need, ultimately, is your own custom segment factory. 
All still written in Java - but in a "different kind" of Java.

Maurizio


On 23/04/2020 01:48, Maurizio Cimadamore wrote:
>
> On 23/04/2020 01:34, Maurizio Cimadamore wrote:
>> No. But that doesn't seem to be a very important issue? If you don't 
>> need the original safe segment, you can just ditch it and leave it to 
>> the GC? 
>
> Ok, I see what you mean now.
>
> You basically want that calling close() on the unsafe segment calls 
> the cleanup action (unmap) of the original segment. And, it seems you 
> don't want to write native code for it either, so you have no way to 
> write a Runnable that does what you want.
>
> When I issued the patch originally, I allowed the possibility of 
> passing an AutoCloseable as a cleanup action (not a Runnable), which 
> meant you could pass the _safe_ segment as the cleanup action, and the 
> cleanup action would have been called regardless of confinement.
>
> During the review process, it was observed that this was perhaps a bit 
> too magic - e.g. calling close() would leave the original segment in 
> an unconsistent state (e.g. isAlive() == true, but memory freed).
>
> I'll think a bit more whether something like that could be achieved in 
> a way that doesn't add too much complexity to the API, I think I see 
> maybe a path to get there (or maybe is the late hour :-) ).
>
> Thanks
> Maurizio
>