MemorySegment JVM memory leak

[Ahmed]

Hello,

Coming back to Linux mmap

The problem with madvise(&db_map[i],4096,MADV_DONTNEED); Will make the memory segment in OS cache/buffer. I did a lot of testing and the OS eat all memory and put it as cache/buffer and OS will never release it until process die.

The attached screenshot explains very well.

I don't think madvise(&db_map[i],4096,MADV_DONTNEED); Is the perfect solution for Linux. There is madvise(&db_map[i],4096,MADV_REMOVE); Which completely return the memory segment as a free memory which it is excellent.

But again madvise(&db_map[i],4096,MADV_REMOVE); Is slow and severely impact performance.

Maybe a good solution to make a queue Of segment pages and quietly free them by madvise(&db_map[i],4096,MADV_REMOVE);

I completely understand OS to put segment in cache/buffer if process does not respect the OS page size. But honestly, I can't understand why OS keep the memory segment in cache/buffer after written to disk in mmap. The whole idea of mmap is to have a direct access and avoid cache/buffering of traditional read/write

By the way, in MAC I did a quick test and mmap is great and politely mark memory segment to free pool after written to disk. Unfortunately I don't have access to any Solaris, but I am very curious to know how it behave there, very long time ago I remember some news about and perfect implementation mmap by Sun Microsystem that time.

How java C source code implement this in mmap. Can you please share c source code of mmap usage ?.

Regards, Ahmed Hamdallah.

-----Original Message-----
From: Maurizio Cimadamore <maurizio.cimadamore at oracle.com> 
Sent: Thursday, April 23, 2020 4:48 AM
To: Uwe Schindler <uschindler at apache.org>; 'Ahmed' <ahmdprog.java at gmail.com>; panama-dev at openjdk.java.net
Subject: Re: MemorySegment JVM memory leak


On 23/04/2020 01:34, Maurizio Cimadamore wrote:
> No. But that doesn't seem to be a very important issue? If you don't 
> need the original safe segment, you can just ditch it and leave it to 
> the GC?

Ok, I see what you mean now.

You basically want that calling close() on the unsafe segment calls the cleanup action (unmap) of the original segment. And, it seems you don't want to write native code for it either, so you have no way to write a Runnable that does what you want.

When I issued the patch originally, I allowed the possibility of passing an AutoCloseable as a cleanup action (not a Runnable), which meant you could pass the _safe_ segment as the cleanup action, and the cleanup action would have been called regardless of confinement.

During the review process, it was observed that this was perhaps a bit too magic - e.g. calling close() would leave the original segment in an unconsistent state (e.g. isAlive() == true, but memory freed).

I'll think a bit more whether something like that could be achieved in a way that doesn't add too much complexity to the API, I think I see maybe a path to get there (or maybe is the late hour :-) ).

Thanks
Maurizio