MemorySegment.ofAddress(...).reinterpret(...)

[Brian S O'Neill]

The application is a high performance database which scales to support 
very large cache sizes with very little GC overhead. The database can 
operate in pure "in memory" mode, or be backed by a file, or be memory 
mapped, or be backed by a block device using O_DIRECT.

The bulk of the cache is allocated up front via mmap(anonymous), and 
page references into the cache are just plain long pointers. The cache 
can grow over time, and so not all pointers refer to the same region.

The critical operations are memory copies and accessing primitive types: 
MemorySegment.get/set(JAVA_BYTE, etc). Memory copies are performed 
between pages and also to/from Java byte arrays.

I could replace all pointer references with MemorySegment references 
(with great difficulty), but this would bloat the total heap size and 
increase GC overhead.

Currently I'm seeing a 3% performance regression using the foreign 
memory API compared to using Unsafe. One trick that I'm using is to 
refer to the entire address space using a static final field: ALL = 
MemorySegment.NULL.reinterpret(Long.MAX_VALUE). I don't know if this has 
any overhead or not. It seems like access to it should be completely 
unchecked.

For memory copies I can try copying to/from the "ALL" segment, in which 
case I don't need to allocate temporary MemorySegments at all. I'll 
report back with my findings. Perhaps I just really need a convenient 
reference to all memory, unchecked, and with low overhead.


On 2023-07-05 04:12 PM, Jorn Vernee wrote:
> Hello,
> 
> The usual way of attaching a size to a memory segment would be through a 
> target address layout [1]. MS::reinterpret is meant for cases where the 
> layout of the pointee is not statically known (and can not be captured 
> with a memory layout). But, both cases assume you're reading/writing 
> MemorySegments directly, instead of going through longs. Are neither of 
> these an option for you? Could you explain your use case a bit more?
> 
> I think we briefly considered having multiple MS::ofAddress overloads. 
> IIRC we decided against it, because the current MS::ofAddress is not a 
> restricted method, but, an overload that sets the size of the returned 
> segment would need to be restricted since the segment would be 
> accessible. So, adding more overloads would create a split between them 
> in terms of restricted-ness, which is not great.
> 
> I'll also note that, someone could define a utility method that fuses 
> the two operations, e.g.:
> 
> static MemorySegment longToSizedSegment(long baseAddress, long byteSize) {
>      return MemorySegment.ofAddress(baseAddress).reinterpret(byteSize);
> }
> 
> That is to say: I think we have the right primitives available in the 
> API. Though, your experience is a useful data point. We might want to 
> add some useful methods in the future if we see enough demand for them.
> 
> Thanks,
> Jorn
> 
> [1]: 
> https://download.java.net/java/early_access/jdk21/docs/api/java.base/java/lang/foreign/AddressLayout.html#withTargetLayout(java.lang.foreign.MemoryLayout)
> 
> On 06/07/2023 00:19, Brian S O'Neill wrote: