Unsafe vs MemorySegments / Bounds checking...

[Brian S O'Neill]

On 2024-10-30 11:00 AM, Maurizio Cimadamore wrote:
> I think this discussion is spiralling out of control (which is ok :-) ).
> 
> First, I believe the inlining traces you are pointing are either 
> including transient failures from C1. I recommend only using -XX: 
> +PrintInlining together with -XX:-TieredCompilation and see if that 
> makes the output a little better. Sometimes I also throw in -Xbatch, 
> which makes the output a bit less chaotic to look at (but that can 
> affect performance).

I'll try this out in a bit.

> 
> Stepping back, the problem at hand is that there might be some 
> simplifications we could do to make the call stack of memory access var 
> handle simpler. Provided we can back up some of these intuitions with 
> hard evidence, and that we can simplify the calls _without losing 
> functionality_ sure, that all sounds great. (Although I don't think 
> there's many low hanging fruit in there -- but I would be glad to be 
> proven wrong of course!).

Yes. The hard evidence is the fact that the best performance achieved so 
far is by using an unintuitive VarHandle transform. The straightforward 
approach of using a MemorySegment directly isn't as efficient.

You want more evidence? Other people are struggling with this and have 
posted their concerns to the mailing list. As for myself, I've burned 
away dozens of hours of my life trying resolve these issues. You've 
spent time yourself chasing things down and proposed the VarHandle 
transformation in the first place. It shouldn't be this way.

> 
> What we will **not** do is split the FFM API in two parts - safe and 
> unsafe. This is not just about var handles - that's only a piece of the 
> story. There's bulk copy, and there's also Linker access. Once there's a 
> blessed way to remove checks in one place, one can ask for ways to 
> disable checks everywhere else.

I'm not suggesting this. The FFM API already supports safe and unsafe 
access, although it uses the term "restricted".

> 
> IMHO the most maintainable solution for your code is to use a segment 
> whose bound is Long.MAX_VALUE. That won't be _exactly_ like Unsafe 
> (because of the sign check), but it will be pretty close -- and at least 
> you won't have to maintain long chains of adapted var handle and cross 
> your fingers that everything optimizes correctly everywhere (which I 
> suspect will make inlining of the methods in your library a bit more 
> predictable).

I tried this originally and again just now, but the performance is 
worse, by about 1.5x. The sign check cannot be the primary issue.

> 
> But, from where I look, it seems to me that where we have landed is not 
> that bad? There's a *safe* API to access off-heap memory that in most 
> cases gives you the same performance as Unsafe. In some pathological 
> cases, you can give up some safety (with MemorySegment::reinterpret and 
> MAX_VALUE) to get as close to Unsafe as possible. We realize that, at 
> the time being, even that might not be enough. Over time, it is possible 
> that the implementation of the FFM API will be improved, or that Hotspot 
> inlining will get better (or both!). If past history is of any guidance, 
> these things have a tendency to get better over time. Fixing this by 
> adding *permanent* API warts is not the right way to approach this.
> 

The current approach is near to the Unsafe class in performance, but it 
doesn't offer any safety features. I can access an illegal address just 
as unsafely with the FFM API as I could before. It's a double regression 
(performance and simplicity) in exchange for nothing.

The ValueLayout class provides access to a VarHandle which is really 
close to what I want. This method itself is a specialized variant of the 
method inherited from MemoryLayout, and it's restricted too.

A new "convenience" method which returns a plain memory-accessing 
VarHandle could be implemented using the current VarHandle trick, and 
then optimized later with a more direct implementation. It's no more of 
a wart on the API than current the ValueLayout override.

Should the method belong in the ValueLayout class? I don't know. What 
should it be named? I don't know. Whatever it is, it makes for a much 
more seamless transition away from the Unsafe class.

I've not seen any date for when the Unsafe class is to be removed, but 
I'll keep using it in it's deprecated fashion until then, unless the 
necessary improvements are made to the FFM sub system. But if these 
issues aren't brought up early, then there might not be any incentive to 
make these improvements until the Unsafe class is removed, and more 
complaints flow in.


> Maurizio
> 
> 
> On 30/10/2024 17:30, Brian S O'Neill wrote:
>> Even when all the inlining transformations work, it's still expensive 
>> because the compiler has to do a bunch of extra work. And when the 
>> calling method needs to get recompiled (as it often does), all the 
>> inlining transforms need to be done all over again.
>>
>> In the end, the code (hopefully) gets transformed down into the 
>> original Unsafe method calls that would have been written instead. A 
>> set of more direct VarHandles might help, which could perform the 
>> basic low-level accesses and copies that the Unsafe class provides.
>>
>> The current trick of transforming the VarHandle provided by the 
>> ValueLayout class is a hack. It only (mostly) works for reasons that 
>> are known to the FFM implementers. When I try other tricks with 
>> VarHandles and MemorySegments, it either doesn't work as well or the 
>> performance is substantially worse. The current hack is also extremely 
>> fragile, as evidenced by the JDK 23 regression.