taming resource scopes

[Maurizio Cimadamore]

On 02/06/2021 11:33, Ty Young wrote:
> MemorySegments/scopes don't prevent freed memory from being accessed 
> If you enable restricted methods and invoke the CLinker.freeMemory 
> method. This seems like a bug with the method itself since it's 
> officially part of Panama's API, even if part of the "unsafe" part. 
> Custom made free bindings may not automatically call close, of course, 
> but CLinker.freeMemory maybe should?
That's the meaning of restricted - if you call these methods, you 
abandon safety guarantees. In the case of CLinker::freeMemory, the 
reason this method (and his companion CLinker::allocateMemory) exist, is 
to provide a plain wrapper around malloc/free. Sometimes native code 
gives an address to you and expects you to free it. You don't have a 
segment, nor a scope. You just have an address. The only wa
>
>
> On the topic of free memory, what's the deal with calling free from 
> different bindings/methods? If you do:
>
>
> MemorySegment segment = MemorySegment.allocateNative(4, 
> ResourceScope.newSharedScope());
>
> CLinker.freeMemory(segment.address());
> CLinker.freeMemory(segment.address());
>
>
> or, with hand made bindings:
>
>
> MemorySegment segment = MemorySegment.allocateNative(4, 
> ResourceScope.newSharedScope());
>
> clib.free(segment.address());
> clib.free(segment.address());
>
>
> On Linux you get:
>
>
> free(): double free detected in tcache 2
>
>
> as expected but mixing free bindings:
>
>
> MemorySegment segment = MemorySegment.allocateNative(4, 
> ResourceScope.newSharedScope());
>
> clib.free(segment.address());
> CLinker.freeMemory(segment.address());
>
>
> gives nothing nor does using the same bindings in different methods.Is 
> this expected?

First, you should never mix safe segments with unsafe, restricted code. 
CLinker::freeMemory is not something you should look at if you are after 
writing safe code. I explained above when these methods are meant to be 
used.

The message you see come from the guts of `free`. As long as your 
clib::free indeed point to the same free function, then the difference 
is probably explained in terms of a failure in the heuristics which 
detects double free.

If your clib::free links a _different_ free method, then this is more 
expected, as different allocators typically do not cooperate with each 
other. But since you are seeing same problem with calling same free 
binding on different methods, I'd put that down to a failure in heuristics.

But the point remains: CLinker::free offers (as you can see) no safety 
guarantees - which is why it's a restricted method and why you have to 
pass a flag on the command line to use it.

Maurizio

>
>
> On 6/2/21 3:16 AM, Chris Vest wrote:
>> You can build an Arc (Atomic Reference Counted) like thing as a library,
>> but without deep language integration it will be easy for the contained
>> reference to escape the Arc.
>> Thankfully MemorySegments will still prevent you from accessing freed
>> memory.
>> An alternative to building a generic Arc is to build specialised 
>> atomically
>> reference counting wrappers for each thing you wish to have reference
>> counted, and then delegate calls instead of exposing the inner 
>> reference.
>> You still run into problems that, without language integration, object
>> references can be shared without any enforcement of counter adjustments.
>> This makes usage awkward and introduces a new class of bugs that can 
>> occur
>> in programs.
>> You might also run into an issue that your object can now be in one of
>> three states: owned, shared, and closed/freed/released – the distinction
>> between owned and shared is the new thing.
>> Objects that are in a shared state might not support all of the 
>> operations
>> of an owned object.
>> Rust tracks this information in the type system as part of the language.
>> I built a prototype buffer implementation where I tracked this 
>> information
>> at runtime, but discarded the idea because it is, again, awkward and 
>> error
>> prone.
>> There were many places where the code had to check and branch on the
>> owned/shared state, and failure to do so would cause exceptions at 
>> runtime.
>> So my point is that Rust and Swift have good reference counting stories
>> because of specific language features, and without similar language
>> features I don't think having a generic or general Arc class in Java 
>> will
>> be very helpful.
>>
>> Cheers,
>> Chris