[foreign-memaccess+abi] RFR: 8294309: Downcall and Upcall unboxing code should reject heap segments

[Jorn Vernee]

On Mon, 3 Oct 2022 21:16:41 GMT, Jorn Vernee <jvernee at openjdk.org> wrote:

> Recently I noticed that we don't check for heap segments anymore when unboxing memory segments to be passed to native code as addresses. This allows someone to pass a heap segment, which will be unboxed simply by calling `MemorySegment::address` which for heap segments returns the byte offset into the array. This is of course not a valid pointer.
> 
> This patch changes the unboxing logic to reject heap segments with an `IllegalArgumentException`.

test/jdk/java/foreign/TestUpcallException.java line 41:

> 39: import java.io.IOException;
> 40: 
> 41: public class TestUpcallException extends UpcallTestHelper {

I refactored this class to use a common utility method in the super class, which is also used by the new test

test/jdk/java/foreign/UpcallTestHelper.java line 38:

> 36: import static org.testng.Assert.assertTrue;
> 37: 
> 38: public class UpcallTestHelper extends NativeTestHelper {

I thought about just adding this to NativeTestHelper, but didn't want to require every dependent test to have to add `@library /test/lib`

-------------

PR: https://git.openjdk.org/panama-foreign/pull/737