Can't find the bug

[Bernd Müller]

Hi Maurizio,

thank you so much for your explanation. Because the test itself was taken from JDK's tests
I was out of good advice. Probably, it would have take me days to recognize that the
'ADDRESS.asUnbounded()' was missing.

Tried both alternatives. Works fine.

Kind Regards,

Bernd


Am 04.01.23 um 11:26 schrieb Maurizio Cimadamore:
> Hi Bernd,
> the problem in your code lies here:
> 
> ```
> ((MemorySegment)strcat.invokeExact(buf, other))
>                                         .getUtf8String(0);  // <----------------------
> ```
> 
> That is, you are dereferencing the result of a "strcat" call. Now, while you "know" that the 
> resulting pointer will be a zero-terminated string of some known size, the Java runtime has no way 
> to know that. Attached to the pointer there could be one char, two char, or infinite - the linker 
> has no way to know.
> 
> For this reason, all pointers returned by the linker are modelled as "zero-length memory segments" 
> (or ZLM in short). This is described here:
> 
> https://download.java.net/java/early_access/jdk20/docs/api/java.base/java/lang/foreign/MemorySegment.html#wrapping-addresses
> 
> To "resize" a ZLM you can create a _new_ segment, using the MemorySegment::ofAddress factory, which 
> takes a raw address, a size and a scope.
> 
> Or, more simply, you can use an *unbounded* address layout when constructing the downcall method 
> handle, as this:
> 
> ```
> strcat = linker.downcallHandle(linker.defaultLookup().find("strcat").get(),
> FunctionDescriptor.of(ADDRESS.asUnbounded(), ADDRESS, ADDRESS));
> ```
> 
> This will tell the linker to resize the returned segment to the maximal size (Long.MAX_VALUE), which 
> will make your dereference work.
> 
> Which path you take is up to you - the former gives you more control - and more safety; the latter 
> might be useful if you just want "things to work". But both involve a "leap of faith" safety-wise, 
> as the Java runtime is trusting you to know what's coming out of that pointer. For this reason, both 
> paths will require a call to a so called "restricted" method, which will produce a runtime warning 
> (which can be useful when diagnosing the source of unsafety in case of e.g. a JVM crash).
> 
> Hope this helps.
> 
> Maurizio
> 
> On 04/01/2023 07:52, Bernd Müller wrote:
>> Dear Gavin,
>>
>> thank you very much for extracting the code.
>> Eventually, it would be much easier to do a "mvn test". To do so
>> please clone:
>> https://urldefense.com/v3/__https://github.com/BerndMuller/ffm2__;!!ACWV5N9M2RV99hQ!PmaiPHl2BbBSjKjJr5SwpasFBcCCILMjVmcHQjs4n10ixKwpJyHK3aucRIBDoLKroPXW46Wtfvqm_mHjWecm3dfSiaicIK_yzGM$
>> Error still:
>> StdLibTest.foo:49->strcat:60 » IndexOutOfBounds Out of bound access on segment MemorySegment{ 
>> array: Optional.empty address:139631472807584 limit: 0 }; new offset = 0; new length = 1
>>
>> looks for me like strcat changes the array size
>>
>> Kind Regards,
>>
>> Bernd
>>
>> Am 04.01.23 um 02:12 schrieb Gavin Ray:
>>> Maurizio, the code I was able to get out of the email is below:
>>>
>>> public class StdLibTest {
>>>
>>> private static MethodHandle strlen; // C's strlen() function
>>> private static MethodHandle strcat; // C's strcat() function
>>>
>>>      @BeforeAll
>>> public static void locateFunctions() {
>>> Linker linker = Linker.nativeLinker();
>>> strlen = linker.downcallHandle(linker.defaultLookup().find("strlen").get(),
>>> FunctionDescriptor.of(JAVA_LONG, ADDRESS));
>>> strcat = linker.downcallHandle(linker.defaultLookup().find("strcat").get(),
>>> FunctionDescriptor.of(ADDRESS, ADDRESS, ADDRESS));
>>>      }
>>>
>>>
>>>      @Test
>>> public void strlen() throws Throwable {
>>> try (Arena arena = Arena.openConfined()) {
>>> MemorySegment segment = arena.allocateUtf8String("Hello");
>>> int length = (int) (long) strlen.invoke(segment);
>>> Assertions.assertSame(5, length);
>>>          }
>>>      }
>>>
>>>      @Test
>>> public void foo() throws Throwable {
>>> String result = strcat("hello", "world");
>>>      }
>>>      /**
>>>       * This is a one to one copy of strcat test taken from  StdLibTest.java test from OpenJDK
>>>       */
>>> String strcat(String s1, String s2) throws Throwable {
>>> try (var arena = Arena.openConfined()) {
>>> MemorySegment buf = arena.allocate(s1.length() + s2.length() + 1);
>>> buf.setUtf8String(0, s1);
>>> MemorySegment other = arena.allocateUtf8String(s2);
>>> return ((MemorySegment)strcat.invokeExact(buf, other)).getUtf8String(0);
>>>          }
>>>      }
>>> }
>>>
>>> On Tue, Jan 3, 2023 at 12:55 PM Maurizio Cimadamore <maurizio.cimadamore at oracle.com 
>>> <mailto:maurizio.cimadamore at oracle.com>> wrote:
>>>
>>>     Hi Bernd,
>>>     this mailing list truncates attachments. If you want us to look at some
>>>     code you will have to paste it somewhere :-)
>>>
>>>     Cheers
>>>     Maurizio
>>>
>>>     On 03/01/2023 17:14, Bernd Müller wrote:
>>>      > Dear all,
>>>      >
>>>      > I try to train FFM. I even master quick sort but became desperate on
>>>      > strcon.
>>>      >
>>>      > Attached you will find a Maven project with the strcon test copied one
>>>      > to one from
>>>      > OpenJDK 20 sources.
>>>      >
>>>      > "mvn test" results in
>>>      >
>>>      > [ERROR] Errors:
>>>      > [ERROR]   StdLibTest.foo:49->strcat:60 » IndexOutOfBounds Out of bound
>>>      > access on segment MemorySegment{ array: Optional.empty
>>>      > address:139636573277136 limit: 0 }; new offset = 0; new length = 1
>>>      >
>>>      > I am running OpenJDK 64-Bit Server VM (build 20-ea+29-2280, mixed
>>>      > mode, sharing) on Fedora 36.
>>>      >
>>>      > Please, can someone help ?
>>>      >
>>>      > Kind Regards,
>>>      >
>>>      > Bernd
>>>      >
>>>
>>
>>


-- 
Prof. Dr. Bernd Müller

Ostfalia
Hochschule für angewandte Wissenschaften
- Hochschule Braunschweig/Wolfenbüttel -
Fakultät Informatik
Salzdahlumer Straße 46/48
38302 Wolfenbüttel

Tel  +49 5331 939 31160
Fax  +49 5331 939 31004
Web  www.ostfalia.de / www.pdbm.de