[foreign] RFR 8214194: Make TestUpcall & TestDowncall more fine grained

[Jorn Vernee]

I like that suggestion. I'm still passing in the method name as well, 
since testng is not calling toString on the passed Method, but instead 
constructing some kind of default String representation for the test 
report, so the String is there for the name to show up.

Updated webrev: 
http://cr.openjdk.java.net/~jvernee/panama/webrevs/maketests/webrev.01/

Cheers,
Jorn

Maurizio Cimadamore schreef op 2018-11-26 12:44:
> Looks good - I would personally remove this loop:
> 
> for (Method m : headerCls.getDeclaredMethods()) { ... }
> 
> And let the data provider handle it.You could do that by having the
> DowncallTest constructor accept the header class and the library
> object. Then the data provider could loop over all the declared
> methods, and return one of them. And the main test logic would create
> a scope (with try with resources), create some checks, instantiate
> arguments, do the downcall, check the results and then close the scope
> (by closing the TWR). I think that would separate the concerns a bit
> more.
> 
> Thanks
> Maurizio
> 
> On 25/11/2018 23:35, Jorn Vernee wrote:
> 
>> Hi,
>> 
>> While in the process of fixing Windows bugs I ran into a crash with
>> both TestUpcall and TestDowncall (since fixed).
>> 
>> The problem for me was that it was hard to tell which native
>> function the test was actually crashing on, since the only thing
>> that showed up in the report was the method filter used as a
>> parameter to jextract.
>> 
>> I have changed these 2 tests to create a single TestNG test for each
>> invoked native method, and added stderr prints so you can see when
>> the test crashed e.g.
>> 
>> Calling f11_I_IPS_PP......done
>> Calling f15_P_PDS_PFI......done
>> Calling f13_F_FSD_FD...
>> 
>> Will mean that the test 'f13_F_FSD_FD' crashed.
>> 
>> I'm also passing in the name of the method that's being called with
>> a data provider since the parameters will show up in the test
>> report, you can see exactly which methods failed (but didn't crash)
>> e.g:
>> 
>> test TestDowncall$DowncallTest.testDownCall("f3_V_FPD_"):
>> success
>> test TestDowncall$DowncallTest.testDownCall("f7_V_SFS_PII"):
>> failed
>> test TestDowncall$DowncallTest.testDownCall("f2_V_ISF_FDI"):
>> success
>> 
>> This really helped me with debugging and it seems like a useful
>> thing to have, so I'd like to offer it up as a contribution.
>> 
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214194
>> Webrev:
>> 
> http://cr.openjdk.java.net/~jvernee/panama/webrevs/maketests/webrev.00/
>> 
>> 
>> Cheers,
>> Jorn