[foreign] RFR 8214194: Make TestUpcall & TestDowncall more fine grained

[Maurizio Cimadamore]

Looks good. I believe the System.err.println is redundant as testng will 
printout the combination it's working on... I'll do some experiment 
ahead of pushing and see what output I get.

Thanks
Maurizio

On 26/11/2018 14:08, Jorn Vernee wrote:
> I like that suggestion. I'm still passing in the method name as well, 
> since testng is not calling toString on the passed Method, but instead 
> constructing some kind of default String representation for the test 
> report, so the String is there for the name to show up.
>
> Updated webrev: 
> http://cr.openjdk.java.net/~jvernee/panama/webrevs/maketests/webrev.01/
>
> Cheers,
> Jorn
>
> Maurizio Cimadamore schreef op 2018-11-26 12:44:
>> Looks good - I would personally remove this loop:
>>
>> for (Method m : headerCls.getDeclaredMethods()) { ... }
>>
>> And let the data provider handle it.You could do that by having the
>> DowncallTest constructor accept the header class and the library
>> object. Then the data provider could loop over all the declared
>> methods, and return one of them. And the main test logic would create
>> a scope (with try with resources), create some checks, instantiate
>> arguments, do the downcall, check the results and then close the scope
>> (by closing the TWR). I think that would separate the concerns a bit
>> more.
>>
>> Thanks
>> Maurizio
>>
>> On 25/11/2018 23:35, Jorn Vernee wrote:
>>
>>> Hi,
>>>
>>> While in the process of fixing Windows bugs I ran into a crash with
>>> both TestUpcall and TestDowncall (since fixed).
>>>
>>> The problem for me was that it was hard to tell which native
>>> function the test was actually crashing on, since the only thing
>>> that showed up in the report was the method filter used as a
>>> parameter to jextract.
>>>
>>> I have changed these 2 tests to create a single TestNG test for each
>>> invoked native method, and added stderr prints so you can see when
>>> the test crashed e.g.
>>>
>>> Calling f11_I_IPS_PP......done
>>> Calling f15_P_PDS_PFI......done
>>> Calling f13_F_FSD_FD...
>>>
>>> Will mean that the test 'f13_F_FSD_FD' crashed.
>>>
>>> I'm also passing in the name of the method that's being called with
>>> a data provider since the parameters will show up in the test
>>> report, you can see exactly which methods failed (but didn't crash)
>>> e.g:
>>>
>>> test TestDowncall$DowncallTest.testDownCall("f3_V_FPD_"):
>>> success
>>> test TestDowncall$DowncallTest.testDownCall("f7_V_SFS_PII"):
>>> failed
>>> test TestDowncall$DowncallTest.testDownCall("f2_V_ISF_FDI"):
>>> success
>>>
>>> This really helped me with debugging and it seems like a useful
>>> thing to have, so I'd like to offer it up as a contribution.
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214194
>>> Webrev:
>>>
>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/maketests/webrev.00/
>>>
>>>
>>> Cheers,
>>> Jorn