[rfc][icedtea-web] Refactor of LiveConnect Tests Version 2

[Andrew Azores]

On 06/18/2014 10:52 AM, Jiri Vanek wrote:
> On 06/18/2014 03:28 PM, Andrew Azores wrote:
>> On 06/18/2014 06:08 AM, Jiri Vanek wrote:
>>> On 06/17/2014 05:28 PM, Andrew Azores wrote:
>>>> On 06/12/2014 03:58 PM, Jie Kang wrote:
>>>>> Hello,
>>>>>
>>>>>> I ran the reproducer suite with this patch applied last night. 
>>>>>> There are
>>>>>> still some tests which are failing but not marked as KnownToFail. I
>>>>>> haven't checked if these failures are repeatable however. Can you 
>>>>>> look
>>>>>> into this?
>>>>>>
>>>>>> JavascriptSetTest - AppletJToJSSet_2DArrayElement_Test
>>>>>> JSToJFuncResolTest - 
>>>>>> AppletJSToJFuncResol_inheritedClassToParent1_Test
>>>>>> JavascriptFuncReturnTest -
>>>>>> AppletJToJSFuncReturn_{number,boolean,JSObject,String,Object}_Test
>>>>>>
>>>>> I've checked the failures and fixed the FuncReturn test cases (typo).
>>>>>
>>>>> The JavascriptSetTest has some weird behaviour where the failure 
>>>>> occurs because the applet's init
>>>>> method didn't print out "applet initialized". This failure does 
>>>>> not happen very often at all and I
>>>>> will look into it.
>>>>>
>>>>> The JSToJFuncResolTest failure has been marked Known to Fail. I've 
>>>>> checked the test and found
>>>>> another bug in the LiveConnect code unrelated to the fixes made in 
>>>>> this patch. I have prepared
>>>>> another patch that fixes this bug and will send it after this patch.
>>>>>
>>>>> Thanks for the review!
>>>>>> Thanks,
>>>>>>
>>>>>> -- 
>>>>>> Andrew A
>>>>>>
>>>>>>
>>>>> -- 
>>>>> Jie Kang
>>>>
>>>> Just ran the tests, everything appears to be sorted out now.
>>>>
>>>> I'll wait before pushing to see if Jiri is strongly objecting to 
>>>> the "shared" resources folder,
>>>> otherwise, I think this is good to go.
>>> Unluckily, yes, I'm still a bit against. I probably do not 
>>> understand the usecase "lunch one
>>> reproducer"
>>>
>>> When I'm running one reproducer, then I allow it in whitelist, and 
>>> then engine prepare *all*
>>> reproducers, from which my one is lunched.
>>
>> Yea, that's exactly what I think is most problematic about ITW's 
>> reproducer suite. It's huge and
>> sluggish. The use case for running one reproducer in isolation is 
>> that it is then much faster and
>> more convenient to run tests targeting a specific patch being developed.
>
> By no other way you can run any reproducer, which is using another 
> reproducer's code.
>
>>
>>>
>>> My mine reason against the explicit shared resources is, that 
>>> actually also freshly prepared
>>> (compiled!) signed,  custom and simple reproducers can be shared 
>>> with each other.
>>>
>>> I Really think adding special case to copy some special resources is 
>>> misleading from point of
>>> above sentence.
>>
>> I'm not sure what you mean about the signed, custom, and simple being 
>> shared with each other. Yes,
>> they can share resources, or even compiled classes. How is it not 
>> better to have the shared
>
> to me, it osunds more confusing to have shared resources handled by 
> different way then shared code.
>
>> resources in a directory labelled "shared resources" rather than 
>> hidden away in "some" resources
>> directory of "some" reproducer that happens to have them? If you are 
>> working on a reproducer that
>> uses shared resources and it is not the reproducer that "hosts" those 
>> shared resources, how do you
>> know where to actually find them? You have to actually go use find or 
>> grep or something on the
>> entire tests/reproducers directory to discover that it's in some 
>> completely unrelated location. Why
>
> Yes. I know this is a bit unhappy. But it is quite rare, and to have 
> comment like <!-- tis htmlk is using JS libray palced in 
> someReproducer's resources --> seem enough to me.  And Imho there is 
> no simple cure for this.
>
> I really think that adding soeme palce for shared resources will make 
> the mess even bigger.
>
>
>> is this ever preferable?
>>
>>>
>>> I do not wot to block this much longer, so please chat about it 
>>> today evening all three of us.
>>>
>>> Sorry for disagreeing,
>>>  J.
>>>
>>
>> Thanks,
>>
>

I think then the right thing to do is not to leave things as they are 
but to also introduce a directory (maybe the same directory) where 
shared code can be placed. Jie proposed this before but I didn't see the 
need for it currently, but it is required if the reproducers are ever 
going to be run in isolation, which I would really like to see become 
possible. I don't know if there are any additional difficulties in 
implementing shared code vs simply shared resources, since the shared 
resources just have to be copied.

Thanks,

-- 
Andrew A