[rfc][icedtea-web] Refactor of LiveConnect Tests Version 2

[Jiri Vanek]

On 06/18/2014 05:38 PM, Andrew Azores wrote:
> On 06/18/2014 10:52 AM, Jiri Vanek wrote:
>> On 06/18/2014 03:28 PM, Andrew Azores wrote:
>>> On 06/18/2014 06:08 AM, Jiri Vanek wrote:
>>>> On 06/17/2014 05:28 PM, Andrew Azores wrote:
>>>>> On 06/12/2014 03:58 PM, Jie Kang wrote:
>>>>>> Hello,
>>>>>>
>>>>>>> I ran the reproducer suite with this patch applied last night. There are
>>>>>>> still some tests which are failing but not marked as KnownToFail. I
>>>>>>> haven't checked if these failures are repeatable however. Can you look
>>>>>>> into this?
>>>>>>>
>>>>>>> JavascriptSetTest - AppletJToJSSet_2DArrayElement_Test
>>>>>>> JSToJFuncResolTest - AppletJSToJFuncResol_inheritedClassToParent1_Test
>>>>>>> JavascriptFuncReturnTest -
>>>>>>> AppletJToJSFuncReturn_{number,boolean,JSObject,String,Object}_Test
>>>>>>>
>>>>>> I've checked the failures and fixed the FuncReturn test cases (typo).
>>>>>>
>>>>>> The JavascriptSetTest has some weird behaviour where the failure occurs because the applet's init
>>>>>> method didn't print out "applet initialized". This failure does not happen very often at all
>>>>>> and I
>>>>>> will look into it.
>>>>>>
>>>>>> The JSToJFuncResolTest failure has been marked Known to Fail. I've checked the test and found
>>>>>> another bug in the LiveConnect code unrelated to the fixes made in this patch. I have prepared
>>>>>> another patch that fixes this bug and will send it after this patch.
>>>>>>
>>>>>> Thanks for the review!
>>>>>>> Thanks,
>>>>>>>
>>>>>>> --
>>>>>>> Andrew A
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Jie Kang
>>>>>
>>>>> Just ran the tests, everything appears to be sorted out now.
>>>>>
>>>>> I'll wait before pushing to see if Jiri is strongly objecting to the "shared" resources folder,
>>>>> otherwise, I think this is good to go.
>>>> Unluckily, yes, I'm still a bit against. I probably do not understand the usecase "lunch one
>>>> reproducer"
>>>>
>>>> When I'm running one reproducer, then I allow it in whitelist, and then engine prepare *all*
>>>> reproducers, from which my one is lunched.
>>>
>>> Yea, that's exactly what I think is most problematic about ITW's reproducer suite. It's huge and
>>> sluggish. The use case for running one reproducer in isolation is that it is then much faster and
>>> more convenient to run tests targeting a specific patch being developed.
>>
>> By no other way you can run any reproducer, which is using another reproducer's code.
>>
>>>
>>>>
>>>> My mine reason against the explicit shared resources is, that actually also freshly prepared
>>>> (compiled!) signed,  custom and simple reproducers can be shared with each other.
>>>>
>>>> I Really think adding special case to copy some special resources is misleading from point of
>>>> above sentence.
>>>
>>> I'm not sure what you mean about the signed, custom, and simple being shared with each other. Yes,
>>> they can share resources, or even compiled classes. How is it not better to have the shared
>>
>> to me, it osunds more confusing to have shared resources handled by different way then shared code.
>>
>>> resources in a directory labelled "shared resources" rather than hidden away in "some" resources
>>> directory of "some" reproducer that happens to have them? If you are working on a reproducer that
>>> uses shared resources and it is not the reproducer that "hosts" those shared resources, how do you
>>> know where to actually find them? You have to actually go use find or grep or something on the
>>> entire tests/reproducers directory to discover that it's in some completely unrelated location. Why
>>
>> Yes. I know this is a bit unhappy. But it is quite rare, and to have comment like <!-- tis htmlk
>> is using JS libray palced in someReproducer's resources --> seem enough to me.  And Imho there is
>> no simple cure for this.
>>
>> I really think that adding soeme palce for shared resources will make the mess even bigger.
>>
>>
>>> is this ever preferable?
>>>
>>>>
>>>> I do not wot to block this much longer, so please chat about it today evening all three of us.
>>>>
>>>> Sorry for disagreeing,
>>>>  J.
>>>>
>>>
>>> Thanks,
>>>
>>
>
> I think then the right thing to do is not to leave things as they are but to also introduce a
> directory (maybe the same directory) where shared code can be placed. Jie proposed this before but I
> didn't see the need for it currently, but it is required if the reproducers are ever going to be run
> in isolation, which I would really like to see become possible. I don't know if there are any
> additional difficulties in implementing shared code vs simply shared resources, since the shared
> resources just have to be copied.


Well you are thinking about the shared code as about some library, which appelts/apps will reuse. 
That must not happen. The reproducers myust be as isolated as possible.


Even more we are speaking about this,. even more I'm sure the shared dir, as was introduced can nto 
go in.

But Patch, which will add the functionality to lunch single reproducer (and be able to handle its 
dependeces) may come in as separate (quite a big one I would guess) changeset.

J.