[rfc] [icedtea-web] blacklist for reproducers

[Omair Majid]

On 04/10/2012 07:42 AM, Jiri Vanek wrote:
> I was  afraid you will say something like this...

You know me too well ;)

> So lets dig what is necessary to do.
> We have

[ snip lots of analysis ]

Thanks for the in-depth analysis.

And I agree; jnlp_testengine contains too much and does not really
belong there.

> 1) move all tests from tests/netx/jnlp_testsengine to tests/netx/unit
> where they belongs IMHO

A slightly different approach might be to move jnlp_testsengine itself
to a top level directory in icedtea-web. jnlp_testsengine isn't a test
itself so I would be happy if we moved it out from under tests/. I dont
think moving the tests from tests/netx/jnlp_testsengine to
tests/netx/unit is a good idea; they are not netx's tests. How about
creating another directory (jnlp_testengine under tests/) to contain
tests for the jnlp_engine itself? (which are distinct from netx's or the
plugin's tests).

> 1.5) With this refactoring I would like to extract all inner classes
> from ServerAccess

+1 from me

> 2) fix compile and runtime class-path of unittests for
> tests/netx/jnlp_testsengine

This sounds fine.

> 3) during reproducers run launch *just* compiled testcases

Could you clarify what you mean here?

> 4) no more tests in tests/netx/jnlp_testsengine, framework will be
> tested in unittests (same as (1) but just for ensuring..:) )

See my points for 1, then :)

> 4.5) unittets and reprodyucers runs should have same classapth in
> runtime/compiletime. => unify them in some variable(s)(?)

No objections here (if it works, of course; there may be some subtle
issues).

> 5) I'm also for adding tests/netx/jnlp_testsengine to compile class-path
> of junit-runner (as it looks useful from [1,2] point of view.

I haven't seen [1] or [2] yet. But we can consider that separately,
right? I am not opposed to it - but let's consider that on it's own merits.

> 6) Maybe after all of above done the jnlp_testsengine can be renamed and
> moved somewhere else. (and also jnlp_tests probably should be moved to
> tests/netx/) But This should be done as separate changeset.

I was going to suggest that myself. +1

> 1-6 - ServerAccess api should not be changed, although maybe some
> methods private/public/protected statuses will be transformed due to
> moving of tests)

It's not a public API, so we can live with making changes to it, though
it wont be pretty. And reviewing it will be painful.

> What do you think?

This sounds fantastic!

> For launching just some tests I would like then introduce some
> whitelist. The logic will be same as was introduced in  this discarded 
> patch, but default value will be ".*"  Before wanted run it will be
> possible to enter regex matching the set of reproducers determined for run.

Sure, that sounds great; I am very happy with this. I would be happy to
review the patches, but please try and keep them small/separate.

Cheers,
Omair