[rfc] [icedtea-web] blacklist for reproducers

[Omair Majid]

On 04/12/2012 08:05 AM, Jiri Vanek wrote:
>> [ snip lots of analysis ]

So I am going to preface this email by pointing out that I like the
overall change that you are proposing. Most of my
arguments/disagreements/suggestions are about minute details. Perhaps
you should just ignore them :)

> So here is my suggestion, why so, see inline below. Namely the changes are:
> * jnlp_testengine renamed to testsExtensions, and remains in tests
> directory

This is probably going into the area of bikeshedding, but I feel the
current name is better. Looking at existing names, I would be happy with
jnlp-test-engine (jnlp-test-runner, or maybe even jnlp-runner).

> * all tests from jnlp_testengine are dedicated to
> tests/testsExtensionsTests

I am guessing you meant 'moved'.

> * testsExtensions I would like to pack into junit-runner.jar

Is this necessary? I would like to keep them separate. When I originally
wrote the code for junit-runner (licensed under CPL), I intentionally
kept it separate from the rest of the (L)GPL(+Classpath) code.

> * directory jnlp_tests have been renamed to reproducers and moved from
> tests to tests/netx

Sounds fantastic!

>> And I agree; jnlp_testengine contains too much and does not really
>> belong there.
> Yap I have renamed it to testsExtensions (or tests-extensions?) For now
> this extension is just virtual server and few util methods, But with eg
> [1] or [2] there will be little bit more, and there always can be more
> extensions necessary in future.

Hm.. I don't know what's the right thing to do here. But I feel like
extensions for testing (that is, extra annotations, utility methods)
should be kept separate from the test engine (which just runs tests).

>>
>>> 1) move all tests from tests/netx/jnlp_testsengine to tests/netx/unit
>>> where they belongs IMHO
>>
>> A slightly different approach might be to move jnlp_testsengine itself
>> to a top level directory in icedtea-web.
> 
> Hmhm. I'm against top level directory. Overall - it is still just for
> testing, so it should stay in tests. Also there is eg junit-runner in
> tests. Those two subdirectories (junit-runner and testsExtensions) 
> should be on same level.

My reasoning for moving it to a top level directory is that it needs
tests, while junit-runner doesn't :)

>> jnlp_testsengine isn't a test
>> itself so I would be happy if we moved it out from under tests/. I dont
>> think moving the tests from tests/netx/jnlp_testsengine to
>> tests/netx/unit is a good idea; they are not netx's tests. How about
>> creating another directory (jnlp_testengine under tests/) to contain
>> tests for the jnlp_engine itself? (which are distinct from netx's or the
>> plugin's tests).
> Aboslutly agree:)
> So what about testsExtensionsTests (or tests-extensions-tests) on same
> level as netx (which contains unittests and  pactests (and I would like
> to/I have  move(d) reproducers into here) and styles and junit-runne
> rand testsExtensions?

I agree that testsExtensionTests (can we have a better name, please?)
should be in the same level as netx.

> but - when to run content of testsExtensionsTests? I would like to
> include it inside netx unit test (or let it in netx reproducers tests
> run as it is now) rather then run special tests-run for them.

I think it would make sense to run it when we build jnlp_testengine.
This ensure that the jnlp_testengine just built works as expected.


>>> 3) during reproducers run launch *just* compiled testcases
>>
>> Could you clarify what you mean here?
> Right now also classes which are not "test-calses" (eg everything under
> jnlp_testengine) are launched. It is definietly wrong. Some of this will
> be easily fixed by this refactoring, but some issues can remain.

Ah. so by "just" you meant "only" (as opposed to what just means in the
acronym JIT). Makes complete sense now :)


> Here I have (maybe interesting) idea:
> All our java tests are executed from junit-runner classes, which are
> packed to junit-runner.jar
> I would (very!) like to include also all the testsExtensions clases in
> this jar. It will make the unifications of classpaths much more
> straightforward (and this jar hmhm... can be useful by its cntent)

Please see my note about about mixing the CPL licensed code with other
things.

Cheers,
Omair