[rfc] [icedtea-web] blacklist for reproducers

[Jiri Vanek]

> [ snip lots of analysis ]

So here is my suggestion, why so, see inline below. Namely the changes are:
* jnlp_testengine renamed to testsExtensions, and remains in tests directory
* all tests from jnlp_testengine are dedicated to tests/testsExtensionsTests
* testsExtensions I would like to pack into junit-runner.jar
* directory jnlp_tests have been renamed to reproducers and moved from tests to tests/netx
==:


├── extra
│   └── net
│       └── sourceforge
│           └── javaws
│               └── about
│                   └── resources
├── launcher
├── netx
│   ├── javax
│   │   └── jnlp
│   └── net
│       └── sourceforge
│           ├── jnlp
...
│           └── nanoxml
├── plugin
│   ├── docs
│   ├── icedteanp
│   │   └── java
...
│   └── tests
│       └── LiveConnect
└── tests
     ├── junit-runner
     ├── netx
     │   ├── pac
     │   ├── reproducers
     │   │   ├── signed
...
     │   │   └── simple
...
     │   └── unit
     │       └── net
     │           └── sourceforge
     │               └── jnlp
...
     ├── report-styles
     ├── testsExtensions
     │   └── net
     │       └── sourceforge
     │           └── jnlp
     └── testsExtensionsTests
         └── net
             └── sourceforge
                 └── jnlp

> And I agree; jnlp_testengine contains too much and does not really
> belong there.
Yap I have renamed it to testsExtensions (or tests-extensions?) For now this extension is just 
virtual server and few util methods, But with eg [1] or [2] there will be little bit more, and there 
always can be more extensions necessary in future.
>
>> 1) move all tests from tests/netx/jnlp_testsengine to tests/netx/unit
>> where they belongs IMHO
>
> A slightly different approach might be to move jnlp_testsengine itself
> to a top level directory in icedtea-web.

Hmhm. I'm against top level directory. Overall - it is still just for testing, so it should stay in 
tests. Also there is eg junit-runner in tests. Those two subdirectories (junit-runner and 
testsExtensions)  should be on same level.
>jnlp_testsengine isn't a test
> itself so I would be happy if we moved it out from under tests/. I dont
> think moving the tests from tests/netx/jnlp_testsengine to
> tests/netx/unit is a good idea; they are not netx's tests. How about
> creating another directory (jnlp_testengine under tests/) to contain
> tests for the jnlp_engine itself? (which are distinct from netx's or the
> plugin's tests).
Aboslutly agree:)
So what about testsExtensionsTests (or tests-extensions-tests) on same level as netx (which contains 
unittests and  pactests (and I would like to/I have  move(d) reproducers into here) and styles and 
junit-runne rand testsExtensions?

but - when to run content of testsExtensionsTests? I would like to include it inside netx unit test 
(or let it in netx reproducers tests run as it is now) rather then run special tests-run for them.
>
>> 1.5) With this refactoring I would like to extract all inner classes
>> from ServerAccess
>
> +1 from me
>
>> 2) fix compile and runtime class-path of unittests for
>> tests/netx/jnlp_testsengine
>
> This sounds fine.
>
>> 3) during reproducers run launch *just* compiled testcases
>
> Could you clarify what you mean here?
Right now also classes which are not "test-calses" (eg everything under jnlp_testengine) are 
launched. It is definietly wrong. Some of this will be easily fixed by this refactoring, but some 
issues can remain.
>
>> 4) no more tests in tests/netx/jnlp_testsengine, framework will be
>> tested in unittests (same as (1) but just for ensuring..:) )
>
> See my points for 1, then :)
>
>> 4.5) unittets and reprodyucers runs should have same classapth in
>> runtime/compiletime. =>  unify them in some variable(s)(?)
>
> No objections here (if it works, of course; there may be some subtle
> issues).
probably...for sure.. Lets see how I will deal with it :)
>
>> 5) I'm also for adding tests/netx/jnlp_testsengine to compile class-path
>> of junit-runner (as it looks useful from [1,2] point of view.
>
> I haven't seen [1] or [2] yet. But we can consider that separately,
> right? I am not opposed to it - but let's consider that on it's own merits.

Here I have (maybe interesting) idea:
All our java tests are executed from junit-runner classes, which are packed to junit-runner.jar
I would (very!) like to include also all the testsExtensions clases in this jar. It will make the 
unifications of classpaths much more straightforward (and this jar hmhm... can be useful by its cntent)
>
>> 6) Maybe after all of above done the jnlp_testsengine can be renamed and
>> moved somewhere else. (and also jnlp_tests probably should be moved to
>> tests/netx/) But This should be done as separate changeset.
>
> I was going to suggest that myself. +1
..well.. necessary - done somehow by itself alone...
>
>
>For launching just some tests I would like then introduce some whitelist

whitelist will be separate patch..somewhen

>

One moire think is that there is some work in progress - especially browser tests[1] (rigt now 
blocked by make links) and some  minor improvements [2], (and maybe  also signed applets and pr905 
reproducer) which I would like to put inside before this re factoring....But hose steps can 
collaborate ..somehow.
And it will give us(well..me.. I'm the rush one here...:-/) time to think about it.

[1]http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2012-March/017799.html
[2]http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20120320/8ef6bdfe/browserTests-BugAnnotation-0001.diff 


O:)

Best regards J.