[rfc][icedtea-web] Fix CacheReproducerTest

[Omair Majid]

Hi,

On 08/30/2013 06:06 PM, Jacob Wisor wrote:
> I always assumed reproducers to be unpredictable on any random system
> by definition?

I am going to ask Jiri to chime in here. He created the reproducer
system so he has probably the best idea of what it is supposed to be :)

That said, I have always taken reproducers to be system-level
integration/regression tests. That is, they test that the entire system
works (either proactively or as a result of a bug fix). They should
always pass.

I don't see the point of tests if the unpredictable. That's almost as
bad as not having tests to begin with - you are forced to run tests and
look at results and analyze them manually.

> I may be wrong here, but in my understanding a reproducer is not a
> general test, it is a special case or kind of test. A reproducer
> tests for or against a specific configuration and/or specific system
> and hence is allowed to fail on some systems.

I think it makes more sense for the test to declare itself as passing if
it is not applicable on a system (eg: it tests a POSIX-specific quirk).
Or did you mean something else?

> Imho a reproducer
> should not cover default configuration tests. It just gives a hint on
> what to fix to make an application or program work on a specific
> system or with a specific configuration. That said, a reproducer's
> results are per se unpredictable by definition. They are of course
> predictable with a specific configuration and/or on a specific
> system.
> 
> Consequently imho testing for the default configuration should be put
> into a general test that must succeed on any system. Any specific
> configuration tests should be put into a reproducer that is allowed
> to fail on some systems (until the app/program has been adapted).

Well, I guess we may have some different ideas about what a reproducer
is supposed to be, but at least we both agree that we need system-level
tests that check that the default configuration is correct :)

Cheers,
Omair

-- 
PGP Key: 66484681 (http://pgp.mit.edu/)
Fingerprint = F072 555B 0A17 3957 4E95  0056 F286 F14F 6648 4681