Testing Jigsaw modules

[Sander Mak]

Hi Gunnar,

I think you're going to want to read this (large...) thread: http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-February/006366.html (continued at http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-March/006587.html), which towards the end (especially http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-March/007152.html) nicely lays out the options for white/black-box testing. You can use -Xmodule/-Xpatch to compile and run your test code as if it were part of the module under test, which addresses at least your 3rd point.


Sander

On 01 Jun 2016, at 15:06, Gunnar Morling <gunnar at hibernate.org<mailto:gunnar at hibernate.org>> wrote:

Hi,

Are the already best practices and recommendations when it comes to testing
Jigsaw modules?

Having made some good progress with enabling Hibernate Validator to be used
with the module system, I'm looking into running our test suite on Jigsaw
now.

I figured I'd declare a separate module, so I can add requirements for the
test dependencies we have (e.g. TestNG). Some observations from that:

1) We have lots of unit tests which directly instantiate and test non-API
classes from the module under test. I added -XaddExports for essentially
all the non-exported packages for compiling and running the test module.
This works but seems quite verbose.

2) TestNG reflectively instantiates the classes to be tested. For that I
needed to export all the packages in the test module to the TestNG module.
Again a bit verbose.

3) There are cases where a test in the test module is located in the same
package as the class under compilation. This is disallowed by the module
system, so I'm unsure how I should go about unit testing classes with
default visibility.

4) Hibernate Validator reflectively accesses types from the user's
application in order to do its job. These classes are not necessarily part
of the user module's API, e.g. they might wish to put Bean Validation
constraints to an internally used model type. For that to work, IIUC, the
user needs to add qualified exports of these internal packages to Hibernate
Validator. Is that correct?

5) Using the module system revealed a wrong import in our code base
(ValidationException from xml.bind instead of Bean Validation).

Some questions:

Should tests be run with Jigsaw enabled to begin with? I think yes, so one
e.g. is sure that module descriptors are correct and complete but I would
be interested in what others think.

Regarding 1) and 2), I'm wondering whether a more flexible syntax for
-XaddExports would be useful (include/exclude with wildcards?). Admittedly,
though, an IDE or build tool would likely be able to help with that in
practice, once tools have caught up with Jigsaw. Also a notion similar to
"fragment bundles" in OSGi would help, as that'd essentially allow tests to
be part of the module under test, solving 1).

3) seems like a real issue to me if indeed it means one cannot test
default-visible classes with the module system enabled.

Regarding 4),  would the user have to do that for the combination of all
packages and all reflection-using libraries they work with? If so, that
seems like a considerable piece of work for them. Some more powerful switch
akin to "allow modules X, Y and Z to access all non-exported types" might
be handy, compared to listing all packages one by one.

On 5), kudos to the module system for that!

Any thoughts around this? I'd be very glad to hear how others are dealing
with these issues.

Thanks,

--Gunnar