From sormuras at gmail.com  Wed Feb  5 03:02:44 2020
From: sormuras at gmail.com (Christian Stein)
Date: Wed, 5 Feb 2020 04:02:44 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte, from jdk-dev]
Message-ID: <CAJi8+g6fw1irjrtAL5Z_bMaiuXeY_gnm33Gbkh5jpRNXS3Ww-g@mail.gmail.com>

Hi,

I took the freedom moving this thread to "jigsaw-dev", as
suggested by David Holmes, in order to reply to it in a
follow-up mail. I hope, Robert doesn't mind that I copy his
text from "jdk-dev" below to start with the same context.

Robert Scholte wrote in
http://mail.openjdk.java.net/pipermail/jdk-dev/2020-February/003880.html

background:
while developing the maven-compiler-plugin I've often resisted for adding
module descriptors to the test-classes.
Test classes won't result in a deliverable.
I've been able to construct the right set of arguments for the compiler
based on the pom.xml and the module descriptor and I hadn't seen an example
yet that should change my opinion.

However, recently I received a small project that required java.sql, but
only during test. Because they didn't really understand why the
test-compilation failed, this module was added to the module descriptor
with a comment ( /* only for test */ )
My advice was replacing it with the following to the maven-compiler-plugin:
<execution>
  <id>default-testCompile</id>
  <configuration>
    <compilerArgs>
      <compilerArg>--add-modules=java.sql</compilerArg>
    </compilerArgs>
  </configuration>
</execution>


But something similar needs to happen for surefire, to run the tests.

This is sadly one example where the user is forced to understand these
additional flags. I could try to solve this with a custom (Maven) solution
to avoid the need for flags configuration, but that would make it tool
specific, whereas I believe this should be solved by the JDK.

What might have helped would be something like this:
src/test/java/module-info.java
[....] {
  requires java.sql;
}

I've put [...] on purpose, because in this case I don't think the module
should have the same name as the "main" module descriptor, because it is
that module. An anonymous module is already better, but that is not allowed
in the descriptor.

One solution that might fit here is the introduction of a new modifier:
patch
patch module some.module {

   requires java.sql;
}
This describes clear the purpose of the module, and as the "patch" already
implies: it should be handled with care.
Allowing such module descriptor should help adopting the modular system
faster. It doesn't require the knowledge of the module related flags, the
buildtool can solve it. As the developer is already familiar with the main
module descriptor, adding a patched module descriptor when required is just
a simple step.

Both java and javac could benefit from this change, and you might consider
to not allow to package with a patched module descriptor.

regards,
Robert Scholte

From sormuras at gmail.com  Wed Feb  5 03:47:07 2020
From: sormuras at gmail.com (Christian Stein)
Date: Wed, 5 Feb 2020 04:47:07 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte,
 from jdk-dev]
In-Reply-To: <CAJi8+g6fw1irjrtAL5Z_bMaiuXeY_gnm33Gbkh5jpRNXS3Ww-g@mail.gmail.com>
References: <CAJi8+g6fw1irjrtAL5Z_bMaiuXeY_gnm33Gbkh5jpRNXS3Ww-g@mail.gmail.com>
Message-ID: <CAJi8+g7it57f4W45dm+W1zxh707vpCUV0TdBbpNyCVxrSBzGJA@mail.gmail.com>

// Hope this email will be technically associated as a reply for the
// "RFE simplify usage of patched module" thread.

I agree with Robert that a user-friendly and (build) tool-agnostic way
to express additional module directives and modifiers for testing
purposes only is ... a good thing to have.

As a workaround, I proposed, implemented and promoted a tool-agnostic
`module-info.test` file in [0] which, as of today, I no longer really like
that
much. The standard javac/java command options and arguments listed
as plain text lines in that file require users to know the module-related
options of javac/java meant to be primarily used by (build) tools.

Perhaps this discussion leads to a solution that supersedes the purpose
of "module-info.test" and renders
https://youtrack.jetbrains.com/issue/IDEA-222831
as no longer needed.

Robert Scholte wrote in
> http://mail.openjdk.java.net/pipermail/jdk-dev/2020-February/003880.html
>
> [...]
> What might have helped would be something like this:
> src/test/java/module-info.java
> [....] {
>   requires java.sql;
> }
>
> I've put [...] on purpose, because in this case I don't think the module
>
should have the same name as the "main" module descriptor, because it
>
is that module. [...]
>


This "solution" already works today. As described in [0] you may use the
existing module-info.java-DSL to describe such an additions (of extra
modules
needed for testing) by (I) repeating (most) directives from the "main"
module
descriptor and (II) add those needed for testing. This includes adding more
module modifiers, like `open` as well.

Using the same module name as `src/main/java/module-info.java`, of course.


> One solution that might fit here is the introduction of a new modifier:
> patch
> patch module some.module {
>
>    requires java.sql;
> }
> This describes clear the purpose of the module, and as the "patch" already
>
implies: it should be handled with care. [...]
>

Such a modifier would remedy the need to repeat directives from the "main"
module. It would effectively render step (I) from above not longer required.

Quote from [0]:
> Note: Copying parts from the main module descriptor manually is brittle.
> The ?Java 9 compatible build tool? pro[1] solves this by auto-merging a
> main and test module descriptor on-the-fly.

Robert, you chose `some.module` as the name in this example? Is it
the same name as the "main" module?

Cheers,
Christian

[0]:
https://sormuras.github.io/blog/2018-09-11-testing-in-the-modular-world#white-box-modular-testing-with-module-infojava

[1]: https://github.com/forax/pro

From sormuras at gmail.com  Wed Feb  5 04:19:47 2020
From: sormuras at gmail.com (Christian Stein)
Date: Wed, 5 Feb 2020 05:19:47 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte,
 from jdk-dev]
In-Reply-To: <CAJi8+g7it57f4W45dm+W1zxh707vpCUV0TdBbpNyCVxrSBzGJA@mail.gmail.com>
References: <CAJi8+g6fw1irjrtAL5Z_bMaiuXeY_gnm33Gbkh5jpRNXS3Ww-g@mail.gmail.com>
 <CAJi8+g7it57f4W45dm+W1zxh707vpCUV0TdBbpNyCVxrSBzGJA@mail.gmail.com>
Message-ID: <CAJi8+g6F2DUDAdmOFU85UZgU7kmu0zKCavttVRy89i0DGeL7ew@mail.gmail.com>

On Wed, Feb 5, 2020 at 4:47 AM Christian Stein <sormuras at gmail.com> wrote:

> // Hope this email will be technically associated as a reply for the
> // "RFE simplify usage of patched module" thread.
>
> I agree with Robert that a user-friendly and (build) tool-agnostic way
> to express additional module directives and modifiers for testing
> purposes only is ... a good thing to have.
>
> As a workaround, I proposed, implemented and promoted a tool-agnostic
> `module-info.test` file in [/] which, as of today, I no longer really like
> that
> much. The standard javac/java command options and arguments listed
> as plain text lines in that file require users to know the module-related
> options of javac/java meant to be primarily used by (build) tools.
>
>
Here's a Maven-based project that demonstrates most (all?) interesting
main/test scope mixing combinations: https://github.com/micromata/sawdust
Instead of using the Surefire plugin to launch tests, it uses [0]. The
latter
supports the `module-info.test` configuration file, which is used in one of
the examples: [1]

Relevant in this context are the two examples whose names starts with
"modular-whitebox-".

With the "patch modifier"-proposal applied as suggested by Robert,
this manually constructed test module descriptor:

open module foo {
  exports foo;
  requires org.junit.jupiter.api;
}

from [2] would shrink to:

open patch module foo {
  requires org.junit.jupiter.api;
}

Only the additional "open" modifier and the additional requires directive
remain.
The other (here the single exports directive) directives are copied from the
non-patched module "foo".

Cheers,
Christian

[0]: https://github.com/sormuras/junit-platform-maven-plugin
[1]:
https://github.com/micromata/sawdust/blob/master/modular-whitebox-patch-runtime/src/test/resources/module-info.test
[2]:
https://github.com/micromata/sawdust/blob/master/modular-whitebox-patch-compile/src/test/java/module-info.java

From simone.bordet at gmail.com  Wed Feb  5 07:18:48 2020
From: simone.bordet at gmail.com (Simone Bordet)
Date: Wed, 5 Feb 2020 08:18:48 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte,
 from jdk-dev]
Message-ID: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>

Hi,

> With the "patch modifier"-proposal applied as suggested by Robert,
> this manually constructed test module descriptor:
>
> open module foo {
>   exports foo;
>   requires org.junit.jupiter.api;
> }
>
> from [2] would shrink to:
>
> open patch module foo {
>   requires org.junit.jupiter.api;
> }

While I was involved in the original report, I have concerns about its security.

Would not anyone be able to patch an existing module without the
author's consent?
For example:

patch module org.eclipse.jetty.client {
  exports org.eclipse.jetty.client.internal;
  opens org.eclipse.jetty.client;
}

Doing the same on the command line keeps the end user in control,
rather than having the end user possibly scan hundreds of jar to see
if someone snuck in a patched module descriptor.

However, the need for such "test" module descriptor is evident.

What if patched module descriptors are only effective when a command
line option is present, say "--allow-patch-descriptors", or something
like that?

Thanks!

-- 
Simone Bordet
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless.   Victoria Livschitz

From rfscholte at apache.org  Wed Feb  5 07:38:16 2020
From: rfscholte at apache.org (Robert Scholte)
Date: Wed, 05 Feb 2020 08:38:16 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
Message-ID: <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>

Hi Simone,

I understand your concern, but the patched module descriptor doesn't have to (or should not) replace the?--patch-module option. This proposal is about the additional options you now need to put on the commandline, but which already fit in the module descriptor.

thanks,
Robert
On 5-2-2020 08:19:21, Simone Bordet <simone.bordet at gmail.com> wrote:
Hi,

> With the "patch modifier"-proposal applied as suggested by Robert,
> this manually constructed test module descriptor:
>
> open module foo {
> exports foo;
> requires org.junit.jupiter.api;
> }
>
> from [2] would shrink to:
>
> open patch module foo {
> requires org.junit.jupiter.api;
> }

While I was involved in the original report, I have concerns about its security.

Would not anyone be able to patch an existing module without the
author's consent?
For example:

patch module org.eclipse.jetty.client {
exports org.eclipse.jetty.client.internal;
opens org.eclipse.jetty.client;
}

Doing the same on the command line keeps the end user in control,
rather than having the end user possibly scan hundreds of jar to see
if someone snuck in a patched module descriptor.

However, the need for such "test" module descriptor is evident.

What if patched module descriptors are only effective when a command
line option is present, say "--allow-patch-descriptors", or something
like that?

Thanks!

--
Simone Bordet
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz

From simone.bordet at gmail.com  Wed Feb  5 07:57:32 2020
From: simone.bordet at gmail.com (Simone Bordet)
Date: Wed, 5 Feb 2020 08:57:32 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte,
 from jdk-dev]
In-Reply-To: <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
Message-ID: <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>

Hi Robert,

On Wed, Feb 5, 2020 at 8:38 AM Robert Scholte <rfscholte at apache.org> wrote:
>
> Hi Simone,
>
> I understand your concern, but the patched module descriptor doesn't have to (or should not) replace the --patch-module option. This proposal is about the additional options you now need to put on the commandline, but which already fit in the module descriptor.

I understand it does not replace --patch-module.
I understand it adds the additional "requires", "opens", etc.

But how do you stop a library that uses Jetty to ship a jar containing
a patched module file that exports and opens things in Jetty that the
Jetty authors did not want to export or open, without users knowing
it?

jetty-client.jar -> contains legit module-info.class
library.jar -> contains patched descriptor that patches jetty-client
app.jar -> my application with a legit module-info.class

java --module-path jetty-client.jar:library.jar:app.jar --module
app/com.app.Main

With this command line, does the Java runtime parse and enable the
patched descriptor contained in library.jar, opening up jetty-client?
If not, how would you enable it in Maven?

Am I missing something?

Thanks!

-- 
Simone Bordet
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless.   Victoria Livschitz

From rfscholte at apache.org  Wed Feb  5 16:44:26 2020
From: rfscholte at apache.org (Robert Scholte)
Date: Wed, 05 Feb 2020 17:44:26 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
Message-ID: <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>

Hi Simone,

to me the commandline doesn't change, so in your case the library.jar won't patch jetty-client.jar.

It will only be patch as you already would do right now:
java --module-path jetty-client.jar:library.jar:app.jar --module?app/com.app.Main --patch-module jetty.client=/path/to/library.jar


For Maven plugins I don't expect a lot of changes. I should probably use the name in the patched module descriptor instead of automatically choosing the main module descriptor, but these are little adjustments.

Having a patched module descriptor in a jar might be awkward, hence maybe the packager shouldn't allow to add it, nor other tools to use it.
But these are details open for discussion.

Robert
On 5-2-2020 08:57:53, Simone Bordet <simone.bordet at gmail.com> wrote:
Hi Robert,

On Wed, Feb 5, 2020 at 8:38 AM Robert Scholte wrote:
>
> Hi Simone,
>
> I understand your concern, but the patched module descriptor doesn't have to (or should not) replace the --patch-module option. This proposal is about the additional options you now need to put on the commandline, but which already fit in the module descriptor.

I understand it does not replace --patch-module.
I understand it adds the additional "requires", "opens", etc.

But how do you stop a library that uses Jetty to ship a jar containing
a patched module file that exports and opens things in Jetty that the
Jetty authors did not want to export or open, without users knowing
it?

jetty-client.jar -> contains legit module-info.class
library.jar -> contains patched descriptor that patches jetty-client
app.jar -> my application with a legit module-info.class

java --module-path jetty-client.jar:library.jar:app.jar --module
app/com.app.Main

With this command line, does the Java runtime parse and enable the
patched descriptor contained in library.jar, opening up jetty-client?
If not, how would you enable it in Maven?

Am I missing something?

Thanks!

--
Simone Bordet
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz

From sormuras at gmail.com  Thu Feb  6 07:35:58 2020
From: sormuras at gmail.com (Christian Stein)
Date: Thu, 6 Feb 2020 08:35:58 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte,
 from jdk-dev]
In-Reply-To: <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
Message-ID: <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>

I agree with you, that the user (the build tool in most of the cases) should
stay in control which module descriptor is in effect for every module.

As my domain is testing, and I guess the majority of uses-cases is within
the realm of intra-module (white-box) testing, I'll expound on the topic
with
a complete example.


# GIVEN

- src/foo/main/java/module-info.java

module foo {
  // here be dragons
}

- src/foo/test/java/module-info.java

open $PATCH module foo {
  // here be same dragons as "main" foo declares
  requires org.junit.jupiter.api;
  requires org.assertj.core;
  requires java.sql;
}

Note, without the new $PATCH feature, you may just copy and paste
the "main" foo directives to the "test" foo module already today.


# WHEN

Compile the "test" module against the already compiled and optionally
packaged "main" module via:

javac --module-source-path src/*/test/java --patch-module
foo=out/classes/main/foo --module foo
jar --create --file out/modules/test/foo.jar -C  out/classes/test/foo .


# THEN

Execute JUnit Platform Console Launcher selecting the "test" foo module
and scanning it for tests. Here, all 3rd-party modules are already put into
a project-local "lib/" directory:

java
  --module-path out/modules/test/foo.jar:out/modules/main:lib
  --patch-module foo=out/modules/main/foo.jar
  --module org.junit.platform.console
  --scan-module foo


You see, there's still a command line switch required to get the patching
going. This command will (a) load the patched foo module, that one that
reads all modules required for testing and (b) resolve the classes under
test from the main module.

Yes, if someone (or the build tool) jars the test and main classes into
a single JAR file, you may create a "patched by default" module which
is no in control of the original author -- but, no change to what you
may already do today.

Cheers,
Christian


On Wed, Feb 5, 2020 at 5:44 PM Robert Scholte <rfscholte at apache.org> wrote:

> Hi Simone,
>
> to me the commandline doesn't change, so in your case the library.jar
> won't patch jetty-client.jar.
>
> It will only be patch as you already would do right now:
> java --module-path jetty-client.jar:library.jar:app.jar
> --module app/com.app.Main --patch-module jetty.client=/path/to/library.jar
>
>
> For Maven plugins I don't expect a lot of changes. I should probably use
> the name in the patched module descriptor instead of automatically choosing
> the main module descriptor, but these are little adjustments.
>
> Having a patched module descriptor in a jar might be awkward, hence maybe
> the packager shouldn't allow to add it, nor other tools to use it.
> But these are details open for discussion.
>
> Robert
> On 5-2-2020 08:57:53, Simone Bordet <simone.bordet at gmail.com> wrote:
> Hi Robert,
>
> On Wed, Feb 5, 2020 at 8:38 AM Robert Scholte wrote:
> >
> > Hi Simone,
> >
> > I understand your concern, but the patched module descriptor doesn't
> have to (or should not) replace the --patch-module option. This proposal is
> about the additional options you now need to put on the commandline, but
> which already fit in the module descriptor.
>
> I understand it does not replace --patch-module.
> I understand it adds the additional "requires", "opens", etc.
>
> But how do you stop a library that uses Jetty to ship a jar containing
> a patched module file that exports and opens things in Jetty that the
> Jetty authors did not want to export or open, without users knowing
> it?
>
> jetty-client.jar -> contains legit module-info.class
> library.jar -> contains patched descriptor that patches jetty-client
> app.jar -> my application with a legit module-info.class
>
> java --module-path jetty-client.jar:library.jar:app.jar --module
> app/com.app.Main
>
> With this command line, does the Java runtime parse and enable the
> patched descriptor contained in library.jar, opening up jetty-client?
> If not, how would you enable it in Maven?
>
> Am I missing something?
>
> Thanks!
>
> --
> Simone Bordet
> ---
> Finally, no matter how good the architecture and design are,
> to deliver bug-free software with optimal performance and reliability,
> the implementation technique must be flawless. Victoria Livschitz
>

From sormuras at gmail.com  Fri Feb  7 12:41:22 2020
From: sormuras at gmail.com (Christian Stein)
Date: Fri, 7 Feb 2020 13:41:22 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte,
 from jdk-dev]
In-Reply-To: <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
Message-ID: <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>

On Thu, Feb 6, 2020 at 8:35 AM Christian Stein <sormuras at gmail.com> wrote:

> I agree with you, that the user (the build tool in most of the cases)
> should
> stay in control which module descriptor is in effect for every module.
>
> As my domain is testing, and I guess the majority of uses-cases is within
> the realm of intra-module (white-box) testing, I'll expound on the topic
> with
> a complete example.
>
>
> # GIVEN
>
> - src/foo/main/java/module-info.java
>

Reading this post a day later, I assume there more questions raised than
answers given by this "foo" example. If that's the case, I apologize for the
confusion -- my tunnel vision of "living on the testing side of the modular
world" led to jumps in the story I wanted to tell.

This time, I created a project at [0] with a detailed description on its
front
page, i.e the README.md file. I read that new story thrice and hope to have
a more complete example prepared. It is based on the "Project Jigsaw:
Module System Quick-Start Guide" [1] and extends the example modules
with testing. Java-based build programs accompany the project to make
it easy to "replay" (read: build) it on every supported platform.

If any new questions appear, I'm happy to answer those on this ML.

If you want me to copy "the new story" as plain text to this ML as well,
just ping me. I refrained copying it directly, as it spans some pages
and contains ASCII-art images, quotes, log excerpts...

Cheers,
Christian

[0]: https://github.com/sormuras/java-module-patching
[1]: https://openjdk.java.net/projects/jigsaw/quick-start

From alex.buckley at oracle.com  Mon Feb 10 22:03:13 2020
From: alex.buckley at oracle.com (Alex Buckley)
Date: Mon, 10 Feb 2020 14:03:13 -0800
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
Message-ID: <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>

Hi Christian,

On 2/7/2020 4:41 AM, Christian Stein wrote:
> This time, I created a project at [0] with a detailed description on its
> front page, i.e the README.md file.
> 
> [0]: https://github.com/sormuras/java-module-patching

To restate:

- You're saying that, today, it's brittle to copy directives from 
src/org.astro/main/java/module-info.java to 
src/org.astro/test/java/module-info.java. (And having copied them, you 
still need to `open` the test module and add some `requires`.)

- You suggest that, in future, there will still be a 
src/org.astro/test/java/module-info.java file which is of interest to 
test frameworks.

What you're hoping is that new syntax will let you invert the patching:

- Today, you set the module-path so that out/modules/test/org.astro.jar 
is the "primary" version of the module; then you redefine everything in 
it except module-info by overlaying out/modules/main/org.astro.jar.

- In future, you want to have out/modules/main/org.astro.jar as the 
"primary" version, and redefine only its module-info by specifying the 
sidecar out/modules/test/org.astro.jar. The sidecar would have some 
syntax to identify that its declaration of org.astro is strictly 
additive to the "primary" version. You would set the module-path to 
out/modules/main:out/modules/test:lib so that the module system (1) 
finds the "primary" version in out/modules/main and (2) augments its 
module-info with sidecars found in out/modules/test and lib. I assume 
some new command line option would enable or enumerate the sidecars 
explicitly, because obviously the dependences and exports of 
out/modules/main/org.astro.jar shouldn't depend on which JAR files 
happen to be lurking deep in the module-path.

Stepping back, the core issue is that once the true "primary" version of 
a module is built -- out/modules/main/org.astro.jar -- you don't want to 
change it. No build or test tool wants to physically rewrite its 
module-info to require test-time dependencies at test time, and then to 
not require such dependencies at release time. You want the module 
system to virtually rewrite the module-info instead. And that's already 
possible, as long as you put on your test-colored sunglasses and view 
out/modules/test/org.astro.jar as the "primary" version, and 
out/modules/main/org.astro.jar as the overlay ... once the tests have 
run, go back to viewing out/modules/main/org.astro.jar as the "primary" 
version. Introducing a new kind of module descriptor, with merging by 
the module system according to new command line options, seems like a 
lot of overhead that can already be worked around by tools at test-time.

Alex

From rfscholte at apache.org  Wed Feb 12 21:08:17 2020
From: rfscholte at apache.org (Robert Scholte)
Date: Wed, 12 Feb 2020 22:08:17 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
Message-ID: <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>

It is kind of unfortunate that this topic has been hijacked for something I don't see as an issue.
I've been able to build up the right set of commandline arguments for this case for both javac and java

With *just* the dependencies(respecting compile + test scope) from the pom, and the module descriptor I've been able to compile the main/test sources and run the tests.

However, recently I've informed by this case: if the test sources use one of the java.* modules (that are not used by the main sources) the only correct way to solve it now is by adding the required flags by hand (and only to the test-compile configuration!).
This is hard to explain and instead of diving into the specifications, understanding what's happening, you'll see that they choose for the easy workaround: add this "test scoped" module as a required module to the module descriptor.?
In the end this will do more harm than good.?A bit off-topic, but with Maven we already see that adding dependencies is simple, however cleaning up unused dependencies is not. Hence the exclude-option for dependencies is used frequently.?Unlike Maven dependencies, you cannot exclude required modules, so here it is even more important that the list of required modules is the minimum required list.

To prevent these workarounds and to provide an easier way to patch a module via a dedicated descriptor will help keeping the module system cleaner.
I'm very pleased I've been able to hide most logic behind the required flags.?
This is a valid use case and in my opinion deserves a solution without explicit knowledge of these flags by the enduser (a tool like Maven should solve this, but this can only be done if it has access to this information.)
Introducing an extra modifier looks like one solution, maybe there is a better one.

thanks,
Robert

On 10-2-2020 23:03:28, Alex Buckley <alex.buckley at oracle.com> wrote:
Hi Christian,

On 2/7/2020 4:41 AM, Christian Stein wrote:
> This time, I created a project at [0] with a detailed description on its
> front page, i.e the README.md file.
>
> [0]: https://github.com/sormuras/java-module-patching

To restate:

- You're saying that, today, it's brittle to copy directives from
src/org.astro/main/java/module-info.java to
src/org.astro/test/java/module-info.java. (And having copied them, you
still need to `open` the test module and add some `requires`.)

- You suggest that, in future, there will still be a
src/org.astro/test/java/module-info.java file which is of interest to
test frameworks.

What you're hoping is that new syntax will let you invert the patching:

- Today, you set the module-path so that out/modules/test/org.astro.jar
is the "primary" version of the module; then you redefine everything in
it except module-info by overlaying out/modules/main/org.astro.jar.

- In future, you want to have out/modules/main/org.astro.jar as the
"primary" version, and redefine only its module-info by specifying the
sidecar out/modules/test/org.astro.jar. The sidecar would have some
syntax to identify that its declaration of org.astro is strictly
additive to the "primary" version. You would set the module-path to
out/modules/main:out/modules/test:lib so that the module system (1)
finds the "primary" version in out/modules/main and (2) augments its
module-info with sidecars found in out/modules/test and lib. I assume
some new command line option would enable or enumerate the sidecars
explicitly, because obviously the dependences and exports of
out/modules/main/org.astro.jar shouldn't depend on which JAR files
happen to be lurking deep in the module-path.

Stepping back, the core issue is that once the true "primary" version of
a module is built -- out/modules/main/org.astro.jar -- you don't want to
change it. No build or test tool wants to physically rewrite its
module-info to require test-time dependencies at test time, and then to
not require such dependencies at release time. You want the module
system to virtually rewrite the module-info instead. And that's already
possible, as long as you put on your test-colored sunglasses and view
out/modules/test/org.astro.jar as the "primary" version, and
out/modules/main/org.astro.jar as the overlay ... once the tests have
run, go back to viewing out/modules/main/org.astro.jar as the "primary"
version. Introducing a new kind of module descriptor, with merging by
the module system according to new command line options, seems like a
lot of overhead that can already be worked around by tools at test-time.

Alex

From alex.buckley at oracle.com  Wed Feb 12 23:11:49 2020
From: alex.buckley at oracle.com (Alex Buckley)
Date: Wed, 12 Feb 2020 15:11:49 -0800
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
 <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
Message-ID: <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>

On 2/12/2020 1:08 PM, Robert Scholte wrote:
> To prevent these workarounds and to provide an easier way to patch a
> module via a dedicated descriptor will help keeping the module
> system cleaner.

It will lead to "split modules" on the modulepath, which will cause just 
as many maintenance headaches as split packages on the classpath. Yes, 
there is some maintenance benefit if a module explicitly declares that 
it patches (i.e. becomes part of) another module (which might then have 
to explicitly declare that it allows patching) ... but for a developer 
to understand the resulting module graph requires looking at everything 
on the modulepath, which is no better than having to look at everything 
on the classpath. In Java, a declaration -- whether a module, a class, 
or a method -- happens in one source file, and it's up to tools to 
rewrite declarations if other interesting source files are known to 
those tools.

> However, recently I've informed by this case: if the test sources
> use one of the java.* modules (that are not used by the main sources)
> the only correct way to solve it now is by adding the required flags
> by hand (and only to the test-compile configuration!). This is hard
> to explain and instead of diving into the specifications, 
> understanding what's happening, you'll see that they choose for the
> easy workaround: add this "test scoped" module as a required module
> to the module descriptor.

Is there nothing that Maven can do to make the test-compile 
configuration easier to create? Maven has all the source code at its 
fingertips, including knowledge of module directories which seem to 
declare the same module more than once because JUnit recommends it, yet 
still Maven makes the user laboriously write out the command line flags 
for patching?

Alex

From rfscholte at apache.org  Fri Feb 14 08:02:31 2020
From: rfscholte at apache.org (Robert Scholte)
Date: Fri, 14 Feb 2020 09:02:31 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
 <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
Message-ID: <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>

I am aware that the current patch concept is already complicating things.
Apart from the impact of the requested implementation, the information+options should all be available already: the patch module descriptor should be considered as an alias for some related commandline options.


With a tool like Maven it is already possible to compile and run patched modules, but in case of requiring java.* modules it is very awkward that it requires additional configuration in the pom instead of some dedicated module descriptor, which feels most natural.
You'll get an exception like:?package java.sql is declared in module java.sql, but module some.module.name does not read it
And the first thing that comes to mind is: add java.sql to the (main) module descriptor. But this should not be done.

Other options to avoid this would be:?
- source code scanning to figure out all used modules. To me not an option: way too expensive/resource consuming.
- by default add all java.* modules, since they should be available anyway.
All other solutions will be a tool specific solution, even though they're struggling all with the same problem.

To me having the correct (main) module descriptor should be the ultimate goal. People should not be tempted to apply workarounds on this descriptor to make things work for tests (even though those module are for sure part of the java runtime)
Especially because of the impact wrong module descriptors would have and which cannot be adjusted.

thanks,
Robert
On 13-2-2020 00:12:02, Alex Buckley <alex.buckley at oracle.com> wrote:
On 2/12/2020 1:08 PM, Robert Scholte wrote:
> To prevent these workarounds and to provide an easier way to patch a
> module via a dedicated descriptor will help keeping the module
> system cleaner.

It will lead to "split modules" on the modulepath, which will cause just
as many maintenance headaches as split packages on the classpath. Yes,
there is some maintenance benefit if a module explicitly declares that
it patches (i.e. becomes part of) another module (which might then have
to explicitly declare that it allows patching) ... but for a developer
to understand the resulting module graph requires looking at everything
on the modulepath, which is no better than having to look at everything
on the classpath. In Java, a declaration -- whether a module, a class,
or a method -- happens in one source file, and it's up to tools to
rewrite declarations if other interesting source files are known to
those tools.

> However, recently I've informed by this case: if the test sources
> use one of the java.* modules (that are not used by the main sources)
> the only correct way to solve it now is by adding the required flags
> by hand (and only to the test-compile configuration!). This is hard
> to explain and instead of diving into the specifications,
> understanding what's happening, you'll see that they choose for the
> easy workaround: add this "test scoped" module as a required module
> to the module descriptor.

Is there nothing that Maven can do to make the test-compile
configuration easier to create? Maven has all the source code at its
fingertips, including knowledge of module directories which seem to
declare the same module more than once because JUnit recommends it, yet
still Maven makes the user laboriously write out the command line flags
for patching?

Alex

From alex.buckley at oracle.com  Fri Feb 14 18:06:10 2020
From: alex.buckley at oracle.com (Alex Buckley)
Date: Fri, 14 Feb 2020 10:06:10 -0800
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
 <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
Message-ID: <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>

On 2/14/2020 12:02 AM, Robert Scholte wrote:
> To me having the correct (main) module descriptor should be the ultimate 
> goal. People should not be tempted to apply workarounds on this 
> descriptor to make things work for tests (even though those module are 
> for sure part of the java runtime)
> Especially because of the impact wrong module descriptors would have and 
> which cannot be adjusted.

Nowhere else in Java is there the ability to "patch" one declaration by 
making further declarations in source code, so I don't think there's a 
way forward for a `patch module ...` declaration. I understand that it 
looks good when viewed through the lens of testing, where there's 
exactly one secondary module declaration (the test module) and it's 
tightly scoped to support white-box tests. However, we have to consider 
if it's a good idea in general for a module to be defined by parts, 
where anyone could throw multiple `patch module foo {..}` declarations 
on the module path and have each of them modify the "master" foo module.

The answer is that it's not a good idea: (i) different `requires` 
clauses might clash (i.e. read modules which export the same packages), 
(ii) one patch might `open` the module while another patch doesn't want 
it open (and there's no way to say non-open), and (iii) the content of 
packages in the unified module is now determined by the order in which 
the "master" module and the "patch" modules are deployed on the module path.

These things are so undesirable that even `--patch-module` is unable to 
override module declarations! That said, command line options can have 
superpowers that would never be expressible in source code declarations, 
so one option might be a `--patch-module-descriptor` option. This would 
be orthogonal to `--patch-module`, which deliberately has leaky 
classpath-style semantics for its specified content, and that's neither 
desirable nor useful here. Within limits, `--patch module-descriptor` 
would merge the declaration of the "patch" module into the "master" 
module, and fail hard if the limits are broken. What are the limits? A 
starting point is the rules for versioned module descriptors in a 
modular multi-release JAR -- 
http://openjdk.java.net/jeps/238#Modular-multi-release-JAR-files -- and 
perhaps it's reasonable to allow additional _qualified_ exports in the 
"patch" module.

Now, I am not "proposing" `--patch-module-descriptor` for a future JDK. 
I am recognizing that certain use cases involve changing a module's 
dependences and encapsulation in a tightly scoped way. A different way 
to address those use cases would be to ship the logic just described for 
`--patch-module-descriptor` in a launcher offered by the test framework 
itself. If JUnit creates its own module layer, then it can define module 
descriptors at run time how ever it likes. It's not clear to me if 
Christian & co. have looked into java.lang.ModuleLayer, 
java.lang.module.Configuration, and java.lang.module.ModuleFinder. If 
they have, we'd love to hear their experiences. If they haven't, they 
should.

Alex

From sormuras at gmail.com  Fri Feb 14 18:25:06 2020
From: sormuras at gmail.com (Christian Stein)
Date: Fri, 14 Feb 2020 19:25:06 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte,
 from jdk-dev]
In-Reply-To: <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
Message-ID: <CAJi8+g6tC=bz85umgNFat2hUwF0BOgvEYyUnGG7b1wc2gxDDbg@mail.gmail.com>

Hi Alex,

thanks for your reply. Please find answers inlined below.

Note: I just realized that you already replied to Robert, and
from what I saw, the `--patch-module-descriptor` option idea would
indeed be a workable solution. Have to re-read your reply two or
three times to fully grasp that suggestion, though. I'll send this
answer anyway -- to round-up and effectively close this sub-thread.

On Mon, Feb 10, 2020 at 11:03 PM Alex Buckley <alex.buckley at oracle.com>
wrote:

> Hi Christian,
>
> On 2/7/2020 4:41 AM, Christian Stein wrote:
> > This time, I created a project at [0] with a detailed description on its
> > front page, i.e the README.md file.
> >
> > [0]: https://github.com/sormuras/java-module-patching
>
> To restate:
>
> - You're saying that, today, it's brittle to copy directives from
> src/org.astro/main/java/module-info.java to
> src/org.astro/test/java/module-info.java. (And having copied them, you
> still need to `open` the test module and add some `requires`.)
>
> - You suggest that, in future, there will still be a
> src/org.astro/test/java/module-info.java file which is of interest to
> test frameworks.
>
>
That is correct. It is and will be relevant to compile and run
test-related modules. That includes inter-module (black box) test
modules and in(tra)-module (white box) test modules. The former
may use the standard `module-info.java` syntax as of today. As they
use an unique name, no patching is required. The latter, re-use
the name of their associated module on purpose. Those in-module
tests are expected to use the same package name -- allowing access
to package private members.


> What you're hoping is that new syntax will let you invert the patching:
>

No, not at all.

My hope is to re-use the same well-known `module-info.java` syntax
to define the additional needs (modifiers and directives) of the
in-module test module. Similar to the standalone intra-module test
module.



> - Today, you set the module-path so that out/modules/test/org.astro.jar
> is the "primary" version of the module; then you redefine everything in
> it except module-info by overlaying out/modules/main/org.astro.jar.
>


Exactly.


> - In future, you want to have out/modules/main/org.astro.jar as the
> "primary" version, and redefine only its module-info by specifying the
> sidecar out/modules/test/org.astro.jar.

[...]


No. I still want out/modules/test/org.astro.jar to be and stay my
only version of the org.astro module that is relevant at test compile
and runtime time.

I want some help when declaring src/org.astro/test/java/module-info.java
It should "extend" the src/org.astro/main/java/module-info.java
description. At compile and/or runtime.

I don't want to manually merge the orginial modifiers already defined
in src/org.astro/main/java/module-info.java



> Stepping back, the core issue is that once the true "primary" version of
> a module is built -- out/modules/main/org.astro.jar -- you don't want to
> change it.

[...]


True. That's why I prefer to have standalone "test" versions of all modules.
All in the sense of inter-module (black box) and in(-tra)-module (white box)
testing.

I'll cut it here and continue on the  `--patch-module-descriptor` thread
later.

Cheers,
Christian

From sormuras at gmail.com  Fri Feb 14 19:29:34 2020
From: sormuras at gmail.com (Christian Stein)
Date: Fri, 14 Feb 2020 20:29:34 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte,
 from jdk-dev]
In-Reply-To: <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
 <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
 <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
Message-ID: <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>

Hi Alex,

On Fri, Feb 14, 2020 at 7:06 PM Alex Buckley <alex.buckley at oracle.com>
wrote:

> [...]

Now, I am not "proposing" `--patch-module-descriptor` for a future JDK.
> I am recognizing that certain use cases involve changing a module's
> dependences and encapsulation in a tightly scoped way.


I can clearly envision the superpowers of the not (yet) proprosed
`--patch-module-descriptor` option unleashed upon the
Modular Testing World solving the long-standing issue of
resorting to either the class-path or the command-line when
it comes to in-(tra)-module (white box) testing. [0]

Let's explore the feature set of such an option and its limits together.
Looking forward to help with and test-drive that solution.

Three or two years ago, I contributed the "tester plugin" to pro [1].
Pro is a Java build tool that works seamlessly with modules and was
created be R?mi Forax.
That pro tool already performs such a merge action when for in-(tra)-
module (white box) test modules (can you please specify simple
token for such a type of module?!).
Here is the relevant implementation [2] which may guide us
when exploring the feature set. I guess, R?mi will comment on
this topic, too. As he is, if I recall correctly, "the father of the idea".

A different way
> to address those use cases would be to ship the logic just described for
> `--patch-module-descriptor` in a launcher offered by the test framework
> itself.


That's too late in the build game. You already need this merged
descriptor at compile time. And you need the user to define the
additional elements (modifiers and directives), as no build tool
may infer them always correctly by inspecting the test sources...


> If JUnit creates its own module layer, [...]


No, we don't. We "just" make use of the ModuleReader when
scanning a module for potential test classes. That logic [3] is
shipped in a MR-JAR since two years ago and works flawlessly.


> [...] then it can define module descriptors at run time

how ever it likes [...]


Like said before, I think, that's too late in the build game. And
still we had to require the user to pass the extra information
to our framework ... in non-standard manner. Using the manually
merged in-(tra)-module (white box) test modules works for us
since day one of Java 9.


> . It's not clear to me if
> Christian & co. have looked into java.lang.ModuleLayer,
> java.lang.module.Configuration, and java.lang.module.ModuleFinder.

If they have, we'd love to hear their experiences.


Yes, I did: [4]

Works like a charm a test runtime when you want to open a module
to the modules of the build tool. Or here, as you might have guessed,
read the extra information required for testing provided by the user
in a `module-info.test` [5] formatted file. The major drawback is the
simple text format that this `module-info.test` uses: brittle as best.

Summarizing, I like the _not_ proposed `--patch-module-descriptor` option.

Cheers,
Christian

[0]:
https://sormuras.github.io/blog/2018-09-11-testing-in-the-modular-world.html#white-box-modular-testing-with-module-infojava
[1]: https://github.com/forax/pro/tree/master/plugins/tester
[2]:
https://github.com/forax/pro/blob/2ddd9425cb95617b6dfd6c7d077ed387a5f6809c/src/main/java/com.github.forax.pro.helper/com/github/forax/pro/helper/ModuleHelper.java#L338-L378
[3]:
https://github.com/junit-team/junit5/blob/master/junit-platform-commons/src/main/java9/org/junit/platform/commons/util/ModuleUtils.java
[4]:
https://github.com/sormuras/junit-platform-isolator/tree/master/junit-platform-isolator-java-11/src/main/java/de/sormuras/junit/platform/isolator
[5]:
https://sormuras.github.io/blog/2018-09-11-testing-in-the-modular-world.html#white-box-modular-testing-with-extra-java-command-line-options

From alex.buckley at oracle.com  Fri Feb 14 21:29:36 2020
From: alex.buckley at oracle.com (Alex Buckley)
Date: Fri, 14 Feb 2020 13:29:36 -0800
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
 <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
 <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
 <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>
Message-ID: <58fe94c7-53cc-dc71-5c67-0e70aee39d3b@oracle.com>

On 2/14/2020 11:29 AM, Christian Stein wrote:
>> A different way to address those use cases would be to ship the
>> logic just described for `--patch-module-descriptor` in a launcher
>> offered by the test framework itself.
>  
> That's too late in the build game. You already need this merged
> descriptor at compile time. And you need the user to define the
> additional elements (modifiers and directives), as no build tool
> may infer them always correctly by inspecting the test sources...

Fair point, and thanks for reminding me about your "Testing In The 
Modular World" document.

The thing I'm having trouble with is that javac already lets you specify 
the --add-reads options on org.junit.jupiter.api & friends that are 
needed to compile test code. If Maven expects the user to painfully 
configure the POM to pass --patch-module-descriptor to javac, why can't 
the POM offer an easy way to make Maven itself (in the `test-compile` 
phase?) pass some defined-behind-the-scenes --add-reads options?

Similarly, at run time, you propose to always open the main module to 
the same org.junit.platform.commons module -- `module-info.test` is more 
or less the same for every JUnit project -- so why doesn't the POM have 
an easy way to let Maven itself (in the `test` phase?) pass --add-opens 
to java?

There are no new modularity "primitives" here (by which I mean features 
of the module system itself, such as open modules) ... there is just 
detailed configuration which, in the testing context, should be done by 
the build tool rather than the user.

Alex

From sormuras at gmail.com  Sat Feb 15 05:34:36 2020
From: sormuras at gmail.com (Christian Stein)
Date: Sat, 15 Feb 2020 06:34:36 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte,
 from jdk-dev]
In-Reply-To: <58fe94c7-53cc-dc71-5c67-0e70aee39d3b@oracle.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
 <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
 <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
 <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>
 <58fe94c7-53cc-dc71-5c67-0e70aee39d3b@oracle.com>
Message-ID: <CAJi8+g7WY1Ecw1LT6uY2xvJgjJPLAwCJFASXPiMRs9rdGvK4Rw@mail.gmail.com>

On Fri, Feb 14, 2020 at 10:31 PM Alex Buckley <alex.buckley at oracle.com>
wrote:

> On 2/14/2020 11:29 AM, Christian Stein wrote:
> >> A different way to address those use cases would be to ship the
> >> logic just described for `--patch-module-descriptor` in a launcher
> >> offered by the test framework itself.
> >
> > That's too late in the build game. You already need this merged
> > descriptor at compile time. And you need the user to define the
> > additional elements (modifiers and directives), as no build tool
> > may infer them always correctly by inspecting the test sources...
>
> Fair point, and thanks for reminding me about your "Testing In The
> Modular World" document.
>
> The thing I'm having trouble with is that javac already lets you specify
> the --add-reads options on org.junit.jupiter.api & friends that are
> needed to compile test code. If Maven expects the user to painfully
> configure the POM to pass --patch-module-descriptor to javac, why can't
> the POM offer an easy way to make Maven itself (in the `test-compile`
> phase?) pass some defined-behind-the-scenes --add-reads options?
>
>
Maven and other build tools would add the `--patch-module-descriptor`
automacially and point to the user-defined "patch module compilation unit"
that only contains the extra directives: like `requires java.sql;`, or
`requires org.testng.core;` or ... `requires org.junit.jupiter;`. Or all of
them.

Those extra directives needed by test modules to compile and run
are a deliberate decision made by the user -- on a per test module basis.
No build tool may infer those decisions correctly.

Similarly, at run time, you propose to always open the main module to
> the same org.junit.platform.commons module -- `module-info.test` is more
>

Not the main module. Only the current test module. Assuming, that you
run tests from a single test module at a time.


> or less the same for every JUnit project -- so why doesn't the POM have
> an easy way to let Maven itself (in the `test` phase?) pass --add-opens
> to java?
>

Opening a module for deep-reflection is just a single aspect of the
"Testing In The Modular World" story. Btw. `open module` or `--add-opens`
is only required iff the test framework of choice reflects into your test
modules. I already have a ServiceProvider-based variation running locally,
that uses this standard technique to pass the execution control to the
main-like entry-point of each test module.


> There are no new modularity "primitives" here (by which I mean features
> of the module system itself, such as open modules) ... there is just
> detailed configuration which, in the testing context, should be done by
> the build tool rather than the user.
>

Except for configuration parts that cannot be inferred. Here the user
has to resort to javac and java command line options, like `--add-reads`,
and must negotiate with the build tool how to pass those options.

And all of this tedious and errorprone textual configuration **after**
having declared a concise, tool-agnostic, well-definend and beautiful
inter-module test module using the official `module-info.java` syntax:

- test.base/test/module-info.java:

open /*test*/ module test.base {
  requires com.greetings; // module under test
  requires org.astro; // module under test
  requires java.sql; // system-provided module needed by test code
  requires org.junit.jupiter; // framework reflecting entry-points
  requires org.assertj.core; // ...and more test-related modules
}

Leaving this compile-able DSL behind **only** for in(tra)-module
(white-box) testing is ... inconvenient and hard to explain. Like
Robert wrote in his initial request to simplify the usage of patch
module.

Assuming `--patch-module-descriptor` was already available, the
org.astro test story would read:

- org.astro/main/module-info.java:

module org.astro {
  exports org.astro;
  requires java.logging;
}

- org.astro/test/module-info.java:

open /*test*/ module org.astro /*extends main module org.astro*/ {
  requires java.sql; // system-provided module needed by test code
  requires org.junit.jupiter; // framework reflecting entry-points
  requires org.assertj.core; // ...and more test-related modules
}

A build tool now easily may detect that `--patch-module-descriptor`
is required at test compile and run time. It could infer it from the
same module name or simply by seeing a main and a test
`module-info.java` compilation unit within the sources of the
org.astro Maven-/Gradle-/pro-/Bach-/any-build-tool-"module".

Build tools could pass the following argument to javac/java
while being in their test-scope tasks. Of course, implying that
org.astro/test/module-info.java is the current "primary" module
descriptor:

--patch-module-descriptor org.astro=org.astro/main/module-info.java

That's it.

The primary test-relevant modular directives are enriched by
those of the main module as declared in org.astro/main/module-info.java
resulting effectively in a synthetic in-module test module descriptor that
resembles the inter-module test descriptor presented above:

- SYNTHETIC test module for org.astro

open /*test*/ module org.astro /*extends main module org.astro*/ {
  requires java.sql; // system-provided module needed by test code
  requires org.junit.jupiter; // framework reflecting entry-points
  requires org.assertj.core; // ...and more test-related modules
  //
  exports org.astro; // patched from org.astro/main/module-info.java
  requires java.logging; // patched from org.astro/main/module-info.java
}

Cheers,
Christian

From Alan.Bateman at oracle.com  Sat Feb 15 09:35:32 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Sat, 15 Feb 2020 09:35:32 +0000
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
 <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
Message-ID: <511f3be8-611e-f2de-ad1c-8905f5403a7c@oracle.com>

On 14/02/2020 08:02, Robert Scholte wrote:
> :
>
> With a tool like Maven it is already possible to compile and run patched modules, but in case of requiring java.* modules it is very awkward that it requires additional configuration in the pom instead of some dedicated module descriptor, which feels most natural.
> You'll get an exception like:?package java.sql is declared in module java.sql, but module some.module.name does not read it
> And the first thing that comes to mind is: add java.sql to the (main) module descriptor. But this should not be done.
>
Can you summarize how this works when compiling tests that reference 
types in modules on the application module path? Is the Maven compiler 
plugin uses `--add-modules ALL-MODULE-PATH` to include they are all 
resolved? If so, I'm just wondering about readability, do you also 
generate a list of --add-reads options too? I guess I'm just wondering 
why the issue is specific to modules that in the run-time image that 
aren't transitively required by the (unpatched) module.

-Alan

From sormuras at gmail.com  Mon Feb 17 08:52:15 2020
From: sormuras at gmail.com (Christian Stein)
Date: Mon, 17 Feb 2020 09:52:15 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte,
 from jdk-dev]
In-Reply-To: <CAJi8+g7WY1Ecw1LT6uY2xvJgjJPLAwCJFASXPiMRs9rdGvK4Rw@mail.gmail.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
 <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
 <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
 <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>
 <58fe94c7-53cc-dc71-5c67-0e70aee39d3b@oracle.com>
 <CAJi8+g7WY1Ecw1LT6uY2xvJgjJPLAwCJFASXPiMRs9rdGvK4Rw@mail.gmail.com>
Message-ID: <CAJi8+g6oEMXgjpqZS_1zSR1-aVshuZsGe4kJHt_mCCpO0CU_UQ@mail.gmail.com>

I extended the example project with some potential use-cases
for the `--patch-module-descriptor` option at [0]. Those use-cases
show which command line option usages can be replaced and
therefore simplified (as this RFE thread was coined) in the form
of the following example:

Today:

  --add-modules java.sql
  --add-reads org.astro=java.sql

With  `--patch-module-descriptor src/org.astro/test/module-info.java`:

  module org.astro { // name specifies the "primary" module to be patched
      requires java.sql; // adds and reads the specified module
  }

Cheers,
Christian

[0]: https://github.com/sormuras/java-module-patching#examples

>

From alex.buckley at oracle.com  Tue Feb 18 17:42:10 2020
From: alex.buckley at oracle.com (Alex Buckley)
Date: Tue, 18 Feb 2020 09:42:10 -0800
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <CAJi8+g7WY1Ecw1LT6uY2xvJgjJPLAwCJFASXPiMRs9rdGvK4Rw@mail.gmail.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
 <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
 <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
 <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>
 <58fe94c7-53cc-dc71-5c67-0e70aee39d3b@oracle.com>
 <CAJi8+g7WY1Ecw1LT6uY2xvJgjJPLAwCJFASXPiMRs9rdGvK4Rw@mail.gmail.com>
Message-ID: <af60e919-79dd-994e-e1e9-644a914d55af@oracle.com>

On 2/14/2020 9:34 PM, Christian Stein wrote:
> Assuming `--patch-module-descriptor` was already available, the
> org.astro test story would read:
> 
> - org.astro/main/module-info.java:
> 
> module org.astro {
>    exports org.astro;
>    requires java.logging;
> }
> 
> - org.astro/test/module-info.java:
> 
> open /*test*/ module org.astro /*extends main module org.astro*/ {
>    requires java.sql; // system-provided module needed by test code
>    requires org.junit.jupiter; // framework reflecting entry-points
>    requires org.assertj.core; // ...and more test-related modules
> }
> 
> A build tool now easily may detect that `--patch-module-descriptor`
> is required at test compile and run time. It could infer it from the
> same module name or simply by seeing a main and a test
> `module-info.java` compilation unit within the sources of the
> org.astro Maven-/Gradle-/pro-/Bach-/any-build-tool-"module".
> 
> Build tools could pass the following argument to javac/java
> while being in their test-scope tasks. Of course, implying that
> org.astro/test/module-info.java is the current "primary" module
> descriptor:
> 
> --patch-module-descriptor org.astro=org.astro/main/module-info.java
> 
> That's it.
> 
> The primary test-relevant modular directives are enriched by
> those of the main module as declared in org.astro/main/module-info.java
> resulting effectively in a synthetic in-module test module descriptor that
> resembles the inter-module test descriptor presented above:
> 
> - SYNTHETIC test module for org.astro
> 
> open /*test*/ module org.astro /*extends main module org.astro*/ {
>    requires java.sql; // system-provided module needed by test code
>    requires org.junit.jupiter; // framework reflecting entry-points
>    requires org.assertj.core; // ...and more test-related modules
>    //
>    exports org.astro; // patched from org.astro/main/module-info.java
>    requires java.logging; // patched from org.astro/main/module-info.java
> }

Good description of how the main module 
(org.astro/main/module-info.java) and the test module 
(org.astro/test/module-info.java) interact to form a test-scoped module. 
If a build tool has detected the presence of the test module, then the 
build tool can use javax.lang.model.element.ModuleElement to inspect the 
test module's declaration and infer what it requires, whether it's open, 
etc. Then, the build tool can enrich the main module by passing 
--add-modules, --add-reads, and --add-opens to javac/java while in a 
test scope. No need for --patch-module-descriptor in the JDK.

Alex

From sormuras at gmail.com  Wed Feb 19 11:46:38 2020
From: sormuras at gmail.com (Christian Stein)
Date: Wed, 19 Feb 2020 12:46:38 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte,
 from jdk-dev]
In-Reply-To: <af60e919-79dd-994e-e1e9-644a914d55af@oracle.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
 <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
 <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
 <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>
 <58fe94c7-53cc-dc71-5c67-0e70aee39d3b@oracle.com>
 <CAJi8+g7WY1Ecw1LT6uY2xvJgjJPLAwCJFASXPiMRs9rdGvK4Rw@mail.gmail.com>
 <af60e919-79dd-994e-e1e9-644a914d55af@oracle.com>
Message-ID: <CAJi8+g6jtawUr1+J3U64qNbXUH+tY_nV+4QVdLnfwAXD0GJCCQ@mail.gmail.com>

On Tue, Feb 18, 2020 at 6:42 PM Alex Buckley <alex.buckley at oracle.com>
wrote:

>
> Good description of how the main module
> [...]
> etc. Then, the build tool can enrich the main module by passing
> --add-modules, --add-reads, and --add-opens to javac/java while in a
> test scope. No need for --patch-module-descriptor in the JDK.
>
>
But, as Robert already noted, each build tool will come up with
a different notation for their users to configure those additional
elements. That's nothing wrong with it per se -- just that there's
already a common notation for providing modular information:
"module-info.java"

And it's hard to explain why users may write "module-info.java"
files for inter-module testing that build pass 1:1 to javac and
friends -- have to either, depending on the tool, supply:

  - command-line flags... [Maven]
  - or a module-info.test file... [Gradle via Plugin]
  - or module-info.java file... [Pro, Bach.java]
  - or some-other-module-layer-magic configuration...

only for in-module testing.

javac and friends are the "best" choices to combine and
unify those superpowers. Only the user-facing notation
needs some love.

Cheers,
Christian

From alex.buckley at oracle.com  Wed Feb 19 17:02:46 2020
From: alex.buckley at oracle.com (Alex Buckley)
Date: Wed, 19 Feb 2020 09:02:46 -0800
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <CAJi8+g6jtawUr1+J3U64qNbXUH+tY_nV+4QVdLnfwAXD0GJCCQ@mail.gmail.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
 <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
 <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
 <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>
 <58fe94c7-53cc-dc71-5c67-0e70aee39d3b@oracle.com>
 <CAJi8+g7WY1Ecw1LT6uY2xvJgjJPLAwCJFASXPiMRs9rdGvK4Rw@mail.gmail.com>
 <af60e919-79dd-994e-e1e9-644a914d55af@oracle.com>
 <CAJi8+g6jtawUr1+J3U64qNbXUH+tY_nV+4QVdLnfwAXD0GJCCQ@mail.gmail.com>
Message-ID: <9c009566-ec99-c7a4-aae2-7551d70b5c59@oracle.com>

On 2/19/2020 3:46 AM, Christian Stein wrote:
> On Tue, Feb 18, 2020 at 6:42 PM Alex Buckley <alex.buckley at oracle.com>
> wrote:
>>
>> Good description of how the main module
>> [...]
>> etc. Then, the build tool can enrich the main module by passing
>> --add-modules, --add-reads, and --add-opens to javac/java while in a
>> test scope. No need for --patch-module-descriptor in the JDK.
>>
> But, as Robert already noted, each build tool will come up with
> a different notation for their users to configure those additional
> elements. That's nothing wrong with it per se -- just that there's
> already a common notation for providing modular information:
> "module-info.java"
> 
> And it's hard to explain why users may write "module-info.java"
> files for inter-module testing that build pass 1:1 to javac and
> friends -- have to either, depending on the tool, supply:
> 
>    - command-line flags... [Maven]
>    - or a module-info.test file... [Gradle via Plugin]
>    - or module-info.java file... [Pro, Bach.java]
>    - or some-other-module-layer-magic configuration...
> 
> only for in-module testing.

I don't know how you concluded any of the above. I explicitly said (and 
you removed from the quote) that: "If a build tool has detected the 
presence of the test module, then the build tool can use 
javax.lang.model.element.ModuleElement to inspect the test module's 
declaration and infer what it requires, whether it's open, etc." -- 
"test module's declaration" means "org.astro/test/module-info.java".

That is, if a test framework intends that a test module has a 
module-info.java file to augment the main module's module-info.java file 
(and maybe not all test frameworks want to work like that...), then a 
build tool plugin for that test framework is well able to inspect the 
former file and turn it into command line options for javac/java in the 
test scope. The JDK isn't in the business of standardizing test 
framework practices, and already provides the APIs you need to implement 
any practice you like.

Alex

From forax at univ-mlv.fr  Tue Feb 25 16:30:51 2020
From: forax at univ-mlv.fr (Remi Forax)
Date: Tue, 25 Feb 2020 17:30:51 +0100 (CET)
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <9c009566-ec99-c7a4-aae2-7551d70b5c59@oracle.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
 <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>
 <58fe94c7-53cc-dc71-5c67-0e70aee39d3b@oracle.com>
 <CAJi8+g7WY1Ecw1LT6uY2xvJgjJPLAwCJFASXPiMRs9rdGvK4Rw@mail.gmail.com>
 <af60e919-79dd-994e-e1e9-644a914d55af@oracle.com>
 <CAJi8+g6jtawUr1+J3U64qNbXUH+tY_nV+4QVdLnfwAXD0GJCCQ@mail.gmail.com>
 <9c009566-ec99-c7a4-aae2-7551d70b5c59@oracle.com>
Message-ID: <77299880.918903.1582648251554.JavaMail.zimbra@u-pem.fr>

----- Mail original -----
> De: "Alex Buckley" <alex.buckley at oracle.com>
> ?: "jigsaw-dev" <jigsaw-dev at openjdk.java.net>
> Envoy?: Mardi 18 F?vrier 2020 18:42:10
> Objet: Re: RFE simplify usage of patched module [by Robert Scholte, from jdk-dev]

> On 2/14/2020 9:34 PM, Christian Stein wrote:
>> Assuming `--patch-module-descriptor` was already available, the
>> org.astro test story would read:
>>
>> - org.astro/main/module-info.java:
>>
>> module org.astro {
>>    exports org.astro;
>>    requires java.logging;
>> }
>>
>> - org.astro/test/module-info.java:
>>
>> open /*test*/ module org.astro /*extends main module org.astro*/ {
>>    requires java.sql; // system-provided module needed by test code
>>    requires org.junit.jupiter; // framework reflecting entry-points
>>    requires org.assertj.core; // ...and more test-related modules
>> }
>>
>> A build tool now easily may detect that `--patch-module-descriptor`
>> is required at test compile and run time. It could infer it from the
>> same module name or simply by seeing a main and a test
>> `module-info.java` compilation unit within the sources of the
>> org.astro Maven-/Gradle-/pro-/Bach-/any-build-tool-"module".
>>
>> Build tools could pass the following argument to javac/java
>> while being in their test-scope tasks. Of course, implying that
>> org.astro/test/module-info.java is the current "primary" module
>> descriptor:
>>
>> --patch-module-descriptor org.astro=org.astro/main/module-info.java
>>
>> That's it.
>>
>> The primary test-relevant modular directives are enriched by
>> those of the main module as declared in org.astro/main/module-info.java
>> resulting effectively in a synthetic in-module test module descriptor that
>> resembles the inter-module test descriptor presented above:
>>
>> - SYNTHETIC test module for org.astro
>>
>> open /*test*/ module org.astro /*extends main module org.astro*/ {
>>    requires java.sql; // system-provided module needed by test code
>>    requires org.junit.jupiter; // framework reflecting entry-points
>>    requires org.assertj.core; // ...and more test-related modules
>>    //
>>    exports org.astro; // patched from org.astro/main/module-info.java
>>    requires java.logging; // patched from org.astro/main/module-info.java
>> }
>
> Good description of how the main module
> (org.astro/main/module-info.java) and the test module
> (org.astro/test/module-info.java) interact to form a test-scoped module.
> If a build tool has detected the presence of the test module, then the
> build tool can use javax.lang.model.element.ModuleElement to inspect the
> test module's declaration and infer what it requires, whether it's open,
> etc. Then, the build tool can enrich the main module by passing
> --add-modules, --add-reads, and --add-opens to javac/java while in a
> test scope.

Hi Alex,
no you can't because you can not reconstruct a module-info from the flags --add-modules, --add-reads, etc.

Those flags has been created to patch (mostly the JDK) module metadata so an application using the classpath can open/patch the JDK modules metadata enough to be able to run.
The semantics of a module-info is richer than the semantics of the flags, by example, there is no way to define a transitive dependency or to define a service.
I believe that roughly half of the semantics of a module-info has no equivalent in term of flags (and for good reason, those flags have not been created to describe a module-info).

> No need for --patch-module-descriptor in the JDK.

We need a way to express that the module graph when testing is different from the one when running in production.
--path-module-descriptor is a way to get that.

>
> Alex

R?mi

From alex.buckley at oracle.com  Tue Feb 25 17:11:07 2020
From: alex.buckley at oracle.com (Alex Buckley)
Date: Tue, 25 Feb 2020 09:11:07 -0800
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <512011703.883447.1582647385713.JavaMail.zimbra@u-pem.fr>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
 <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
 <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>
 <58fe94c7-53cc-dc71-5c67-0e70aee39d3b@oracle.com>
 <CAJi8+g7WY1Ecw1LT6uY2xvJgjJPLAwCJFASXPiMRs9rdGvK4Rw@mail.gmail.com>
 <af60e919-79dd-994e-e1e9-644a914d55af@oracle.com>
 <512011703.883447.1582647385713.JavaMail.zimbra@u-pem.fr>
Message-ID: <dc467b01-c936-44d1-fa64-026044d9a11c@oracle.com>

On 2/25/2020 8:16 AM, Remi Forax wrote:
> no you can't because you can not reconstruct a module-info from the
> flags --add-modules, --add-reads, etc.
> 
> Those flags has been created to patch (mostly the JDK) module
> metadata so an application using the classpath can open/patch the JDK
> modules metadata enough to be able to run. The semantics of a
> module-info is richer than the semantics of the flags, by example,
> there is no way to define a transitive dependency or to define a
> service. I believe that roughly half of the semantics of a
> module-info has no equivalent in term of flags (and for good reason,
> those flags have not been created to describe a module-info).

In the context of JUnit, the module-info.java file of a test module does 
not define a transitive dependency or define a service. It opens a 
module and adds some direct dependencies -- clauses that are expressible 
with the current command line options. Other test frameworks may want to 
arrange things that are not expressible with the current command line 
options today, but there is no guarantee that --patch-module-descriptor 
would be able to handle those things either, e.g., if the test module 
wishes to remove dependencies or exports of the main module rather than 
always add them. But for what JUnit needs, a build tool plugin could be 
created today.

>> No need for --patch-module-descriptor in the JDK.
> 
> We need a way to express that the module graph when testing is
> different from the one when running in production. 
> --patch-module-descriptor is a way to get that.

And there are other ways, where the build tools (or their plugins) take 
responsibility for arranging the test-time module graph. This may 
require more work on the part of build tool/plugin maintainers than 
simply telling their users to write JDK command line options in the 
middle of a config file.

Alex

From sormuras at gmail.com  Tue Feb 25 17:13:01 2020
From: sormuras at gmail.com (Christian Stein)
Date: Tue, 25 Feb 2020 18:13:01 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte,
 from jdk-dev]
In-Reply-To: <511f3be8-611e-f2de-ad1c-8905f5403a7c@oracle.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <Mailbird-cbd303af-e85d-4161-a0e4-491b857f001a@apache.org>
 <CAFWmRJ083coQgEKB9pSdUQEp38LJ4yJYJN_nLyEq6iL0bY_xNw@mail.gmail.com>
 <Mailbird-7dad4f22-3e31-4959-9ef8-7ce6495a0ee2@apache.org>
 <CAJi8+g4J2QAYb+sJeiPY=ns33W8YQbnCyfnJBt=kmUSk-0k2TQ@mail.gmail.com>
 <CAJi8+g5y9_OvXEQP487RGijGDeQaX-tRZucHRrYoPnsuSBOytg@mail.gmail.com>
 <bc73e505-0297-6fd2-f1d1-61d94fd54a88@oracle.com>
 <Mailbird-e6d233db-9c57-4984-be82-b3d423625faf@apache.org>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
 <511f3be8-611e-f2de-ad1c-8905f5403a7c@oracle.com>
Message-ID: <CAJi8+g4WugUKMRJWCz1yGYEXzBFcRwKK6m=awxA=iGuRHfbMxA@mail.gmail.com>

Hi Alan,

the issue with modules that are provided by the run-time
image and only required at test compile- and run-time is
that with Maven you need to have the GAV (Group Id,
Artifact Id, and Version) of a dependence in order to
declare a read-edge to it. Module "java.sql" for example
can't be expressed as a GAV in the pom.xml file. Here,
the user has to resort using explicit command line
options offered by javac and java to satisfy the needs
of the "maven-compiler-plugin" and also the default
test launcher, namely "maven-surefire-plugin".

In contrast to that, all modules on the application
module path can be patched by Maven already by
using the local or external GAV coordinates.

Cheers,
Christian

On Sat, Feb 15, 2020 at 10:35 AM Alan Bateman <Alan.Bateman at oracle.com>
wrote:

> On 14/02/2020 08:02, Robert Scholte wrote:
> > :
> >
> > With a tool like Maven it is already possible to compile and run patched
> modules, but in case of requiring java.* modules it is very awkward that it
> requires additional configuration in the pom instead of some dedicated
> module descriptor, which feels most natural.
> > You'll get an exception like: package java.sql is declared in module
> java.sql, but module some.module.name does not read it
> > And the first thing that comes to mind is: add java.sql to the (main)
> module descriptor. But this should not be done.
> >
> Can you summarize how this works when compiling tests that reference
> types in modules on the application module path? Is the Maven compiler
> plugin uses `--add-modules ALL-MODULE-PATH` to include they are all
> resolved? If so, I'm just wondering about readability, do you also
> generate a list of --add-reads options too? I guess I'm just wondering
> why the issue is specific to modules that in the run-time image that
> aren't transitively required by the (unpatched) module.
>
-Alan
>

From alex.buckley at oracle.com  Tue Feb 25 17:24:20 2020
From: alex.buckley at oracle.com (Alex Buckley)
Date: Tue, 25 Feb 2020 09:24:20 -0800
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <dc467b01-c936-44d1-fa64-026044d9a11c@oracle.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
 <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
 <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>
 <58fe94c7-53cc-dc71-5c67-0e70aee39d3b@oracle.com>
 <CAJi8+g7WY1Ecw1LT6uY2xvJgjJPLAwCJFASXPiMRs9rdGvK4Rw@mail.gmail.com>
 <af60e919-79dd-994e-e1e9-644a914d55af@oracle.com>
 <512011703.883447.1582647385713.JavaMail.zimbra@u-pem.fr>
 <dc467b01-c936-44d1-fa64-026044d9a11c@oracle.com>
Message-ID: <bd75f5ff-d986-9389-f434-9faee37b6d30@oracle.com>

On 2/25/2020 9:11 AM, Alex Buckley wrote:
> And there are other ways, where the build tools (or their plugins) take 
> responsibility for arranging the test-time module graph. This may 
> require more work on the part of build tool/plugin maintainers than 
> simply telling their users to write JDK command line options in the 
> middle of a config file.

Roy knows what I mean: 
https://twitter.com/royvanrijn/status/1232309684788432896

From rfscholte at apache.org  Tue Feb 25 18:48:22 2020
From: rfscholte at apache.org (Robert Scholte)
Date: Tue, 25 Feb 2020 19:48:22 +0100
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <bd75f5ff-d986-9389-f434-9faee37b6d30@oracle.com>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
 <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
 <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>
 <58fe94c7-53cc-dc71-5c67-0e70aee39d3b@oracle.com>
 <CAJi8+g7WY1Ecw1LT6uY2xvJgjJPLAwCJFASXPiMRs9rdGvK4Rw@mail.gmail.com>
 <af60e919-79dd-994e-e1e9-644a914d55af@oracle.com>
 <512011703.883447.1582647385713.JavaMail.zimbra@u-pem.fr>
 <dc467b01-c936-44d1-fa64-026044d9a11c@oracle.com>
 <bd75f5ff-d986-9389-f434-9faee37b6d30@oracle.com>
Message-ID: <Mailbird-04c8e673-b53c-4c9f-a00e-7f9e41ff96ad@apache.org>

Since my request is only about required modules, I'll solely focus on that.

The responsibility of a build tool is to provide the right set of modules as described in the module descriptor.
But this module descriptor doesn't specify where these modules are coming from, so the build tool should provide some metadata file to define that.

There's no discussion that such module descriptor is useful for the main sources.

For the test sources it is unclear.
One reason to not provide a module descriptor is because it will never be a requirement for other modules.
And what about its name, as it is actually a required module and at the same also the same module.

One reason to provide a module descriptor is for the build tool to know which modules are required, so it can provide the right set of modules.
But how would such descriptor for test sources look like? Well, in the end it must contain everything from the main module descriptor + the test requirements.
I assume that copy/pasting the main module descriptor for a new test module descriptor + adding the extra requirements is not the preferred solution.

Hence in case of compiling test sources Maven does it the other way around. It's metadata file already contains the modules required for testing, aka test scoped dependencies.
And with this Maven is capable to construct the right set of arguments to compile the test sources, giving proper --add-read arguments, etc.
But there's one small set of modules that is not defined in this metadata file: the java.* and jdk.* modules. (this in my only issue!)

In case of Maven, even if we could find a way to define these modules as dependencies, this won't simplify the usage of the modular system.?
Why does one need to define if for test sources and not main sources? For consistency we could ask users to add them for both main and test sources.
Any solution in the pom file is a workaround: in the end they are just a set of sources files that depend on modules and for that reason deserves a module descriptor.
But due to its special purpose it should somehow be based on the main module descriptor.

Without it people will specify these test required?java.*/jdk.*?modules in the only place they know to specify any required modules: the main module descriptor.
Having a mixture here of required modules for both main and test to avoid compilation errors is something we should refuse.

regards,
Robert
On 25-2-2020 18:24:51, Alex Buckley <alex.buckley at oracle.com> wrote:
On 2/25/2020 9:11 AM, Alex Buckley wrote:
> And there are other ways, where the build tools (or their plugins) take
> responsibility for arranging the test-time module graph. This may
> require more work on the part of build tool/plugin maintainers than
> simply telling their users to write JDK command line options in the
> middle of a config file.

Roy knows what I mean:
https://twitter.com/royvanrijn/status/1232309684788432896

From alex.buckley at oracle.com  Tue Feb 25 19:28:26 2020
From: alex.buckley at oracle.com (Alex Buckley)
Date: Tue, 25 Feb 2020 11:28:26 -0800
Subject: RFE simplify usage of patched module [by Robert Scholte, from
 jdk-dev]
In-Reply-To: <Mailbird-04c8e673-b53c-4c9f-a00e-7f9e41ff96ad@apache.org>
References: <CAFWmRJ0JPLmW8JiCBuzcT39kvXqYR3ZQi6Tb+vuWDyAsfPbYuA@mail.gmail.com>
 <3b2b2295-0035-f011-87db-a55ae3f1115b@oracle.com>
 <Mailbird-69e06b09-cc21-40d2-a698-a0bfe57e4382@apache.org>
 <dc61bc48-1389-752d-ff40-444c805e4b37@oracle.com>
 <CAJi8+g6dF63oTKE9JbVkPTLCt95fC6She-f8O1L3=mBQogmA6g@mail.gmail.com>
 <58fe94c7-53cc-dc71-5c67-0e70aee39d3b@oracle.com>
 <CAJi8+g7WY1Ecw1LT6uY2xvJgjJPLAwCJFASXPiMRs9rdGvK4Rw@mail.gmail.com>
 <af60e919-79dd-994e-e1e9-644a914d55af@oracle.com>
 <512011703.883447.1582647385713.JavaMail.zimbra@u-pem.fr>
 <dc467b01-c936-44d1-fa64-026044d9a11c@oracle.com>
 <bd75f5ff-d986-9389-f434-9faee37b6d30@oracle.com>
 <Mailbird-04c8e673-b53c-4c9f-a00e-7f9e41ff96ad@apache.org>
Message-ID: <d241e675-0f8d-eeef-93c8-32df9c817d72@oracle.com>

On 2/25/2020 10:48 AM, Robert Scholte wrote:
> For the test sources it is unclear.
> One reason to not provide a module descriptor is because it will never 
> be a requirement for other modules.
> And what about its name, as it is actually a required module and at the 
> same also the same module.
> 
> One reason to provide a module descriptor is for the build tool to know 
> which modules are required, so it can provide the right set of modules.
> But how would such descriptor for test sources look like? Well, in the 
> end it must contain everything from the main module descriptor + the 
> test requirements.
> I assume that copy/pasting the main module descriptor for a new test 
> module descriptor + adding the extra requirements is not the preferred 
> solution.

Correct, that is no-one's preferred solution. My preferred solution (and 
I think Christian's) is for the test module's descriptor to stay 
physically independent of the main module's descriptor. Then, the build 
tool (i) FINDS the test module's descriptor and (ii) MERGES it logically 
with the main module's descriptor. Suppose the merged descriptor says 
that the main module requires org.junit -- the build tool turns that 
into an --add-modules command when compiling and running tests. Maven 
could be doing this today, helping users to focus on their white-box 
test modules rather than asking users to type obscure --add-* commands 
in the POM.

Furthermore, from what Christian has posted, the effect of merging is 
always the same anyway -- require some JUnit modules, and open the main 
module. So, in the common case, people don't even need to write a test 
module descriptor; they just need the Maven test scope to inject the 
customary --add-* commands when compiling and running code in the main 
module.

Alex