From ooo_saturn7 at mail.ru  Fri Dec  4 14:20:02 2020
From: ooo_saturn7 at mail.ru (=?UTF-8?B?QWxleCBPcmxvdg==?=)
Date: Fri, 04 Dec 2020 17:20:02 +0300
Subject: =?UTF-8?B?RmluZCBzZXJ2aWNlIGluIGEgbGF5ZXIgd2l0aCBtdWx0aXBsZSBjbGFzc2xv?=
 =?UTF-8?B?YWRlcnM=?=
Message-ID: <1607091602.636681216@f481.i.mail.ru>


Hello all,
?
I have a layer with many class loaders (created using public static ModuleLayer.Controller defineModulesWithManyLoaders?(Configuration cf,? List<ModuleLayer> parentLayers, ClassLoader parentLoader)).
?
Let?s suppose we have there three modules (a,b,c). One of the module provides FooService. As I understand to find AService in this layer we can use class loader of ANY module in this layer (any of a,b,c). For example
?
ServiceLoader.load(FooService.class, layer.modules().stream().findAny().get().getClassLoader());
?
Could anyone say if this statement and this code are correct or not?
?
--
Best regards, Alex Orlov

From ooo_saturn7 at mail.ru  Fri Dec  4 14:25:47 2020
From: ooo_saturn7 at mail.ru (=?UTF-8?B?QWxleCBPcmxvdg==?=)
Date: Fri, 04 Dec 2020 17:25:47 +0300
Subject: =?UTF-8?B?RmluZCBzZXJ2aWNlIGluIGEgbGF5ZXIgd2l0aCBtdWx0aXBsZSBjbGFzc2xv?=
 =?UTF-8?B?YWRlcnNbMl0=?=
Message-ID: <1607091947.915269199@f481.i.mail.ru>


Sorry, there was a typo in the previous message.
?
Correct > As I understand to find * FooService * in this layer we can use class loader of ANY module in this layer (any of a,b,c)...
?
--
Best regards, Alex Orlov

From Alan.Bateman at oracle.com  Fri Dec  4 15:34:36 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 4 Dec 2020 15:34:36 +0000
Subject: Find service in a layer with multiple classloaders[2]
In-Reply-To: <1607091947.915269199@f481.i.mail.ru>
References: <1607091947.915269199@f481.i.mail.ru>
Message-ID: <a6c2af94-9c97-c136-1fef-56794e7da019@oracle.com>

On 04/12/2020 14:25, Alex Orlov wrote:
> Sorry, there was a typo in the previous message.
>   
> Correct > As I understand to find * FooService * in this layer we can use class loader of ANY module in this layer (any of a,b,c)...
>
Yes, you can specify any of the class loaders to load(service, 
ClassLoader) or just specify the module layer to load(ModuleLayer, 
service). The javadoc for both methods has all the details and hopefully 
it is clear.

-Alan

From info at j-kuhn.de  Wed Dec  9 22:45:51 2020
From: info at j-kuhn.de (Johannes Kuhn)
Date: Wed, 9 Dec 2020 23:45:51 +0100
Subject: Module.addOpens should log if the package has been opened for illegal
 access for the caller
Message-ID: <3a0fe5e1-d9a1-27bd-a176-939c11364f37@j-kuhn.de>

Hope I got the right mailing list, otherwise please direct me there.

Module.addOpens currently doesn't produce an illegal access warning if 
the target package has only be opened for illegal access.

Reproducer:

 ??? import java.lang.reflect.Method;
 ??? import java.net.URL;
 ??? import java.net.URLClassLoader;

 ??? public class NoIllegalWarning {
 ??????? public static void main(String[] args) throws Throwable {
 ??????????? Class<?> clazz = URLClassLoader.class;
 ??????????? addOpens(clazz);
 ??????????? // Use
 ??????????? Method m = URLClassLoader.class.getDeclaredMethod("addURL", 
URL.class);
 ??????????? m.setAccessible(true);
 ??????????? // invoke it here.
 ??????????? System.out.println("done");
 ??????? }

 ??????? public static void addOpens(Class<?> target) {
 ??????????? target.getModule().addOpens(target.getPackageName(), 
NoIllegalWarning.class.getModule());
 ??????? }
 ??? }

In the current maser branch, it has to be executed with 
--illegal-access=permit.
Other than that, the code above doesn't display any warnings.
Removing the "addOpens" call will display a warning.
No subsequent illegal access warning is produced as the package is 
reflectively open now.

Could someone create a bug on JBS? I could not find an existing one there.
I did prepare a patch to fix this: 
https://github.com/DasBrain/jdk/compare/addOpens-illegalAccess

This is the kind of code I don't like to see in production.

- Johannes


From Alan.Bateman at oracle.com  Thu Dec 10 07:59:53 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 10 Dec 2020 07:59:53 +0000
Subject: Module.addOpens should log if the package has been opened for
 illegal access for the caller
In-Reply-To: <3a0fe5e1-d9a1-27bd-a176-939c11364f37@j-kuhn.de>
References: <3a0fe5e1-d9a1-27bd-a176-939c11364f37@j-kuhn.de>
Message-ID: <25428cf3-c061-e3ce-62ba-c47e679dd94b@oracle.com>

On 09/12/2020 22:45, Johannes Kuhn wrote:
> Hope I got the right mailing list, otherwise please direct me there.
>
> Module.addOpens currently doesn't produce an illegal access warning if 
> the target package has only be opened for illegal access.
>
The Module methods were not intended to emit warnings. The warnings are 
supposed to be emitted by calls to AccessibleObject methods that will 
fail when the default switches to deny illegal reflective access by default.

I think your argument is that if any of the standard or JDK modules 
opens a package for illegal reflective access then code on the class 
path can use Module.addOpens to open the package, which has the 
effective of suppressing warnings. That behavior is there to support the 
--add-opens command line option.

So I think this needs a bit of thinking time to see if it's worth trying 
to do anything here. The default has switched for Java 16 and the 
intention is that the --illegal-access option will be removed by a 
future JEP. I agree it would be sad to see hacks like this in 
production, it would be far better to channel the energy being put into 
hacking around warnings to fixing fragile code to not rely on JDK internals.

-Alan

From info at j-kuhn.de  Thu Dec 10 10:32:27 2020
From: info at j-kuhn.de (Johannes Kuhn)
Date: Thu, 10 Dec 2020 11:32:27 +0100
Subject: Module.addOpens should log if the package has been opened for
 illegal access for the caller
In-Reply-To: <25428cf3-c061-e3ce-62ba-c47e679dd94b@oracle.com>
References: <3a0fe5e1-d9a1-27bd-a176-939c11364f37@j-kuhn.de>
 <25428cf3-c061-e3ce-62ba-c47e679dd94b@oracle.com>
Message-ID: <1d5faefe-3c06-66f2-b1a8-3ba657894fb7@j-kuhn.de>

On 10-Dec-20 8:59, Alan Bateman wrote:
> On 09/12/2020 22:45, Johannes Kuhn wrote:
>> Hope I got the right mailing list, otherwise please direct me there.
>>
>> Module.addOpens currently doesn't produce an illegal access warning 
>> if the target package has only be opened for illegal access.
>>
> The Module methods were not intended to emit warnings. The warnings 
> are supposed to be emitted by calls to AccessibleObject methods that 
> will fail when the default switches to deny illegal reflective access 
> by default.

An argument could be made that MethodHandles.privateLookupIn also emits 
a warning - so an other piece of code that treats "open" vs "open for 
illegal access" differently.
Module.addOpens is a 3rd place where it matters that the package is open 
and that doesn't emit warnings.
The other uses of Module.isOpen are related to resources, which I didn't 
look into yet.

So: Why do all reflective operations (except reflective open) emit a 
warning if the package has only been opened for illegal access?

>
> I think your argument is that if any of the standard or JDK modules 
> opens a package for illegal reflective access then code on the class 
> path can use Module.addOpens to open the package, which has the 
> effective of suppressing warnings. That behavior is there to support 
> the --add-opens command line option.
Using --add-opens to suppress the warning is not really the big point - 
if you don't get any warning, then running with --illegal-access=deny 
should not break the program.

Or, to put in in an other way: With --add-opens you don't rely on 
illegal access anymore.

>
> So I think this needs a bit of thinking time to see if it's worth 
> trying to do anything here. The default has switched for Java 16 and 
> the intention is that the --illegal-access option will be removed by a 
> future JEP. I agree it would be sad to see hacks like this in 
> production, it would be far better to channel the energy being put 
> into hacking around warnings to fixing fragile code to not rely on JDK 
> internals.
>
> -Alan

In short, there are 3 reflective areas where Module.isOpen matters: 
AccessibleObject, MethodHandles.privateLookupIn and Module.addOpens.
Module.addOpens doesn't emit a warning, the others do.

If a module has been reflectively opened, then no warning should 
emitted, right.
But reflectively opening a package should emit such a warning if you can 
only do that because it has been opened to you for illegal access.

As an other note, I think we already did enter warning fatigue - "yeah, 
there will be a warning, doesn't matter", "just accept the cookies"...
So the change to have a default of --illegal-access=deny is a good 
thing. Hopefully the result is not "just run it with 
--illegal-access=permit".

And yeah, I'm a bit late to the party.

- Johannes


From Alan.Bateman at oracle.com  Thu Dec 10 11:45:03 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 10 Dec 2020 11:45:03 +0000
Subject: Module.addOpens should log if the package has been opened for
 illegal access for the caller
In-Reply-To: <1d5faefe-3c06-66f2-b1a8-3ba657894fb7@j-kuhn.de>
References: <3a0fe5e1-d9a1-27bd-a176-939c11364f37@j-kuhn.de>
 <25428cf3-c061-e3ce-62ba-c47e679dd94b@oracle.com>
 <1d5faefe-3c06-66f2-b1a8-3ba657894fb7@j-kuhn.de>
Message-ID: <61a6c546-1107-b23e-5028-6c2e91c7c2e1@oracle.com>

On 10/12/2020 10:32, Johannes Kuhn wrote:
> :
>
> If a module has been reflectively opened, then no warning should 
> emitted, right.
> But reflectively opening a package should emit such a warning if you 
> can only do that because it has been opened to you for illegal access.

If --add-opens is used to open the package then there shouldn't be a 
warning. The reflective case may need more consideration. None of the 
standard or JDK modules should be doing this so there may be a case to 
not suppress the warning after VM initialization has completed. In other 
words, the addOpens in your example would be a no-op because java.net is 
already open for illegal reflection access.

-Alan


From thihup at gmail.com  Sat Dec 12 15:05:09 2020
From: thihup at gmail.com (Thiago Henrique Hupner)
Date: Sat, 12 Dec 2020 12:05:09 -0300
Subject: Cannot use a mapper function to get the Platform classloader
Message-ID: <CAENv9VHbZ-OfAQijCDqSi5tdLki5KwSakgXk4UQYHAsHmMBkaQ@mail.gmail.com>

Hi.

I'm playing around with layers and I needed the following:
Create a new layer where it doesn't have access to the boot layer using the
ModuleLayer.empty().
However, I'm getting the exception that java.base is not found. So I used
something like this:

ModuleLayer.boot().defineModules(conf, moduleName ->
myNewLayer.find(moduleName).isPresent() ? myClassLoader :
ModuleFinder.ofSystem().find(moduleName).isPresent() ?
ClassLoader.getPlatformClassLoader() : null).

Then I get the following exception: java.lang.LayerInstantiationException:
loader can't be 'null' or the platform class loader.

Why is there this restriction? How is the correct way of only enabling the
base modules to be found to the next layer and not the unnamed module.

Thanks!

From Alan.Bateman at oracle.com  Sun Dec 13 16:00:31 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Sun, 13 Dec 2020 16:00:31 +0000
Subject: Cannot use a mapper function to get the Platform classloader
In-Reply-To: <CAENv9VHbZ-OfAQijCDqSi5tdLki5KwSakgXk4UQYHAsHmMBkaQ@mail.gmail.com>
References: <CAENv9VHbZ-OfAQijCDqSi5tdLki5KwSakgXk4UQYHAsHmMBkaQ@mail.gmail.com>
Message-ID: <4fb4fdac-870b-093d-4d53-29e4d7508b7f@oracle.com>

On 12/12/2020 15:05, Thiago Henrique Hupner wrote:
> Hi.
>
> I'm playing around with layers and I needed the following:
> Create a new layer where it doesn't have access to the boot layer using the
> ModuleLayer.empty().
> However, I'm getting the exception that java.base is not found. So I used
> something like this:
>
> ModuleLayer.boot().defineModules(conf, moduleName ->
> myNewLayer.find(moduleName).isPresent() ? myClassLoader :
> ModuleFinder.ofSystem().find(moduleName).isPresent() ?
> ClassLoader.getPlatformClassLoader() : null).
>
> Then I get the following exception: java.lang.LayerInstantiationException:
> loader can't be 'null' or the platform class loader.
>
> Why is there this restriction? How is the correct way of only enabling the
> base modules to be found to the next layer and not the unnamed module.
java.base is in the boot layer and any layer you create will ultimately 
have the boot layer as its parent. If I read your mail correctly then it 
sounds like you want to restrict modules so that the only standard 
module that they read is java.base, is that correct? In that case, the 
simplest is to run with `--limit-modules java.base` so that the only 
module in the boot layer is java.base. Alternatively, maybe you should 
start with a minimum configuration like this:

 ??????? Configuration minConfiguration = Configuration.empty().
 ??????????????? resolve(ModuleFinder.ofSystem(), ModuleFinder.of(), 
Set.of("java.base"));

and use that as the parent configuration. If this minimum configuration 
works as the parent configuration, meaning resolve or resolveAndBind 
succeeds, then you can be assured that there are aren't any observable 
modules that require other standard modules. A scan of the observable 
modules to examine their dependences will also suffice. Then just 
ModuleLayer.boot().configuration() as the parent when creating the 
configuration for the child parent. Does that help?

-Alan


From bmarwell at apache.org  Mon Dec 14 20:05:11 2020
From: bmarwell at apache.org (Benjamin Marwell)
Date: Mon, 14 Dec 2020 21:05:11 +0100
Subject: -Xdoclint unrecognized in JDK16 due to new module
Message-ID: <CAKZnr+yeGd49WYyT7j-q10jgwdUm02662k2eJwdeQWBy6qCYgQ@mail.gmail.com>

Hello all,

it caught my attention that doclint functionality was moved to the
jdk.javadoc module [1].

Now, when calling ToolProvider.getSystemJavaCompiler().getTask(?), the
presence of the option '-Xdoclint:-missing' and similar options will throw
this exception [2]:
   java.lang.IllegalArgumentException: error: invalid flag:
-Xdoclint:-missing.

This call is being used in Apache Maven, and the Apache Zookeper project
already suffers from this issue and reported it upstream [3].

Adding the modules to the maven script 'mvn' did not help. But when calling
javac as an external tool (i.e. not via ToolProvider in the same process),
the options are being recognized.

I wonder how the doclint options can be made available to
the ToolProvider.getSystemJavaCompiler().getTask(?) call again.
Maven itself (and in this case: the compiler plugin) is, as you know, a
non-modular project.

Best regards,
Ben

[1] https://bugs.openjdk.java.net/browse/JDK-8252712
[2]
https://github.com/bmarwell/maven-compiler-plugin/runs/1546156526?check_suite_focus=true
[3] https://bugs.openjdk.java.net/browse/JDK-8253996

From jonathan.gibbons at oracle.com  Tue Dec 15 01:36:28 2020
From: jonathan.gibbons at oracle.com (Jonathan Gibbons)
Date: Mon, 14 Dec 2020 17:36:28 -0800
Subject: -Xdoclint unrecognized in JDK16 due to new module
In-Reply-To: <CAKZnr+yeGd49WYyT7j-q10jgwdUm02662k2eJwdeQWBy6qCYgQ@mail.gmail.com>
References: <CAKZnr+yeGd49WYyT7j-q10jgwdUm02662k2eJwdeQWBy6qCYgQ@mail.gmail.com>
Message-ID: <5e7ea03c-6563-619c-1c87-28634708bd8e@oracle.com>

Ben,

jdk.compiler "uses" a service that "jdk.javadoc" provides, so as long as 
"jdk.javadoc"
is available in the image bring executed, I would expect jdk.javadoc to 
be linked in.

There is different minor issue that the system could fail in a more 
friendly way
when the jdk.javadoc module is not available.

-- Jon

On 12/14/20 12:05 PM, Benjamin Marwell wrote:
> Hello all,
>
> it caught my attention that doclint functionality was moved to the
> jdk.javadoc module [1].
>
> Now, when calling ToolProvider.getSystemJavaCompiler().getTask(?), the
> presence of the option '-Xdoclint:-missing' and similar options will throw
> this exception [2]:
>     java.lang.IllegalArgumentException: error: invalid flag:
> -Xdoclint:-missing.
>
> This call is being used in Apache Maven, and the Apache Zookeper project
> already suffers from this issue and reported it upstream [3].
>
> Adding the modules to the maven script 'mvn' did not help. But when calling
> javac as an external tool (i.e. not via ToolProvider in the same process),
> the options are being recognized.
>
> I wonder how the doclint options can be made available to
> the ToolProvider.getSystemJavaCompiler().getTask(?) call again.
> Maven itself (and in this case: the compiler plugin) is, as you know, a
> non-modular project.
>
> Best regards,
> Ben
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8252712
> [2]
> https://github.com/bmarwell/maven-compiler-plugin/runs/1546156526?check_suite_focus=true
> [3] https://bugs.openjdk.java.net/browse/JDK-8253996

From Alan.Bateman at oracle.com  Tue Dec 15 08:26:47 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 15 Dec 2020 08:26:47 +0000
Subject: -Xdoclint unrecognized in JDK16 due to new module
In-Reply-To: <CAKZnr+yeGd49WYyT7j-q10jgwdUm02662k2eJwdeQWBy6qCYgQ@mail.gmail.com>
References: <CAKZnr+yeGd49WYyT7j-q10jgwdUm02662k2eJwdeQWBy6qCYgQ@mail.gmail.com>
Message-ID: <4064324e-6078-53e4-e130-9117bc2ef165@oracle.com>

On 14/12/2020 20:05, Benjamin Marwell wrote:
> Hello all,
>
> it caught my attention that doclint functionality was moved to the
> jdk.javadoc module [1].
>
> Now, when calling ToolProvider.getSystemJavaCompiler().getTask(?), the
> presence of the option '-Xdoclint:-missing' and similar options will throw
> this exception [2]:
>     java.lang.IllegalArgumentException: error: invalid flag:
> -Xdoclint:-missing.
>
> This call is being used in Apache Maven, and the Apache Zookeper project
> already suffers from this issue and reported it upstream [3].
>
> Adding the modules to the maven script 'mvn' did not help. But when calling
> javac as an external tool (i.e. not via ToolProvider in the same process),
> the options are being recognized.
>
> I wonder how the doclint options can be made available to
> the ToolProvider.getSystemJavaCompiler().getTask(?) call again.
> Maven itself (and in this case: the compiler plugin) is, as you know, a
> non-modular project.
I think the Plexus Classworlds launcher sets a funky TCCL that impacts 
the ServiceLoader lookup of DocLint providers. Can the ServiceLoader 
usage in DocLint be changed to use the system class loader rather than 
the TCCL to see if that "fixes it"?

-Alan.

From thihup at gmail.com  Tue Dec 15 12:28:19 2020
From: thihup at gmail.com (Thiago Henrique Hupner)
Date: Tue, 15 Dec 2020 09:28:19 -0300
Subject: Cannot use a mapper function to get the Platform classloader
In-Reply-To: <4fb4fdac-870b-093d-4d53-29e4d7508b7f@oracle.com>
References: <CAENv9VHbZ-OfAQijCDqSi5tdLki5KwSakgXk4UQYHAsHmMBkaQ@mail.gmail.com>
 <4fb4fdac-870b-093d-4d53-29e4d7508b7f@oracle.com>
Message-ID: <CAENv9VHKDzH8X7uSFNYmXOrVdKKwU9eXUBs1HyOQb-yuW5Kr_w@mail.gmail.com>

Yes, that was exactly what I need.

Thank you!

Em dom., 13 de dez. de 2020 ?s 13:00, Alan Bateman <Alan.Bateman at oracle.com>
escreveu:

> On 12/12/2020 15:05, Thiago Henrique Hupner wrote:
> > Hi.
> >
> > I'm playing around with layers and I needed the following:
> > Create a new layer where it doesn't have access to the boot layer using
> the
> > ModuleLayer.empty().
> > However, I'm getting the exception that java.base is not found. So I used
> > something like this:
> >
> > ModuleLayer.boot().defineModules(conf, moduleName ->
> > myNewLayer.find(moduleName).isPresent() ? myClassLoader :
> > ModuleFinder.ofSystem().find(moduleName).isPresent() ?
> > ClassLoader.getPlatformClassLoader() : null).
> >
> > Then I get the following exception:
> java.lang.LayerInstantiationException:
> > loader can't be 'null' or the platform class loader.
> >
> > Why is there this restriction? How is the correct way of only enabling
> the
> > base modules to be found to the next layer and not the unnamed module.
> java.base is in the boot layer and any layer you create will ultimately
> have the boot layer as its parent. If I read your mail correctly then it
> sounds like you want to restrict modules so that the only standard
> module that they read is java.base, is that correct? In that case, the
> simplest is to run with `--limit-modules java.base` so that the only
> module in the boot layer is java.base. Alternatively, maybe you should
> start with a minimum configuration like this:
>
>          Configuration minConfiguration = Configuration.empty().
>                  resolve(ModuleFinder.ofSystem(), ModuleFinder.of(),
> Set.of("java.base"));
>
> and use that as the parent configuration. If this minimum configuration
> works as the parent configuration, meaning resolve or resolveAndBind
> succeeds, then you can be assured that there are aren't any observable
> modules that require other standard modules. A scan of the observable
> modules to examine their dependences will also suffice. Then just
> ModuleLayer.boot().configuration() as the parent when creating the
> configuration for the child parent. Does that help?
>
> -Alan
>
>

From jonathan.gibbons at oracle.com  Tue Dec 15 22:19:56 2020
From: jonathan.gibbons at oracle.com (Jonathan Gibbons)
Date: Tue, 15 Dec 2020 14:19:56 -0800
Subject: -Xdoclint unrecognized in JDK16 due to new module
In-Reply-To: <4064324e-6078-53e4-e130-9117bc2ef165@oracle.com>
References: <CAKZnr+yeGd49WYyT7j-q10jgwdUm02662k2eJwdeQWBy6qCYgQ@mail.gmail.com>
 <4064324e-6078-53e4-e130-9117bc2ef165@oracle.com>
Message-ID: <8b1c1b91-c118-638a-c656-0b3b917970d7@oracle.com>


On 12/15/20 12:26 AM, Alan Bateman wrote:
> On 14/12/2020 20:05, Benjamin Marwell wrote:
>> Hello all,
>>
>> it caught my attention that doclint functionality was moved to the
>> jdk.javadoc module [1].
>>
>> Now, when calling ToolProvider.getSystemJavaCompiler().getTask(?), the
>> presence of the option '-Xdoclint:-missing' and similar options will 
>> throw
>> this exception [2]:
>> ??? java.lang.IllegalArgumentException: error: invalid flag:
>> -Xdoclint:-missing.
>>
>> This call is being used in Apache Maven, and the Apache Zookeper project
>> already suffers from this issue and reported it upstream [3].
>>
>> Adding the modules to the maven script 'mvn' did not help. But when 
>> calling
>> javac as an external tool (i.e. not via ToolProvider in the same 
>> process),
>> the options are being recognized.
>>
>> I wonder how the doclint options can be made available to
>> the ToolProvider.getSystemJavaCompiler().getTask(?) call again.
>> Maven itself (and in this case: the compiler plugin) is, as you know, a
>> non-modular project.
> I think the Plexus Classworlds launcher sets a funky TCCL that impacts 
> the ServiceLoader lookup of DocLint providers. Can the ServiceLoader 
> usage in DocLint be changed to use the system class loader rather than 
> the TCCL to see if that "fixes it"?
>
> -Alan.


Filed as https://bugs.openjdk.java.net/browse/JDK-8258443

-- Jon


From Alan.Bateman at oracle.com  Wed Dec 16 07:32:12 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 16 Dec 2020 07:32:12 +0000
Subject: -Xdoclint unrecognized in JDK16 due to new module
In-Reply-To: <8b1c1b91-c118-638a-c656-0b3b917970d7@oracle.com>
References: <CAKZnr+yeGd49WYyT7j-q10jgwdUm02662k2eJwdeQWBy6qCYgQ@mail.gmail.com>
 <4064324e-6078-53e4-e130-9117bc2ef165@oracle.com>
 <8b1c1b91-c118-638a-c656-0b3b917970d7@oracle.com>
Message-ID: <19c9d350-ec9f-ed6e-9f4e-7ccb45ac663c@oracle.com>



On 15/12/2020 22:19, Jonathan Gibbons wrote:
>
> On 12/15/20 12:26 AM, Alan Bateman wrote:
>> I think the Plexus Classworlds launcher sets a funky TCCL that 
>> impacts the ServiceLoader lookup of DocLint providers. Can the 
>> ServiceLoader usage in DocLint be changed to use the system class 
>> loader rather than the TCCL to see if that "fixes it"?
>>
>> -Alan.
>
>
> Filed as https://bugs.openjdk.java.net/browse/JDK-8258443
Okay, but you already have JDK-8253996. In that bug report, the building 
of zookeeper-jute seems to run the compiler plugin with TCCL set to 
something setup by Plexus Classworlds and this is why the DocLint 
provider in jdk.javadoc is not located at run-time.

-Alan

From gunnar at hibernate.org  Fri Dec 25 10:29:58 2020
From: gunnar at hibernate.org (Gunnar Morling)
Date: Fri, 25 Dec 2020 11:29:58 +0100
Subject: Making jlink Plug-in API public
Message-ID: <CADGJaX-+0ra=a6f955BBjZmgzQys6XcqChJF9UeGgnBcQ5aMsw@mail.gmail.com>

Hi all,

Are there any plans for making the jlink Plug-in API public any time soon,
perhaps for JDK 17?

I think the ability to implement custom plug-ins that are run at linking
time would open up quite a few interesting opportunities, e.g.

- removing unused members and obfuscation
- adding resources like annotation indexes for the whole image
- verifying that exported and required API methods match (avoiding
NoSuchMethodErrors due to adding incompatible module versions to an image)

The latter would be interesting in particular for monolithic applications
built jointly by multiple teams, avoiding the need to recompile the entire
application if only a single module changed, while still having fidelity
that APIs would match.

Thanks,

--Gunnar

From gunnar at hibernate.org  Fri Dec 25 10:32:02 2020
From: gunnar at hibernate.org (Gunnar Morling)
Date: Fri, 25 Dec 2020 11:32:02 +0100
Subject: How to extract a class from "modules" archive
Message-ID: <CADGJaX90d1Bv0Y5zdgf2Jh8w41ujgBriMB=1+kvMF5T7G-Y3nQ@mail.gmail.com>

Hi,

Is the format of the "modules" archive specified and documented somewhere?
Or asked differently, if I wanted to extract the class file for a given JDK
class from the "modules" file coming with the JDK, what would be the best
way to do so?

Thanks a lot,

--Gunnar

From jeffmaury at jeffmaury.com  Fri Dec 25 11:54:35 2020
From: jeffmaury at jeffmaury.com (Jeff MAURY)
Date: Fri, 25 Dec 2020 12:54:35 +0100
Subject: How to extract a class from "modules" archive
In-Reply-To: <CADGJaX90d1Bv0Y5zdgf2Jh8w41ujgBriMB=1+kvMF5T7G-Y3nQ@mail.gmail.com>
References: <CADGJaX90d1Bv0Y5zdgf2Jh8w41ujgBriMB=1+kvMF5T7G-Y3nQ@mail.gmail.com>
Message-ID: <CANyrOm4Kks1hFP9oSk-vcAesiuHuOXp3bush=eWj7Wzen8BetQ@mail.gmail.com>

Use ClassLoader::getSystemResource, or the JRT file system

On Fri, Dec 25, 2020 at 11:32 AM Gunnar Morling <gunnar at hibernate.org>
wrote:

> Hi,
>
> Is the format of the "modules" archive specified and documented somewhere?
> Or asked differently, if I wanted to extract the class file for a given JDK
> class from the "modules" file coming with the JDK, what would be the best
> way to do so?
>
> Thanks a lot,
>
> --Gunnar
>


-- 
Jeff MAURY


"Legacy code" often differs from its suggested alternative by actually
working and scaling.
 - Bjarne Stroustrup

http://www.jeffmaury.com
http://riadiscuss.jeffmaury.com
http://www.twitter.com/jeffmaury

From sundararajan.athijegannathan at oracle.com  Fri Dec 25 12:04:40 2020
From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com)
Date: Fri, 25 Dec 2020 17:34:40 +0530
Subject: How to extract a class from "modules" archive
In-Reply-To: <CADGJaX90d1Bv0Y5zdgf2Jh8w41ujgBriMB=1+kvMF5T7G-Y3nQ@mail.gmail.com>
References: <CADGJaX90d1Bv0Y5zdgf2Jh8w41ujgBriMB=1+kvMF5T7G-Y3nQ@mail.gmail.com>
Message-ID: <ebb8c33c-6153-3725-74d0-a261ef5f0e94@oracle.com>

By tool:

jimage extract <modules_file>

By API:

jrt-fs file system. You can find jrt-fs usage from javac code

-Sundar

On 25/12/20 4:02 pm, Gunnar Morling wrote:
> Hi,
>
> Is the format of the "modules" archive specified and documented somewhere?
> Or asked differently, if I wanted to extract the class file for a given JDK
> class from the "modules" file coming with the JDK, what would be the best
> way to do so?
>
> Thanks a lot,
>
> --Gunnar

From gunnar at hibernate.org  Fri Dec 25 16:28:38 2020
From: gunnar at hibernate.org (Gunnar Morling)
Date: Fri, 25 Dec 2020 17:28:38 +0100
Subject: How to extract a class from "modules" archive
In-Reply-To: <ebb8c33c-6153-3725-74d0-a261ef5f0e94@oracle.com>
References: <CADGJaX90d1Bv0Y5zdgf2Jh8w41ujgBriMB=1+kvMF5T7G-Y3nQ@mail.gmail.com>
 <ebb8c33c-6153-3725-74d0-a261ef5f0e94@oracle.com>
Message-ID: <CADGJaX9w7wjf-CrVt-pgks=oXQn04zieP7YdsvWhsMj+XZwoJA@mail.gmail.com>

Thanks, Sundar and Jeff! The tool approach was what I was looking for, so
thx for the pointer to jimage.

--Gunnar


Am Fr., 25. Dez. 2020 um 13:05 Uhr schrieb <
sundararajan.athijegannathan at oracle.com>:

> By tool:
>
> jimage extract <modules_file>
>
> By API:
>
> jrt-fs file system. You can find jrt-fs usage from javac code
>
> -Sundar
>
> On 25/12/20 4:02 pm, Gunnar Morling wrote:
> > Hi,
> >
> > Is the format of the "modules" archive specified and documented
> somewhere?
> > Or asked differently, if I wanted to extract the class file for a given
> JDK
> > class from the "modules" file coming with the JDK, what would be the best
> > way to do so?
> >
> > Thanks a lot,
> >
> > --Gunnar
>

From gunnar at hibernate.org  Sat Dec 26 11:29:23 2020
From: gunnar at hibernate.org (Gunnar Morling)
Date: Sat, 26 Dec 2020 12:29:23 +0100
Subject: Why is it allowed to have several modules with the same name?
Message-ID: <CADGJaX8qd1fMG_=ZcB7CHQ8oFtM=SR_mOzoyRbMbxk_aQidU9A@mail.gmail.com>

Hi,

So far it was my assumption that the module system would reject
multiple modules with the same name to be present.

But this seems not to be the case; if I have two versions of a module JAR
on my module path -- i.e. two modules which have the same module name and
export the same set of packages -- the java command does not complain (nor
does javac or jlink). Is this an expected behavior?

Thanks,

--Gunnar

From jonathan.gibbons at oracle.com  Sat Dec 26 16:26:40 2020
From: jonathan.gibbons at oracle.com (Jonathan Gibbons)
Date: Sat, 26 Dec 2020 08:26:40 -0800
Subject: Why is it allowed to have several modules with the same name?
In-Reply-To: <CADGJaX8qd1fMG_=ZcB7CHQ8oFtM=SR_mOzoyRbMbxk_aQidU9A@mail.gmail.com>
References: <CADGJaX8qd1fMG_=ZcB7CHQ8oFtM=SR_mOzoyRbMbxk_aQidU9A@mail.gmail.com>
Message-ID: <b6a5a517-53ce-dcac-6adf-23d8c8a17349@oracle.com>

If the two modules with the same name are in different positions on the 
module path, the first one will be seen and will completely hide any 
subsequent occurrences of modules with the same name.

If two modules with the same name are found in the same directory on the 
module path, such as in differently-named jar files, then that is an 
error, because neither can be said to have precedence and hide the other.

-- Jon

On 12/26/20 3:29 AM, Gunnar Morling wrote:
> Hi,
>
> So far it was my assumption that the module system would reject
> multiple modules with the same name to be present.
>
> But this seems not to be the case; if I have two versions of a module JAR
> on my module path -- i.e. two modules which have the same module name and
> export the same set of packages -- the java command does not complain (nor
> does javac or jlink). Is this an expected behavior?
>
> Thanks,
>
> --Gunnar

From gunnar at hibernate.org  Sat Dec 26 16:58:04 2020
From: gunnar at hibernate.org (Gunnar Morling)
Date: Sat, 26 Dec 2020 17:58:04 +0100
Subject: Why is it allowed to have several modules with the same name?
In-Reply-To: <b6a5a517-53ce-dcac-6adf-23d8c8a17349@oracle.com>
References: <CADGJaX8qd1fMG_=ZcB7CHQ8oFtM=SR_mOzoyRbMbxk_aQidU9A@mail.gmail.com>
 <b6a5a517-53ce-dcac-6adf-23d8c8a17349@oracle.com>
Message-ID: <CADGJaX_jQUZd+cEKAOPJuTmd+rgw4M=qkrn+hE5-RdrysWjcAw@mail.gmail.com>

Why is this situation not considered an error though? Relying on ordering
on the module path seems to go against one of the core promises of the
module system: reliability and less "surprises".

--Gunnar


Am Sa., 26. Dez. 2020 um 17:27 Uhr schrieb Jonathan Gibbons <
jonathan.gibbons at oracle.com>:

> If the two modules with the same name are in different positions on the
> module path, the first one will be seen and will completely hide any
> subsequent occurrences of modules with the same name.
>
> If two modules with the same name are found in the same directory on the
> module path, such as in differently-named jar files, then that is an
> error, because neither can be said to have precedence and hide the other.
>
> -- Jon
>
> On 12/26/20 3:29 AM, Gunnar Morling wrote:
> > Hi,
> >
> > So far it was my assumption that the module system would reject
> > multiple modules with the same name to be present.
> >
> > But this seems not to be the case; if I have two versions of a module JAR
> > on my module path -- i.e. two modules which have the same module name and
> > export the same set of packages -- the java command does not complain
> (nor
> > does javac or jlink). Is this an expected behavior?
> >
> > Thanks,
> >
> > --Gunnar
>

From Alan.Bateman at oracle.com  Sat Dec 26 18:03:42 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Sat, 26 Dec 2020 18:03:42 +0000
Subject: Why is it allowed to have several modules with the same name?
In-Reply-To: <b6a5a517-53ce-dcac-6adf-23d8c8a17349@oracle.com>
References: <CADGJaX8qd1fMG_=ZcB7CHQ8oFtM=SR_mOzoyRbMbxk_aQidU9A@mail.gmail.com>
 <b6a5a517-53ce-dcac-6adf-23d8c8a17349@oracle.com>
Message-ID: <35a44a51-a56f-e4e6-3183-469d4ebd2e53@oracle.com>

On 26/12/2020 16:26, Jonathan Gibbons wrote:
> If the two modules with the same name are in different positions on 
> the module path, the first one will be seen and will completely hide 
> any subsequent occurrences of modules with the same name.
>
> If two modules with the same name are found in the same directory on 
> the module path, such as in differently-named jar files, then that is 
> an error, because neither can be said to have precedence and hide the 
> other.

Just to add to Jon's mail, there detail in in the "Module paths" section 
of JEP 261 [1] and the API docs of ModuleFinder.of(Path...) [2] which is 
the API for creating the equivalent of a module path to find modules.

-Alan.

[1] https://openjdk.java.net/jeps/261#Module-paths
[2] 
https://docs.oracle.com/en/java/javase/15/docs/api/java.base/java/lang/module/ModuleFinder.html#of(java.nio.file.Path...)

From gunnar at hibernate.org  Sun Dec 27 09:50:50 2020
From: gunnar at hibernate.org (Gunnar Morling)
Date: Sun, 27 Dec 2020 10:50:50 +0100
Subject: Why is it allowed to have several modules with the same name?
In-Reply-To: <35a44a51-a56f-e4e6-3183-469d4ebd2e53@oracle.com>
References: <CADGJaX8qd1fMG_=ZcB7CHQ8oFtM=SR_mOzoyRbMbxk_aQidU9A@mail.gmail.com>
 <b6a5a517-53ce-dcac-6adf-23d8c8a17349@oracle.com>
 <35a44a51-a56f-e4e6-3183-469d4ebd2e53@oracle.com>
Message-ID: <CADGJaX-Ob5dA_UqjQtFrkiNyFy4k4=0fEmHckhqk7Ry-ZXWHjA@mail.gmail.com>

Thanks for clarifying, Alan. So it is intended and well-documented, it's
still not quite clear to me though *why* that is. Wouldn#t it be more in
line with the module system's goal of reliability to reject such
error-prone configuration?

--Gunnar


Am Sa., 26. Dez. 2020 um 19:04 Uhr schrieb Alan Bateman <
Alan.Bateman at oracle.com>:

> On 26/12/2020 16:26, Jonathan Gibbons wrote:
> > If the two modules with the same name are in different positions on
> > the module path, the first one will be seen and will completely hide
> > any subsequent occurrences of modules with the same name.
> >
> > If two modules with the same name are found in the same directory on
> > the module path, such as in differently-named jar files, then that is
> > an error, because neither can be said to have precedence and hide the
> > other.
>
> Just to add to Jon's mail, there detail in in the "Module paths" section
> of JEP 261 [1] and the API docs of ModuleFinder.of(Path...) [2] which is
> the API for creating the equivalent of a module path to find modules.
>
> -Alan.
>
> [1] https://openjdk.java.net/jeps/261#Module-paths
> [2]
>
> https://docs.oracle.com/en/java/javase/15/docs/api/java.base/java/lang/module/ModuleFinder.html#of(java.nio.file.Path..
> .)
>

From Alan.Bateman at oracle.com  Mon Dec 28 09:00:07 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Mon, 28 Dec 2020 09:00:07 +0000
Subject: Making jlink Plug-in API public
In-Reply-To: <CADGJaX-+0ra=a6f955BBjZmgzQys6XcqChJF9UeGgnBcQ5aMsw@mail.gmail.com>
References: <CADGJaX-+0ra=a6f955BBjZmgzQys6XcqChJF9UeGgnBcQ5aMsw@mail.gmail.com>
Message-ID: <63363488-5389-6829-d83d-2abc9c37eb68@oracle.com>

On 25/12/2020 10:29, Gunnar Morling wrote:
> Hi all,
>
> Are there any plans for making the jlink Plug-in API public any time soon,
> perhaps for JDK 17?
>
> I think the ability to implement custom plug-ins that are run at linking
> time would open up quite a few interesting opportunities, e.g.
>
> - removing unused members and obfuscation
> - adding resources like annotation indexes for the whole image
> - verifying that exported and required API methods match (avoiding
> NoSuchMethodErrors due to adding incompatible module versions to an image)
>
> The latter would be interesting in particular for monolithic applications
> built jointly by multiple teams, avoiding the need to recompile the entire
> application if only a single module changed, while still having fidelity
> that APIs would match.
>
The intention in early revisions of JEP 282 was to expose a plugin API. 
It went through many iterations and re-designs during JDK 9 but it 
didn't get to the point where it was really ready to be exposed by an 
incubator module. There was also a complication at the time with 
incubator modules providing implementations of ToolProvider (that issue 
has since been resolved as it became a blocker for jpackage too).

Is it worth giving it a priority boost and re-examine it now? Hard to 
say as it would likely require several iterations and a lot of review 
cycles to bring this to incubation stage. At the same time, our 
experience with plugins to date is that many of the plugins are deeply 
tied to core modules and just aren't maintainable outside of the jdk 
repo. I think it could also be argued that some of the suggestions that 
you list should be be plugins included in the jdk.jlink module. For 
example indexing of annotations was something that was explored and 
prototyped at one point (the original requirements document for the 
module system JSR had an item along these lines). A tool that scans a 
module path to do static analysis and detect incompatibilities could be 
valuable as a link time plugin. Are these plugins that you would be 
interested in developing, contributing, and maintaining?

-Alan.

From Alan.Bateman at oracle.com  Tue Dec 29 07:57:04 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 29 Dec 2020 07:57:04 +0000
Subject: Why is it allowed to have several modules with the same name?
In-Reply-To: <CADGJaX-Ob5dA_UqjQtFrkiNyFy4k4=0fEmHckhqk7Ry-ZXWHjA@mail.gmail.com>
References: <CADGJaX8qd1fMG_=ZcB7CHQ8oFtM=SR_mOzoyRbMbxk_aQidU9A@mail.gmail.com>
 <b6a5a517-53ce-dcac-6adf-23d8c8a17349@oracle.com>
 <35a44a51-a56f-e4e6-3183-469d4ebd2e53@oracle.com>
 <CADGJaX-Ob5dA_UqjQtFrkiNyFy4k4=0fEmHckhqk7Ry-ZXWHjA@mail.gmail.com>
Message-ID: <e165b14f-44ce-2607-24d4-e8d5df50684f@oracle.com>

On 27/12/2020 09:50, Gunnar Morling wrote:
> Thanks for clarifying, Alan. So it is intended and well-documented, 
> it's still not quite clear to me though *why* that is. Wouldn#t it be 
> more in line with the module system's goal of reliability to reject 
> such error-prone configuration?
Reliable configuration is mostly about ensuring that there aren't 
missing dependencies, cycles between dependencies, or split packages. 
You get reliability with the module path by only reading the first 
definition of a module. If someone put 10 versions of the same module on 
the module path then classes will only be loaded from the first module, 
unlike the class path where you risk loading classes from more than one 
of the artifacts.

If it helps, the java --validate-modules option is useful for detecting 
and reporting issues. It will print an informational message when there 
are multiple definitions of a module on the module path.

-Alan

From gunnar at hibernate.org  Tue Dec 29 19:29:47 2020
From: gunnar at hibernate.org (Gunnar Morling)
Date: Tue, 29 Dec 2020 20:29:47 +0100
Subject: Making jlink Plug-in API public
In-Reply-To: <63363488-5389-6829-d83d-2abc9c37eb68@oracle.com>
References: <CADGJaX-+0ra=a6f955BBjZmgzQys6XcqChJF9UeGgnBcQ5aMsw@mail.gmail.com>
 <63363488-5389-6829-d83d-2abc9c37eb68@oracle.com>
Message-ID: <CADGJaX90EEVgHZ3DvsNbv8oGgD3=p2Me2GDNJxnPcYinP6OaRA@mail.gmail.com>

Am Mo., 28. Dez. 2020 um 10:00 Uhr schrieb Alan Bateman <
Alan.Bateman at oracle.com>:

> On 25/12/2020 10:29, Gunnar Morling wrote:
> > Hi all,
> >
> > Are there any plans for making the jlink Plug-in API public any time
> soon,
> > perhaps for JDK 17?
> >
> > I think the ability to implement custom plug-ins that are run at linking
> > time would open up quite a few interesting opportunities, e.g.
> >
> > - removing unused members and obfuscation
> > - adding resources like annotation indexes for the whole image
> > - verifying that exported and required API methods match (avoiding
> > NoSuchMethodErrors due to adding incompatible module versions to an
> image)
> >
> > The latter would be interesting in particular for monolithic applications
> > built jointly by multiple teams, avoiding the need to recompile the
> entire
> > application if only a single module changed, while still having fidelity
> > that APIs would match.
> >
> The intention in early revisions of JEP 282 was to expose a plugin API.
> It went through many iterations and re-designs during JDK 9 but it
> didn't get to the point where it was really ready to be exposed by an
> incubator module. There was also a complication at the time with
> incubator modules providing implementations of ToolProvider (that issue
> has since been resolved as it became a blocker for jpackage too).
>
> Is it worth giving it a priority boost and re-examine it now? Hard to
> say as it would likely require several iterations and a lot of review
> cycles to bring this to incubation stage.


That's interesting; what is missing from your PoV to make the API an
incubating one?


> At the same time, our
> experience with plugins to date is that many of the plugins are deeply
> tied to core modules and just aren't maintainable outside of the jdk
> repo.


That's fair. OTOH, right now it's really cumbersome for external
contributors to explore the area, as the plug-in API is not exposed and
it's not trivial to have jlink pick up custom plug-ins. I'm not aware of
any external plug-in really.


> I think it could also be argued that some of the suggestions that
> you list should be be plugins included in the jdk.jlink module. For
> example indexing of annotations was something that was explored and
> prototyped at one point (the original requirements document for the
> module system JSR had an item along these lines). A tool that scans a
> module path to do static analysis and detect incompatibilities could be
> valuable as a link time plugin. Are these plugins that you would be
> interested in developing, contributing, and maintaining?
>

I could see myself contributing to a plug-in around signature validation,
depending on how involved that task would be. I did a quick PoC and discuss
it here:


https://www.morling.dev/blog/jlinks-missing-link-api-signature-validation/

This is using the existing Animal Sniffer tool for a basic signature check.
I'm not sure how much effort it'd be to fully implement this, and also
catch things like added non-virtual methods in a base class/interface,
changes to type parameters, and more. Could existing code from the JDK be
re-used for that task?

Thanks,

--Gunnar


-Alan.
>