From Alan.Bateman at oracle.com  Fri Sep  1 08:45:18 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 1 Sep 2017 09:45:18 +0100
Subject: Jigsaw and containers
In-Reply-To: <D8EF6296-2C79-4707-94DA-90C59EDAEA66@gmail.com>
References: <5233a7b5-9afc-10ee-4ec6-c87b6390f472@gmail.com>
 <2eca2846-c31d-9c66-760b-243eea444531@oracle.com>
 <f4e84137-da72-3200-0df0-e68eca5f5e12@gmail.com>
 <57723896.1898851.1504206864547.JavaMail.zimbra@u-pem.fr>
 <D8EF6296-2C79-4707-94DA-90C59EDAEA66@gmail.com>
Message-ID: <443b8fae-fc4b-712d-0e82-9045d8b5ac26@oracle.com>

On 31/08/2017 22:05, Daniel Latr?moli?re wrote:
> Hi R?mi, Alan
>
> I use already jlink to keep only useful (for me) parts of JRE/JDK, and avoid most big thinks (like Java UI, CORBA, RMI, etc).
> Modular JAR are officially for migration [1] and are not used by modules from OpenJDK, even when they are pure Java. If they are a definitive target, why isn't it in OpenJDK a tool to transform a JMOD in a modular jar (at least when it is pure Java bytecode)?
> As I understand jlink as an optimizer tool [2], I would try to use it for all modules and expect it to be able to do so. If I can not do it good, because they resulting image format is too much restricted, I will mitigate to have a less bad solution.
>
Many of the core modules in the JDK have native libraries, launchers, 
configuration files, .. and so lend themselves to the JMOD format with 
its defined sections for each type of resource. There are some modules 
in the JDK that only contain class files and a valid alternative would 
be for the OpenJDK build to package these as modular JARs. So don't read 
anything into the JDK using the JMOD format, the bulk of modules in the 
wider world will be packaged as modular JARs.

BTW:? The page you reference about modular JARs being "officially for 
migration" is an obsolete page from the exploratory phase of Project 
Jigsaw. I'm not sure how you got to that but it's a reminder that there 
are old pages that needs to be labeled as obsolete.

As regards overlaying one run-time image over another then this brings 
too many hazards and complications. If you want libraries of modules 
(which I think is what this thread is really about) then put the modular 
JARs in a directory. If you organize these into layers, base image, a 
directory for common libraries, directories for domain specific 
libraries, ... then they should work well when you union the file 
systems. The resolution that is done at startup will ensure that there 
aren't any missing modules and will catch other issues in the 
configuration too.

-Alan

From rahman.usta.88 at gmail.com  Fri Sep  1 20:21:16 2017
From: rahman.usta.88 at gmail.com (Rahman USTA)
Date: Fri, 1 Sep 2017 23:21:16 +0300
Subject: Moving to Java 9 - module-info.class not found for <module-name>
 module
Message-ID: <CAGe7fv36O6JsYERF=KB1sS=71wWvdHutHwhebkfpnhTKGjQpNQ@mail.gmail.com>

Hi,

I migrate TerminalFX <https://github.com/rahmanusta/TerminalFX> project to
Java 9. To run the project on Java 9 I'm following the steps:

mvn clean install dependency:copy-dependencies
java --module-path
%JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
terminalfx -m terminalfx/com.terminalfx.AppStarter

It works normally. Then, I want to generate a jlink image with the
following script

jlink --module-path
%JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
terminalfx --launcher terminalfx=terminalfx/com.terminalfx.AppStarter
--output target/release

However it gives me the following error;

Error: module-info.class not found for jackson.databind module

My module-info.java is:

module terminalfx {

    requires jackson.annotations;
    requires javafx.graphics;
    requires javafx.controls;
    requires javafx.fxml;
    requires javafx.web;
    requires jackson.core;
    requires pty4j;
    requires jackson.databind;
    requires jna;
    requires jdk.jsobject;

    exports com.terminalfx;
    exports com.terminalfx.config to jackson.databind;

}


and the project is located here in *java9* branch
https://github.com/rahmanusta/TerminalFX

How can I solve this issue?

Thanks.

-- 
Rahman USTA
Istanbul JUG

From alex.buckley at oracle.com  Fri Sep  1 21:06:16 2017
From: alex.buckley at oracle.com (Alex Buckley)
Date: Fri, 01 Sep 2017 14:06:16 -0700
Subject: Moving to Java 9 - module-info.class not found for <module-name>
 module
In-Reply-To: <CAGe7fv36O6JsYERF=KB1sS=71wWvdHutHwhebkfpnhTKGjQpNQ@mail.gmail.com>
References: <CAGe7fv36O6JsYERF=KB1sS=71wWvdHutHwhebkfpnhTKGjQpNQ@mail.gmail.com>
Message-ID: <59A9CBC8.9000306@oracle.com>

On 9/1/2017 1:21 PM, Rahman USTA wrote:
> java --module-path
> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
> terminalfx -m terminalfx/com.terminalfx.AppStarter

(You shouldn't need the --add-modules, since terminalfx is already the 
main module.)

> It works normally. Then, I want to generate a jlink image with the
> following script
>
> jlink --module-path
> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
> terminalfx --launcher terminalfx=terminalfx/com.terminalfx.AppStarter
> --output target/release
>
> However it gives me the following error;
>
> Error: module-info.class not found for jackson.databind module

I suspect jackson.databind is an automatic module. jlink does not 
support linking of automatic modules because they can rely on the 
arbitrary content of the classpath, which goes against the idea of a 
self-contained Java runtime.

Alex

From forax at univ-mlv.fr  Fri Sep  1 21:59:18 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Fri, 1 Sep 2017 23:59:18 +0200 (CEST)
Subject: Moving to Java 9 - module-info.class not found for
 <module-name> module
In-Reply-To: <59A9CBC8.9000306@oracle.com>
References: <CAGe7fv36O6JsYERF=KB1sS=71wWvdHutHwhebkfpnhTKGjQpNQ@mail.gmail.com>
 <59A9CBC8.9000306@oracle.com>
Message-ID: <1618721687.2391736.1504303158586.JavaMail.zimbra@u-pem.fr>

As Alex said,
jackson.databind is not an explicit module.

You should report that issue to their bugtracker.

cheers,
R?mi

----- Mail original -----
> De: "Alex Buckley" <alex.buckley at oracle.com>
> ?: "jigsaw-dev" <jigsaw-dev at openjdk.java.net>
> Envoy?: Vendredi 1 Septembre 2017 23:06:16
> Objet: Re: Moving to Java 9 - module-info.class not found for <module-name> module

> On 9/1/2017 1:21 PM, Rahman USTA wrote:
>> java --module-path
>> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
>> terminalfx -m terminalfx/com.terminalfx.AppStarter
> 
> (You shouldn't need the --add-modules, since terminalfx is already the
> main module.)
> 
>> It works normally. Then, I want to generate a jlink image with the
>> following script
>>
>> jlink --module-path
>> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
>> terminalfx --launcher terminalfx=terminalfx/com.terminalfx.AppStarter
>> --output target/release
>>
>> However it gives me the following error;
>>
>> Error: module-info.class not found for jackson.databind module
> 
> I suspect jackson.databind is an automatic module. jlink does not
> support linking of automatic modules because they can rely on the
> arbitrary content of the classpath, which goes against the idea of a
> self-contained Java runtime.
> 
> Alex

From rahman.usta.88 at gmail.com  Sat Sep  2 11:30:24 2017
From: rahman.usta.88 at gmail.com (Rahman USTA)
Date: Sat, 2 Sep 2017 14:30:24 +0300
Subject: Moving to Java 9 - module-info.class not found for <module-name>
 module
In-Reply-To: <1618721687.2391736.1504303158586.JavaMail.zimbra@u-pem.fr>
References: <CAGe7fv36O6JsYERF=KB1sS=71wWvdHutHwhebkfpnhTKGjQpNQ@mail.gmail.com>
 <59A9CBC8.9000306@oracle.com>
 <1618721687.2391736.1504303158586.JavaMail.zimbra@u-pem.fr>
Message-ID: <CAGe7fv2odN3Z1gF+xJ+tbXbSYXCBh2=dmCGEGrQfxYaru5ig0Q@mail.gmail.com>

Thank you, Remi and Alex.

I think automatic modules can be supported by jlink. It could be a
transition feature like --illegal-access imho.

Kind regards.

2017-09-02 0:59 GMT+03:00 Remi Forax <forax at univ-mlv.fr>:

> As Alex said,
> jackson.databind is not an explicit module.
>
> You should report that issue to their bugtracker.
>
> cheers,
> R?mi
>
> ----- Mail original -----
> > De: "Alex Buckley" <alex.buckley at oracle.com>
> > ?: "jigsaw-dev" <jigsaw-dev at openjdk.java.net>
> > Envoy?: Vendredi 1 Septembre 2017 23:06:16
> > Objet: Re: Moving to Java 9 - module-info.class not found for
> <module-name> module
>
> > On 9/1/2017 1:21 PM, Rahman USTA wrote:
> >> java --module-path
> >> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
> >> terminalfx -m terminalfx/com.terminalfx.AppStarter
> >
> > (You shouldn't need the --add-modules, since terminalfx is already the
> > main module.)
> >
> >> It works normally. Then, I want to generate a jlink image with the
> >> following script
> >>
> >> jlink --module-path
> >> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
> >> terminalfx --launcher terminalfx=terminalfx/com.terminalfx.AppStarter
> >> --output target/release
> >>
> >> However it gives me the following error;
> >>
> >> Error: module-info.class not found for jackson.databind module
> >
> > I suspect jackson.databind is an automatic module. jlink does not
> > support linking of automatic modules because they can rely on the
> > arbitrary content of the classpath, which goes against the idea of a
> > self-contained Java runtime.
> >
> > Alex
>



-- 
Rahman USTA
Istanbul JUG
https://github.com/rahmanusta

From forax at univ-mlv.fr  Sat Sep  2 11:43:19 2017
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Sat, 2 Sep 2017 13:43:19 +0200 (CEST)
Subject: Moving to Java 9 - module-info.class not found for
 <module-name> module
In-Reply-To: <CAGe7fv2odN3Z1gF+xJ+tbXbSYXCBh2=dmCGEGrQfxYaru5ig0Q@mail.gmail.com>
References: <CAGe7fv36O6JsYERF=KB1sS=71wWvdHutHwhebkfpnhTKGjQpNQ@mail.gmail.com>
 <59A9CBC8.9000306@oracle.com>
 <1618721687.2391736.1504303158586.JavaMail.zimbra@u-pem.fr>
 <CAGe7fv2odN3Z1gF+xJ+tbXbSYXCBh2=dmCGEGrQfxYaru5ig0Q@mail.gmail.com>
Message-ID: <686786261.2446055.1504352599651.JavaMail.zimbra@u-pem.fr>

> De: "Rahman USTA" <rahman.usta.88 at gmail.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "Alex Buckley" <alex.buckley at oracle.com>, "jigsaw-dev"
> <jigsaw-dev at openjdk.java.net>
> Envoy?: Samedi 2 Septembre 2017 13:30:24
> Objet: Re: Moving to Java 9 - module-info.class not found for <module-name>
> module

> Thank you, Remi and Alex.
> I think automatic modules can be supported by jlink. It could be a transition
> feature like --illegal-access imho.

The whole idea of jlink is that unlike classical Java it works under the assumption that the world is closed i.e. all the bytecodes that will be loaded is known, so it can do "whole world" optimizations like i know all the lambda forms that will be used by the string concatenation because i know all the call sites that does string concatenation so i can pre-generate them. 
If you have an automatic module, it can have dependencies on jars from the classpath, the close world assumption does not hold. 

If you are in a hacky mood, there is a simple solution to transform an automatic module to an explicit one, create a module-info.java, compile it and update the jar of the automatic module with the module-info.class or better ask the maintainer of the jar to do the work for you :) 

> Kind regards.

regards, 
R?mi 

> 2017-09-02 0:59 GMT+03:00 Remi Forax < [ mailto:forax at univ-mlv.fr |
> forax at univ-mlv.fr ] > :

>> As Alex said,
>> jackson.databind is not an explicit module.

>> You should report that issue to their bugtracker.

>> cheers,
>> R?mi

>> ----- Mail original -----
>>> De: "Alex Buckley" < [ mailto:alex.buckley at oracle.com | alex.buckley at oracle.com
>> > ] >
>>> ?: "jigsaw-dev" < [ mailto:jigsaw-dev at openjdk.java.net |
>> > jigsaw-dev at openjdk.java.net ] >
>> > Envoy?: Vendredi 1 Septembre 2017 23:06:16
>>> Objet: Re: Moving to Java 9 - module-info.class not found for <module-name>
>> > module

>> > On 9/1/2017 1:21 PM, Rahman USTA wrote:
>> >> java --module-path
>> >> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
>> >> terminalfx -m terminalfx/com.terminalfx.AppStarter

>> > (You shouldn't need the --add-modules, since terminalfx is already the
>> > main module.)

>> >> It works normally. Then, I want to generate a jlink image with the
>> >> following script

>> >> jlink --module-path
>> >> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
>> >> terminalfx --launcher terminalfx=terminalfx/com.terminalfx.AppStarter
>> >> --output target/release

>> >> However it gives me the following error;

>> >> Error: module-info.class not found for jackson.databind module

>> > I suspect jackson.databind is an automatic module. jlink does not
>> > support linking of automatic modules because they can rely on the
>> > arbitrary content of the classpath, which goes against the idea of a
>> > self-contained Java runtime.

>> > Alex

> --
> Rahman USTA
> Istanbul JUG
> [ https://github.com/rahmanusta | https://github.com/rahmanusta ]

From kubota.yuji at gmail.com  Tue Sep  5 11:01:57 2017
From: kubota.yuji at gmail.com (KUBOTA Yuji)
Date: Tue, 5 Sep 2017 20:01:57 +0900
Subject: [Question] Accessibility of each modules
Message-ID: <CABU-27NSLsgik+WS6au34gP4z9-p5rPRT3jxAztzPmnDbL7ZkA@mail.gmail.com>

Hi all,

I have a question about accessibility of each modules: unnamed,
automatic and named modules.
* unnamed: the JAR compiled by java 8 and speficied by --class-path.
* automatic: the JAR compiled by Java 8 and specified by --module-path.
* named: the modular JAR.

I ran quick-start's example program on java 9+181 and 8u141, and
confirmed the following results.
* Named modules can call autonamed modules.
* Automatic modules can *NOT* call named modules.
* We can run unnamed modules with only unnamed modules.

You can reproduce it by https://github.com/ykubota/jigsaw-sample-accessibility

I guessed that every modules can be accessed each other if
`--illegal-access=permit` or/and `--add-exports` is given, but it does
not seems to be fine.
I guess that modules and classes (unnamed module) cannot be existed at
the same time because `BuiltinClassLoader` seems to branch by
`packageToModule` field in it.

Is my understanding correct?

Thanks,
Yuji

From Alan.Bateman at oracle.com  Tue Sep  5 11:31:30 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 5 Sep 2017 12:31:30 +0100
Subject: [Question] Accessibility of each modules
In-Reply-To: <CABU-27NSLsgik+WS6au34gP4z9-p5rPRT3jxAztzPmnDbL7ZkA@mail.gmail.com>
References: <CABU-27NSLsgik+WS6au34gP4z9-p5rPRT3jxAztzPmnDbL7ZkA@mail.gmail.com>
Message-ID: <85666e93-148a-761d-9f4c-17c7ba0f2c21@oracle.com>

On 05/09/2017 12:01, KUBOTA Yuji wrote:
> Hi all,
>
> I have a question about accessibility of each modules: unnamed,
> automatic and named modules.
> * unnamed: the JAR compiled by java 8 and speficied by --class-path.
> * automatic: the JAR compiled by Java 8 and specified by --module-path.
> * named: the modular JAR.
For the modular JAR case then the term to use is "explicit module".

>
> I ran quick-start's example program on java 9+181 and 8u141, and
> confirmed the following results.
> * Named modules can call autonamed modules.
Yes, assuming the explicit module reads the automatic module. As an 
automatic module exports all packages then it means code in the explicit 
module can access all public members of public classes in the automatic 
module.

> * Automatic modules can *NOT* call named modules.
Automatic modules read all named modules (in the module graph), they 
also read all unnamed modules. I have not studied your github project 
but you should see that code in the automatic module can access all 
public members of public classes in packages exported by the explicit 
module (because it reads the explicit module).

> * We can run unnamed modules with only unnamed modules.
I'm not what you mean by this because you can mix the class path with 
named modules. When you run HelloWorld on the class path then you are 
doing this, although it might not be obvious.

If it helps then the code in an unnamed module an access all public 
members of public classes in all unnamed modules. It can also access all 
public members of public classes in the packages exported by named 
modules (automatic modules are named modules, they export all packages).

>
> I guessed that every modules can be accessed each other if
> `--illegal-access=permit` or/and `--add-exports` is given, but it does
> not seems to be fine.
The `--add-exports` option is for breaking encapsulation, it can be used 
to change a module so that it exports an otherwise concealed package to 
another named module or to all unnamed modules.

The `--illegal-access=<value>` option is specific to the standard and 
JDK modules.

> I guess that modules and classes (unnamed module) cannot be existed at
> the same time because `BuiltinClassLoader` seems to branch by
> `packageToModule` field in it.
No, that is just an implementation detail. The issue that was chosen not 
to tackle in JDK 9 is the issue of multiple modules in the boot layer 
with the same non-exported package. This topic has a lot of complexity 
and compatibility issues for (non-JDK) code. It will be looked in the 
future.

-Alan

From kubota.yuji at gmail.com  Tue Sep  5 13:08:54 2017
From: kubota.yuji at gmail.com (KUBOTA Yuji)
Date: Tue, 05 Sep 2017 13:08:54 +0000
Subject: [Question] Accessibility of each modules
In-Reply-To: <85666e93-148a-761d-9f4c-17c7ba0f2c21@oracle.com>
References: <CABU-27NSLsgik+WS6au34gP4z9-p5rPRT3jxAztzPmnDbL7ZkA@mail.gmail.com>
 <85666e93-148a-761d-9f4c-17c7ba0f2c21@oracle.com>
Message-ID: <CABU-27NevBXxpYVVNkMtm0iLG17oh7Rb6YA9Nb_QWqaOAgHUJg@mail.gmail.com>

Hi Alan,

Thank you very much for your quickly and detailed explanation!

I think that your explanation make sense except accessibility of automatic
module.
My following example program shows that automatic module can't read
exported package by explicit module.

$ /jdk-9/bin/jar -d --file mlibs/org.astro.jar
org.astro jar:file:///jigsaw-sample/mlibs/org.astro.jar/!module-info.class
exports org.astro
requires java.base mandated

$ /jdk-9/bin/jar -d --file libs/com.greetings.jar
No module descriptor found. Derived automatic module.

com.greetings automatic
requires java.base mandated
contains com.greetings

$ /jdk-9/bin/java -p mlibs:libs -m com.greetings/com.greetings.Main
Exception in thread "main" java.lang.NoClassDefFoundError: org/astro/World
        at com.greetings/com.greetings.Main.main(Main.java:7)
Caused by: java.lang.ClassNotFoundException: org.astro.World
        at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
        at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:185)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:496)
        ... 1 more


Thanks,
Yuji


2017-09-05 20:31 GMT+09:00 Alan Bateman <Alan.Bateman at oracle.com>:
> On 05/09/2017 12:01, KUBOTA Yuji wrote:
>>
>> Hi all,
>>
>> I have a question about accessibility of each modules: unnamed,
>> automatic and named modules.
>> * unnamed: the JAR compiled by java 8 and speficied by --class-path.
>> * automatic: the JAR compiled by Java 8 and specified by --module-path.
>> * named: the modular JAR.
>
> For the modular JAR case then the term to use is "explicit module".
>
>>
>> I ran quick-start's example program on java 9+181 and 8u141, and
>> confirmed the following results.
>> * Named modules can call autonamed modules.
>
> Yes, assuming the explicit module reads the automatic module. As an
> automatic module exports all packages then it means code in the explicit
> module can access all public members of public classes in the automatic
> module.
>
>> * Automatic modules can *NOT* call named modules.
>
> Automatic modules read all named modules (in the module graph), they also
> read all unnamed modules. I have not studied your github project but you
> should see that code in the automatic module can access all public members
> of public classes in packages exported by the explicit module (because it
> reads the explicit module).
>
>> * We can run unnamed modules with only unnamed modules.
>
> I'm not what you mean by this because you can mix the class path with
named
> modules. When you run HelloWorld on the class path then you are doing
this,
> although it might not be obvious.
>
> If it helps then the code in an unnamed module an access all public
members
> of public classes in all unnamed modules. It can also access all public
> members of public classes in the packages exported by named modules
> (automatic modules are named modules, they export all packages).
>
>>
>> I guessed that every modules can be accessed each other if
>> `--illegal-access=permit` or/and `--add-exports` is given, but it does
>> not seems to be fine.
>
> The `--add-exports` option is for breaking encapsulation, it can be used
to
> change a module so that it exports an otherwise concealed package to
another
> named module or to all unnamed modules.
>
> The `--illegal-access=<value>` option is specific to the standard and JDK
> modules.
>
>> I guess that modules and classes (unnamed module) cannot be existed at
>> the same time because `BuiltinClassLoader` seems to branch by
>> `packageToModule` field in it.
>
> No, that is just an implementation detail. The issue that was chosen not
to
> tackle in JDK 9 is the issue of multiple modules in the boot layer with
the
> same non-exported package. This topic has a lot of complexity and
> compatibility issues for (non-JDK) code. It will be looked in the future.
>
> -Alan

From Alan.Bateman at oracle.com  Tue Sep  5 13:16:56 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 5 Sep 2017 14:16:56 +0100
Subject: [Question] Accessibility of each modules
In-Reply-To: <CABU-27NevBXxpYVVNkMtm0iLG17oh7Rb6YA9Nb_QWqaOAgHUJg@mail.gmail.com>
References: <CABU-27NSLsgik+WS6au34gP4z9-p5rPRT3jxAztzPmnDbL7ZkA@mail.gmail.com>
 <85666e93-148a-761d-9f4c-17c7ba0f2c21@oracle.com>
 <CABU-27NevBXxpYVVNkMtm0iLG17oh7Rb6YA9Nb_QWqaOAgHUJg@mail.gmail.com>
Message-ID: <b5d3dda1-fa1d-e6e2-3878-60f687619445@oracle.com>

On 05/09/2017 14:08, KUBOTA Yuji wrote:
>
> My following example program shows that automatic module can't read 
> exported package by explicit module.
>
> $ /jdk-9/bin/jar -d --file mlibs/org.astro.jar
> org.astro jar:file:///jigsaw-sample/mlibs/org.astro.jar/!module-info.class
> exports org.astro
> requires java.base mandated
>
> $ /jdk-9/bin/jar -d --file libs/com.greetings.jar
> No module descriptor found. Derived automatic module.
>
> com.greetings automatic
> requires java.base mandated
> contains com.greetings
>
> $ /jdk-9/bin/java -p mlibs:libs -m com.greetings/com.greetings.Main
> Exception in thread "main" java.lang.NoClassDefFoundError: org/astro/World
> ? ? ? ? at com.greetings/com.greetings.Main.main(Main.java:7)
> Caused by: java.lang.ClassNotFoundException: org.astro.World
> ? ? ? ? at 
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
> ? ? ? ? at 
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:185)
> ? ? ? ? at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:496)
> ? ? ? ? ... 1 more
>
The issue here is that org.astro is not resolved because no module 
requires org.astro. If you add `--add-modules org.astro` to the command 
line then it will ensure that this module is resolved and you should see 
that com.greetings reads it.

-Alan

From kubota.yuji at gmail.com  Tue Sep  5 14:55:58 2017
From: kubota.yuji at gmail.com (KUBOTA Yuji)
Date: Tue, 5 Sep 2017 23:55:58 +0900
Subject: [Question] Accessibility of each modules
In-Reply-To: <b5d3dda1-fa1d-e6e2-3878-60f687619445@oracle.com>
References: <CABU-27NSLsgik+WS6au34gP4z9-p5rPRT3jxAztzPmnDbL7ZkA@mail.gmail.com>
 <85666e93-148a-761d-9f4c-17c7ba0f2c21@oracle.com>
 <CABU-27NevBXxpYVVNkMtm0iLG17oh7Rb6YA9Nb_QWqaOAgHUJg@mail.gmail.com>
 <b5d3dda1-fa1d-e6e2-3878-60f687619445@oracle.com>
Message-ID: <CABU-27NWuy6tQm85JRZO8XdgOO3iuugMPDCKZLQ2sjSGO9yTzg@mail.gmail.com>

2017-09-05 22:16 GMT+09:00 Alan Bateman <Alan.Bateman at oracle.com>:
> The issue here is that org.astro is not resolved because no module requires
> org.astro. If you add `--add-modules org.astro` to the command line then it
> will ensure that this module is resolved and you should see that
> com.greetings reads it.

Thank you very much!

Following your advice, I could investigate the implementation how
required module add to module graphs. (e.g.,
java.lang.module.Resolver)
I think that developers should know how to create module graphs
correctly before modularing our applications step-by-step. I will
re-investigate specifications and implementations, and write an
article in my native language to help my neighbors.

Thanks again!
Yuji

From sormuras at gmail.com  Mon Sep 11 10:03:37 2017
From: sormuras at gmail.com (Christian Stein)
Date: Mon, 11 Sep 2017 12:03:37 +0200
Subject: ServiceLoader usage in automatic module fails
Message-ID: <CAJi8+g5vZwp+Yvk_vh8aawop8otBLvUZy2qRzLgCqNWmy5LZhw@mail.gmail.com>

Hi jigsaw team,

I discovered an issue with the ServiceLoader trying to load services on the
module-path, when the actual ServiceLoader.load() call resides in an
automatic module.

I compiled a small demo at [1] and it's console output is visible at [2].
If you want to re-run the demo, just call "jshell build.jsh" in the root
directory.

In summary, you have to elevate the automatic module to an explicit one,
via adding a module-info.class, to get the ServiceLoader load the service
on the module-path. This can be done with pro's [3] "modulefixer" plugin.

When resorting the class-path and provide a "META-INF/services" file and
also add the module to the class-path, I can get service loaded ... but
only within the unnamed module.

Is it an internal ServiceLoader bug or is this behaviour intented?

Might a theoretical "--add-uses <automatic-module-name>=<SPI>" help here?

Kind regards,
Christian

[1] https://github.com/sormuras/sawdust/tree/master/module-discoverer
[2] https://gist.github.com/sormuras/32ece436447ba5a3eac24e9db5842fb5
[3] https://github.com/forax/pro

From Alan.Bateman at oracle.com  Mon Sep 11 10:52:06 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Mon, 11 Sep 2017 11:52:06 +0100
Subject: ServiceLoader usage in automatic module fails
In-Reply-To: <CAJi8+g5vZwp+Yvk_vh8aawop8otBLvUZy2qRzLgCqNWmy5LZhw@mail.gmail.com>
References: <CAJi8+g5vZwp+Yvk_vh8aawop8otBLvUZy2qRzLgCqNWmy5LZhw@mail.gmail.com>
Message-ID: <bfbce4ac-cb1d-89e7-768e-615f4705486c@oracle.com>



On 11/09/2017 11:03, Christian Stein wrote:
> Hi jigsaw team,
>
> I discovered an issue with the ServiceLoader trying to load services on the
> module-path, when the actual ServiceLoader.load() call resides in an
> automatic module.
>
> I compiled a small demo at [1] and it's console output is visible at [2].
> If you want to re-run the demo, just call "jshell build.jsh" in the root
> directory.
>
> In summary, you have to elevate the automatic module to an explicit one,
> via adding a module-info.class, to get the ServiceLoader load the service
> on the module-path. This can be done with pro's [3] "modulefixer" plugin.
>
> When resorting the class-path and provide a "META-INF/services" file and
> also add the module to the class-path, I can get service loaded ... but
> only within the unnamed module.
Which JAR file contains the service provider? You mentioned "restoring 
the class-path and provide a META-INF/services". Does this mean you 
removed the services configuration file when you had the JAR file on the 
module path as an automatic module?

-Alan

From sormuras at gmail.com  Mon Sep 11 11:12:04 2017
From: sormuras at gmail.com (Christian Stein)
Date: Mon, 11 Sep 2017 13:12:04 +0200
Subject: ServiceLoader usage in automatic module fails
In-Reply-To: <bfbce4ac-cb1d-89e7-768e-615f4705486c@oracle.com>
References: <CAJi8+g5vZwp+Yvk_vh8aawop8otBLvUZy2qRzLgCqNWmy5LZhw@mail.gmail.com>
 <bfbce4ac-cb1d-89e7-768e-615f4705486c@oracle.com>
Message-ID: <CAJi8+g7PLvFHKCGrGQtt+7htDT-8E3H+FcUMbHz9hhQMXsS2gQ@mail.gmail.com>

> Which JAR file contains the service provider?

The service providing interface [1] is inside the api jar [2].
The service loader code resides in the Jupiter engine [3] jar and looks
like [4].

[1] org.junit.jupiter.api.extension.Extension
[2]
http://central.maven.org/maven2/org/junit/jupiter/junit-jupiter-api/5.0.0/
[3]
http://central.maven.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.0.0/
[4]
https://github.com/junit-team/junit5/blob/master/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/ExtensionRegistry.java#L93


> You mentioned "restoring the class-path and provide a META-INF/services".
> Does this mean you removed the services configuration file when you
> had the JAR file on the module path as an automatic module?

No. The "demo" module is an explicit module with no service configuration.
It implements the SPI [1] and declares that in a module descriptor [5].

I wanted to write "resorting to the class-path". When I add a service
configuration file to the explicite demo module [6] at runtime and add
the location of the demo module to the class-path, the demo service
is loaded ... from the unnamed module.

[5]
https://github.com/sormuras/sawdust/blob/master/module-discoverer/demo/src/module-info.java
[6]
https://github.com/sormuras/sawdust/blob/master/module-discoverer/build.jsh#L82


On Mon, Sep 11, 2017 at 12:52 PM, Alan Bateman <Alan.Bateman at oracle.com>
wrote:

>
>
> On 11/09/2017 11:03, Christian Stein wrote:
>
>> Hi jigsaw team,
>>
>> I discovered an issue with the ServiceLoader trying to load services on
>> the
>> module-path, when the actual ServiceLoader.load() call resides in an
>> automatic module.
>>
>> I compiled a small demo at [1] and it's console output is visible at [2].
>> If you want to re-run the demo, just call "jshell build.jsh" in the root
>> directory.
>>
>> In summary, you have to elevate the automatic module to an explicit one,
>> via adding a module-info.class, to get the ServiceLoader load the service
>> on the module-path. This can be done with pro's [3] "modulefixer" plugin.
>>
>> When resorting the class-path and provide a "META-INF/services" file and
>> also add the module to the class-path, I can get service loaded ... but
>> only within the unnamed module.
>>
> Which JAR file contains the service provider? You mentioned "restoring the
> class-path and provide a META-INF/services". Does this mean you removed the
> services configuration file when you had the JAR file on the module path as
> an automatic module?
>
> -Alan
>

From Alan.Bateman at oracle.com  Mon Sep 11 11:17:31 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Mon, 11 Sep 2017 12:17:31 +0100
Subject: ServiceLoader usage in automatic module fails
In-Reply-To: <bfbce4ac-cb1d-89e7-768e-615f4705486c@oracle.com>
References: <CAJi8+g5vZwp+Yvk_vh8aawop8otBLvUZy2qRzLgCqNWmy5LZhw@mail.gmail.com>
 <bfbce4ac-cb1d-89e7-768e-615f4705486c@oracle.com>
Message-ID: <7f4452da-f729-13ea-d691-896308a460cc@oracle.com>



On 11/09/2017 11:52, Alan Bateman wrote:
> :
>
> Which JAR file contains the service provider? You mentioned "restoring 
> the class-path and provide a META-INF/services". Does this mean you 
> removed the services configuration file when you had the JAR file on 
> the module path as an automatic module?
>
Also can you run with `--add-modules demo` as I assume the scenario you 
is that "demo" is not resolved (because no modules `requires demo` and 
no module declares that it uses the service type that demo provides).

-Alan

From sormuras at gmail.com  Mon Sep 11 11:26:15 2017
From: sormuras at gmail.com (Christian Stein)
Date: Mon, 11 Sep 2017 13:26:15 +0200
Subject: ServiceLoader usage in automatic module fails
In-Reply-To: <7f4452da-f729-13ea-d691-896308a460cc@oracle.com>
References: <CAJi8+g5vZwp+Yvk_vh8aawop8otBLvUZy2qRzLgCqNWmy5LZhw@mail.gmail.com>
 <bfbce4ac-cb1d-89e7-768e-615f4705486c@oracle.com>
 <7f4452da-f729-13ea-d691-896308a460cc@oracle.com>
Message-ID: <CAJi8+g4O=P9GMcXnROhAMJsfKnip-6t1e9YaooRf6qYaOkyp1Q@mail.gmail.com>

On Mon, Sep 11, 2017 at 1:17 PM, Alan Bateman <Alan.Bateman at oracle.com>
wrote:

[...]
>
> Also can you run with `--add-modules demo` as I assume the scenario you is
> that "demo" is not resolved (because no modules `requires demo` and no
> module declares that it uses the service type that demo provides).
>
> -Alan
>

Thanks, Alan. That worked like charm!

From Alan.Bateman at oracle.com  Mon Sep 11 12:21:30 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Mon, 11 Sep 2017 13:21:30 +0100
Subject: ServiceLoader usage in automatic module fails
In-Reply-To: <CAJi8+g4O=P9GMcXnROhAMJsfKnip-6t1e9YaooRf6qYaOkyp1Q@mail.gmail.com>
References: <CAJi8+g5vZwp+Yvk_vh8aawop8otBLvUZy2qRzLgCqNWmy5LZhw@mail.gmail.com>
 <bfbce4ac-cb1d-89e7-768e-615f4705486c@oracle.com>
 <7f4452da-f729-13ea-d691-896308a460cc@oracle.com>
 <CAJi8+g4O=P9GMcXnROhAMJsfKnip-6t1e9YaooRf6qYaOkyp1Q@mail.gmail.com>
Message-ID: <340a57a3-c26b-5593-20ec-923d4b853fcc@oracle.com>



On 11/09/2017 12:26, Christian Stein wrote:
> On Mon, Sep 11, 2017 at 1:17 PM, Alan Bateman <Alan.Bateman at oracle.com>
> wrote:
>
> [...]
>> Also can you run with `--add-modules demo` as I assume the scenario you is
>> that "demo" is not resolved (because no modules `requires demo` and no
>> module declares that it uses the service type that demo provides).
>>
>> -Alan
>>
> Thanks, Alan. That worked like charm!
You can also use `--show-module-resolution` to get traces at startup and 
satisfied yourself that it working correctly.

-Alan

From sormuras at gmail.com  Tue Sep 12 10:11:40 2017
From: sormuras at gmail.com (Christian Stein)
Date: Tue, 12 Sep 2017 12:11:40 +0200
Subject: ServiceLoader usage in automatic module fails
In-Reply-To: <340a57a3-c26b-5593-20ec-923d4b853fcc@oracle.com>
References: <CAJi8+g5vZwp+Yvk_vh8aawop8otBLvUZy2qRzLgCqNWmy5LZhw@mail.gmail.com>
 <bfbce4ac-cb1d-89e7-768e-615f4705486c@oracle.com>
 <7f4452da-f729-13ea-d691-896308a460cc@oracle.com>
 <CAJi8+g4O=P9GMcXnROhAMJsfKnip-6t1e9YaooRf6qYaOkyp1Q@mail.gmail.com>
 <340a57a3-c26b-5593-20ec-923d4b853fcc@oracle.com>
Message-ID: <CAJi8+g5VPgYpkAgKOJfbKz-uAO9BCoR+eQce_z-H5P265a2S_w@mail.gmail.com>

On Mon, Sep 11, 2017 at 2:21 PM, Alan Bateman <Alan.Bateman at oracle.com>
 wrote:

> [...]
>
> You can also use `--show-module-resolution` to get traces at startup and
> satisfied yourself that it working correctly.


Using the symbolic constant `--add-modules ALL-MODULE-PATH` works even
better when scanning module contents for test classes.

See [1] for a proof-of-concept implementation that ship with JUnit 5.1...

Cheers,
Christian

[1] https://github.com/junit-team/junit5/pull/1057#issuecomment-328806352

From mandy.chung at oracle.com  Tue Sep 12 22:33:05 2017
From: mandy.chung at oracle.com (mandy chung)
Date: Tue, 12 Sep 2017 15:33:05 -0700
Subject: RFR(S): 8138600: eliminate the need of ModuleLoaderMap.dat for CDS
In-Reply-To: <59B85C30.1000108@oracle.com>
References: <59B85C30.1000108@oracle.com>
Message-ID: <f9041ae4-58f8-d753-b7d5-ad58a03c22e8@oracle.com>

(I move this thread from jdk10-dev to jigsaw-dev which can review the 
jdk change).

The JDK change looks good to me.

Mandy

On 9/12/17 3:14 PM, Calvin Cheung wrote:
> With the fix for JDK-8186842 
> <https://bugs.openjdk.java.net/browse/JDK-8186842>, we no longer need 
> the ModuleLoaderMap to determine the loader type of a class during CDS 
> dump time. This change is for removing the ModuleLoaderMap from the 
> runtime image and the code which references it.
>
> bug: https://bugs.openjdk.java.net/browse/JDK-8138600
>
> webrevs:
> http://cr.openjdk.java.net/~ccheung/8138600/jdk/webrev.00/
> http://cr.openjdk.java.net/~ccheung/8138600/hotspot/webrev.00/
>
> Testing:
> ??? JPRT
> ??? CDS tests
> ??? hs-tier3
>
> thanks,
> Calvin
>


From calvin.cheung at oracle.com  Tue Sep 12 23:29:33 2017
From: calvin.cheung at oracle.com (Calvin Cheung)
Date: Tue, 12 Sep 2017 16:29:33 -0700
Subject: RFR(S): 8138600: eliminate the need of ModuleLoaderMap.dat for CDS
In-Reply-To: <f9041ae4-58f8-d753-b7d5-ad58a03c22e8@oracle.com>
References: <59B85C30.1000108@oracle.com>
 <f9041ae4-58f8-d753-b7d5-ad58a03c22e8@oracle.com>
Message-ID: <59B86DDD.9030309@oracle.com>

Thanks for your review Mandy.

Calvin

On 9/12/17, 3:33 PM, mandy chung wrote:
> (I move this thread from jdk10-dev to jigsaw-dev which can review the 
> jdk change).
>
> The JDK change looks good to me.
>
> Mandy
>
> On 9/12/17 3:14 PM, Calvin Cheung wrote:
>> With the fix for JDK-8186842 
>> <https://bugs.openjdk.java.net/browse/JDK-8186842>, we no longer need 
>> the ModuleLoaderMap to determine the loader type of a class during 
>> CDS dump time. This change is for removing the ModuleLoaderMap from 
>> the runtime image and the code which references it.
>>
>> bug: https://bugs.openjdk.java.net/browse/JDK-8138600
>>
>> webrevs:
>> http://cr.openjdk.java.net/~ccheung/8138600/jdk/webrev.00/
>> http://cr.openjdk.java.net/~ccheung/8138600/hotspot/webrev.00/
>>
>> Testing:
>>     JPRT
>>     CDS tests
>>     hs-tier3
>>
>> thanks,
>> Calvin
>>
>

From gregw at webtide.com  Wed Sep 13 22:12:17 2017
From: gregw at webtide.com (Greg Wilkins)
Date: Thu, 14 Sep 2017 08:12:17 +1000
Subject: Scanning multi version jars?
Message-ID: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>

I hope this is the right group for this question. please redirect me if not.

The Jetty project is trying to implement annotation scanning for multi
version jars and have some concerns with some edge cases, specifically with
inner classes.

A multi versioned jar might contain something like:

   - org/example/Foo.class
   - org/example/Foo$Bar.class
   - META-INF/versions/9/org/example/Foo.class

It is clear that there is a java 9 version of Foo.  But what is unclear is
the inner class Foo$Bar?  Is that only used by the base Foo version? or
does the java 9 version also use the Foo$Bar inner class, but it didn't use
any java 9 features, so the base version is able to be used??

It is unclear from just an index of the jar if we should be
scanning Foo$Bar for annotations.  So currently it appears that we have to
actually scan the Foo class to see if Foo$Bar is referenced and only then
scan Foo$Bar for annotations (and recursive analysis for any Foo$Bar$Bob
class )!

An alternative would be if there was an enforced convention that any
versioned class would also version all of it's inner classes - which may be
a reasonable assumption given that they would be compiled together, but we
see nothing in the specifications that force a jar to be assembled that way.

Any guidance anybody can give would be helpful.

cheers
Greg Wilkins

PS. The JarFile.getEntry method does not appear to respect it's javadoc
with respect to multiversioned jars: it says it will do a search for the
most recent version, however the code indicates that the search is only
done if the base version does not exist.  This is kind of separate issue,
but makes it difficult to defer the behaviour of what to scan to the
implementation in JarFile






-- 
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From stephen.felts at oracle.com  Thu Sep 14 03:15:01 2017
From: stephen.felts at oracle.com (Stephen Felts)
Date: Thu, 14 Sep 2017 03:15:01 +0000 (UTC)
Subject: Scanning multi version jars?
In-Reply-To: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
Message-ID: <94bf15d0-01f3-4a8d-b6b6-4e8c2c8c39f6@default>

We ran into this problem, where we have a closed-set class checker and it has a problem processing MR jar files.

I recommend replacing all inner classes if the ordinary class is versioned.  If the inner class goes away, you would need to stub it so a versioned copy exists.  That is the convention we have adopted for our project but  that is not currently a universal convention (and I believe that we saw one MR jar that didn't follow this convention - maybe the new JAXB jar?).

If you use the JDK9 API to scan classes in a jar file for version 9, you will get META-INF/versions/9/org/example/Foo.class  and org/example/Foo$Bar.class. 

This is the code that we use to read the classes.

>         // the JDK9 mode.  This is accomplished by supplying the "base version" to the constructor.
>         Method baseVersionMethod = null;
>         try {
>           baseVersionMethod = JarFile.class.getMethod("baseVersion");
>         } catch (NoSuchMethodException nsme) {}
>
>         Object baseVersion = null;
>         if (baseVersionMethod != null) {
>           try {
>             baseVersion = baseVersionMethod.invoke(null);
>           } catch (IllegalAccessException | IllegalArgumentException | InvocationTargetException e) {
>             throw new RuntimeException(e);
>           }
>         }
>
>         Constructor<?> jarFileConstructor = null;
>         if (baseVersion != null) {
>           try {
>             jarFileConstructor = JarFile.class.getConstructor(File.class, boolean.class, int.class, baseVersion.getClass());
>           } catch (NoSuchMethodException | SecurityException e) {
>             throw new RuntimeException(e);
>           }
>         }
>
>         JarFile jarFile;
>         if (jarFileConstructor != null) {
>           try {
>             jarFile = (JarFile) jarFileConstructor.newInstance(new File(filePath), Boolean.TRUE, ZipFile.OPEN_READ, baseVersion);
>           } catch (InvocationTargetException i) {
>             Throwable t = i.getCause();
>             if (t instanceof IOException)
>               throw (IOException) t;
>             throw new RuntimeException(t);
>           } catch (InstantiationException | IllegalAccessException | IllegalArgumentException e) {
>             throw new RuntimeException(e);
>           }
>         } else {
>           jarFile = new JarFile(filePath);
>         }

In our case when processing a MR jar, we assume that if a class is versioned, we will ignore all related classes (ordinary and inner).
We need to keep a list of all classes and not process any of them until we get the full list.
That is, we post process the class list so that we drop ordinary classes and inner classes if there is versioned replacement of any of them.
We avoid false failures by taking this approach (we ignore inner classes for older releases that reference a class that might no longer exist).

We know that this is not purely correct.  It's possible that the versioned replacement class references the non-versioned inner class. 
I think that the correct way would be to see if the inner class is referenced by any class file but that isn't doable so you will need to choose a heuristic.

In the case of annotation processing, it seems the heuristic should be to include the non-versioned inner class since it could be referenced by another class somewhere. That should be simple since you don't need to wait to process files.




-----Original Message-----
From: Greg Wilkins [mailto:gregw at webtide.com] 
Sent: Wednesday, September 13, 2017 6:12 PM
To: jigsaw-dev at openjdk.java.net
Subject: Scanning multi version jars?

I hope this is the right group for this question. please redirect me if not.

The Jetty project is trying to implement annotation scanning for multi version jars and have some concerns with some edge cases, specifically with inner classes.

A multi versioned jar might contain something like:

   - org/example/Foo.class
   - org/example/Foo$Bar.class
   - META-INF/versions/9/org/example/Foo.class

It is clear that there is a java 9 version of Foo.  But what is unclear is the inner class Foo$Bar?  Is that only used by the base Foo version? or does the java 9 version also use the Foo$Bar inner class, but it didn't use any java 9 features, so the base version is able to be used??

It is unclear from just an index of the jar if we should be scanning Foo$Bar for annotations.  So currently it appears that we have to actually scan the Foo class to see if Foo$Bar is referenced and only then scan Foo$Bar for annotations (and recursive analysis for any Foo$Bar$Bob class )!

An alternative would be if there was an enforced convention that any versioned class would also version all of it's inner classes - which may be a reasonable assumption given that they would be compiled together, but we see nothing in the specifications that force a jar to be assembled that way.

Any guidance anybody can give would be helpful.

cheers
Greg Wilkins

PS. The JarFile.getEntry method does not appear to respect it's javadoc with respect to multiversioned jars: it says it will do a search for the most recent version, however the code indicates that the search is only done if the base version does not exist.  This is kind of separate issue, but makes it difficult to defer the behaviour of what to scan to the implementation in JarFile






--
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From Alan.Bateman at oracle.com  Thu Sep 14 09:07:47 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 14 Sep 2017 10:07:47 +0100
Subject: Scanning multi version jars?
In-Reply-To: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
Message-ID: <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>



On 13/09/2017 23:12, Greg Wilkins wrote:
> I hope this is the right group for this question. please redirect me if not.
Probably core-libs-dev as this isn't anything to do with modules but in 
any case ...

>
> The Jetty project is trying to implement annotation scanning for multi
> version jars and have some concerns with some edge cases, specifically with
> inner classes.
>
> A multi versioned jar might contain something like:
>
>     - org/example/Foo.class
>     - org/example/Foo$Bar.class
>     - META-INF/versions/9/org/example/Foo.class
>
> It is clear that there is a java 9 version of Foo.  But what is unclear is
> the inner class Foo$Bar?  Is that only used by the base Foo version? or
> does the java 9 version also use the Foo$Bar inner class, but it didn't use
> any java 9 features, so the base version is able to be used??
>
> It is unclear from just an index of the jar if we should be
> scanning Foo$Bar for annotations.  So currently it appears that we have to
> actually scan the Foo class to see if Foo$Bar is referenced and only then
> scan Foo$Bar for annotations (and recursive analysis for any Foo$Bar$Bob
> class )!
Is Foo$Bar public and part of org.example's API? If so then I would 
expect compiling the 9 version of Foo.java will generate a class file 
for each of the inner classes and so the scenario shouldn't arise.? If 
Foo$Bar is not public (and so not part of org.example's API), and you 
scanning non-public classes, then it would need special handling and 
examination of the InnerClasses attribute in both classes. I wouldn't 
expect it will arise too often to cause a performance issue (assuming 
that is the concern).

>
> PS. The JarFile.getEntry method does not appear to respect it's javadoc
> with respect to multiversioned jars: it says it will do a search for the
> most recent version, however the code indicates that the search is only
> done if the base version does not exist.  This is kind of separate issue,
> but makes it difficult to defer the behaviour of what to scan to the
> implementation in JarFile
getEntry/getJarEntry will return a JarEntry if it is present in the base 
section or a versioned section (<= max version you specified when 
opening the JarFile) or both. I agree the javadoc is a bit confusing and 
could be improved but I assume you don't actually have an issue here as 
the JarEntry returned will locate the entry in the versioned section if 
it exists.

-Alan

From weijun.wang at oracle.com  Thu Sep 14 09:58:09 2017
From: weijun.wang at oracle.com (Weijun Wang)
Date: Thu, 14 Sep 2017 17:58:09 +0800
Subject: Scanning multi version jars?
In-Reply-To: <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
Message-ID: <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>


> On Sep 14, 2017, at 5:07 PM, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> 
> On 13/09/2017 23:12, Greg Wilkins wrote:
>> I hope this is the right group for this question. please redirect me if not.
> Probably core-libs-dev as this isn't anything to do with modules but in any case ...

A related question:

I know an MR jar allows you to shadow a class file with a release-specific one, but what if the new release has removed an old class? It will not appear in the release-specific directory but still exists in the root. Should we describe this in the MANIFEST?

Thanks
Max


From Alan.Bateman at oracle.com  Thu Sep 14 10:44:24 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 14 Sep 2017 11:44:24 +0100
Subject: Scanning multi version jars?
In-Reply-To: <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
Message-ID: <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>

On 14/09/2017 10:58, Weijun Wang wrote:
> :
> I know an MR jar allows you to shadow a class file with a release-specific one, but what if the new release has removed an old class? It will not appear in the release-specific directory but still exists in the root. Should we describe this in the MANIFEST?
>
A MR JAR is not intended to support multiple versions of the same 
library, instead the versioned sections are for classes that take 
advantage of newer language or API features. They help with the 
migration from using JDK internal APIs to supported/standard APIs for 
example. So I don't think it should be complicated by an additional list 
of entries to "hide" in the base or overlaid version sections.

Greg's mail doesn't say if Bar is public so I can't tell if his example 
involves an attempted API change or not. Assuming Bar is not public then 
compiling the 9 version of Foo.java will generate Foo.class and no 
Foo$Bar.class. This doesn't mean it's completely orphaned of course as 
there may be other classes in the base section, and in the same package, 
that were compiled with references to Bar. The `jar` tool could do some 
additional validation to catch these references and so avoid 
IncompatibleClassChangeError at runtime (as might arise if 
getEnclosingClass were invoked on the inner class). That would help with 
Greg's annotation scanning scenario too.

-Alan

From gregw at webtide.com  Fri Sep 15 02:09:06 2017
From: gregw at webtide.com (Greg Wilkins)
Date: Fri, 15 Sep 2017 12:09:06 +1000
Subject: Scanning multi version jars?
In-Reply-To: <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
Message-ID: <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>

Alan,

thanks for correcting me on the API of JarFile - I can see it kind of
works, but in a very bizarre way (it gives different content for entries
obtained via the enumerator vs the getJarEntry API, even though both
entries report the same name).  But I'll discuss that elsewhere.



The main issue still remains is that it is entirely unclear what files we
should scan.   I understand the nuanced point that you are trying to make,
ie "that it depends"  on if the class is public or private, if it is an API
change, if it is an alternate implementation rather than a new version of
the same library etc. etc.  I also totally understand that there are
intended uses and unintended uses for this feature.

However, as an implementer of an application container, it does not matter
if I understand the nuances of MR jars and intended usage.  What matters is
do the developers of the 3rd party jars that will be deployed in my
container understand those nuances?    We have to look at jars that are
supplied by third parties, with various levels of understanding, perhaps
with some tricky clever ideas how to mess with the system, and we have to
decide which classes we are going to scan for annotations.

This is NOT a performance issue.  It is a consistency/portability issue. We
have to make exactly the same decisions as all the other application
containers out there, else 3rd party library jars will act differently on
different containers.

Thus it looks like we need some kind of heuristic to guess what the 3rd
party developer intended when they used the MR feature.    Some approaches
will need us to scan all the outer and inner classes to determine if the
inner classes are referenced and if they are public or private.

The heuristic could then be to analyse an inner class IFF it is public and
referenced.    Or perhaps that should be if it is public OR referenced?

Alternately, can we just have an heuristic based only on the index.  If Foo
exists as a versioned class, then only similarly versioned Foo$Bar classes
should be scanned and base Foo$Bar classes will be ignored?

All of these are possible.  But we need an official documented (perhaps
tool enforced) policy so that all containers can implement the same
heuristic so that we can have portability.

Ideally, the containers would not need to implement this heuristic, as it
would be implemented in the enumerator of JarFile.  Unfortunately that is
not the case and the enumerator returns all the entries regardless of
version.   So containers must implement their own enumeration and we need
to make sure we all implement it the same!

regards





On 14 September 2017 at 20:44, Alan Bateman <Alan.Bateman at oracle.com> wrote:

> On 14/09/2017 10:58, Weijun Wang wrote:
>
>> :
>> I know an MR jar allows you to shadow a class file with a
>> release-specific one, but what if the new release has removed an old class?
>> It will not appear in the release-specific directory but still exists in
>> the root. Should we describe this in the MANIFEST?
>>
>> A MR JAR is not intended to support multiple versions of the same
> library, instead the versioned sections are for classes that take advantage
> of newer language or API features. They help with the migration from using
> JDK internal APIs to supported/standard APIs for example. So I don't think
> it should be complicated by an additional list of entries to "hide" in the
> base or overlaid version sections.
>
> Greg's mail doesn't say if Bar is public so I can't tell if his example
> involves an attempted API change or not. Assuming Bar is not public then
> compiling the 9 version of Foo.java will generate Foo.class and no
> Foo$Bar.class. This doesn't mean it's completely orphaned of course as
> there may be other classes in the base section, and in the same package,
> that were compiled with references to Bar. The `jar` tool could do some
> additional validation to catch these references and so avoid
> IncompatibleClassChangeError at runtime (as might arise if
> getEnclosingClass were invoked on the inner class). That would help with
> Greg's annotation scanning scenario too.
>
> -Alan
>



-- 
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From gregw at webtide.com  Fri Sep 15 04:01:41 2017
From: gregw at webtide.com (Greg Wilkins)
Date: Fri, 15 Sep 2017 14:01:41 +1000
Subject: Scanning modules?
Message-ID: <CAAPGdfELUwkW07QEz6vu79zZ4yGRwYsSvFpygfYsnhdgH_oPmA@mail.gmail.com>

All,
another question on scanning, but more jigsaw related than my other
question.

App containers have to scan jars deployed in EARs, WARs etc. for
annotations, fragments, resources etc.  However, many containers also offer
features to be able to optionally scan jars that are on the system
classpath.

For example, Jetty deploys the JSTL jar on the system classpath, but adds
it to the annotation scanning for every webapp deployed.  Similarly when
running in embedded mode, where all jars are on the system class path, we
have been able to find and scan those that matched a pattern.

However, with java9 modules, we have two problems if a jar like jstl.jar is
deployed on at runtime.

Firstly, the system loader is no longer a URLClassLoader, so we cannot scan
to list of provided jar files looking for potential matches of known jars
that we wish to scan.

We can work around that by loading a known class from the known jars and
asking for its location (via it's protection domain).   For a normal jar,
this gives us a file URI, which we can then use as the basis for a scan.

However, I expect that eventually JSTL will be converted to a module, and
then the location will just be something like jrt://apache.jstl.      Which
gives us our second problem - given a known or discovered module, how do we
discover what classes are within that module in order to scan them for
annotations?       We cannot find any API via Module nor the jrt URLHandler
that gives us access to a list of classes?

We can get a lists/sets of Packages,Exports etc. but nothing we can see
gives us the actual classes/resources contained in that module, nor the
location of the source jar file that we could open and thus determine the
contents ourselves.

So some guidance of how we are meant to do discovery of annotations,
fragments etc in a modularized deployment would be very useful.

regards


-- 
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From sormuras at gmail.com  Fri Sep 15 07:00:58 2017
From: sormuras at gmail.com (Christian Stein)
Date: Fri, 15 Sep 2017 09:00:58 +0200
Subject: Scanning modules?
In-Reply-To: <CAAPGdfELUwkW07QEz6vu79zZ4yGRwYsSvFpygfYsnhdgH_oPmA@mail.gmail.com>
References: <CAAPGdfELUwkW07QEz6vu79zZ4yGRwYsSvFpygfYsnhdgH_oPmA@mail.gmail.com>
Message-ID: <CAJi8+g49ibGhw94U18TRCK8hj+S7kZ7NCRAp147+knTJCYOaNg@mail.gmail.com>

Hi Greg,

short disclaimer: I'm learning and testing the "--scan-module" magic at
the moment to implement a similar feature in JUnit 5. See [1] for details.
If I'm talking non-sense, please correct me jigsaw-devs.

To scan the contents of a module you'll need a ModuleReference[2] instance.
You may open() such a reference to obtain a ModuleReader[3] which allows
you to list() all or find() some of the contents of the referenced module.

How to get ModuleReference instances?

There're at least two ways: either ask the runtime to report all resolved
modules in a specific layer. See [4] for details and [5] how I try to obtain
all modules on the module-path.

Or you may build your own ModuleFinder [6] with well-known module locations.
Here [7] is an example how this could work.

Cheers and hope that helps,
Christian

[1] https://github.com/junit-team/junit5/pull/1057/files#diff-
96762765b7aa19e415e25de774366feb
[2] http://download.java.net/java/jdk9/docs/api/java/lang/
module/ModuleReference.html
[3] http://download.java.net/java/jdk9/docs/api/java/lang/
module/ModuleReader.html
[4] http://download.java.net/java/jdk9/docs/api/java/lang/ModuleLayer.html
[5]
https://github.com/junit-team/junit5/pull/1057/files#diff-6e80f389aa0b4148904a9c474b9e8fb8R43
[6] http://download.java.net/java/jdk9/docs/api/java/lang/
module/ModuleFinder.html
[7]
https://github.com/forax/pro/blob/master/src/main/java/com.github.forax.pro.helper/com/github/forax/pro/helper/ModuleHelper.java#L77

On Fri, Sep 15, 2017 at 6:01 AM, Greg Wilkins <gregw at webtide.com> wrote:

> All,
> another question on scanning, but more jigsaw related than my other
> question.
>
> App containers have to scan jars deployed in EARs, WARs etc. for
> annotations, fragments, resources etc.  However, many containers also offer
> features to be able to optionally scan jars that are on the system
> classpath.
>
> For example, Jetty deploys the JSTL jar on the system classpath, but adds
> it to the annotation scanning for every webapp deployed.  Similarly when
> running in embedded mode, where all jars are on the system class path, we
> have been able to find and scan those that matched a pattern.
>
> However, with java9 modules, we have two problems if a jar like jstl.jar is
> deployed on at runtime.
>
> Firstly, the system loader is no longer a URLClassLoader, so we cannot scan
> to list of provided jar files looking for potential matches of known jars
> that we wish to scan.
>
> We can work around that by loading a known class from the known jars and
> asking for its location (via it's protection domain).   For a normal jar,
> this gives us a file URI, which we can then use as the basis for a scan.
>
> However, I expect that eventually JSTL will be converted to a module, and
> then the location will just be something like jrt://apache.jstl.      Which
> gives us our second problem - given a known or discovered module, how do we
> discover what classes are within that module in order to scan them for
> annotations?       We cannot find any API via Module nor the jrt URLHandler
> that gives us access to a list of classes?
>
> We can get a lists/sets of Packages,Exports etc. but nothing we can see
> gives us the actual classes/resources contained in that module, nor the
> location of the source jar file that we could open and thus determine the
> contents ourselves.
>
> So some guidance of how we are meant to do discovery of annotations,
> fragments etc in a modularized deployment would be very useful.
>
> regards
>
>
> --
> Greg Wilkins <gregw at webtide.com> CTO http://webtide.com
>

From Alan.Bateman at oracle.com  Fri Sep 15 07:15:12 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 15 Sep 2017 08:15:12 +0100
Subject: Scanning modules?
In-Reply-To: <CAAPGdfELUwkW07QEz6vu79zZ4yGRwYsSvFpygfYsnhdgH_oPmA@mail.gmail.com>
References: <CAAPGdfELUwkW07QEz6vu79zZ4yGRwYsSvFpygfYsnhdgH_oPmA@mail.gmail.com>
Message-ID: <90300f2c-7b23-aed8-3e89-c8cbc0614e96@oracle.com>

On 15/09/2017 05:01, Greg Wilkins wrote:
> All,
> another question on scanning, but more jigsaw related than my other
> question.
>
> App containers have to scan jars deployed in EARs, WARs etc. for
> annotations, fragments, resources etc.  However, many containers also offer
> features to be able to optionally scan jars that are on the system
> classpath.
>
> For example, Jetty deploys the JSTL jar on the system classpath, but adds
> it to the annotation scanning for every webapp deployed.  Similarly when
> running in embedded mode, where all jars are on the system class path, we
> have been able to find and scan those that matched a pattern.
>
> However, with java9 modules, we have two problems if a jar like jstl.jar is
> deployed on at runtime.
>
> Firstly, the system loader is no longer a URLClassLoader, so we cannot scan
> to list of provided jar files looking for potential matches of known jars
> that we wish to scan.
>
> We can work around that by loading a known class from the known jars and
> asking for its location (via it's protection domain).   For a normal jar,
> this gives us a file URI, which we can then use as the basis for a scan.
>
> However, I expect that eventually JSTL will be converted to a module, and
> then the location will just be something like jrt://apache.jstl.      Which
> gives us our second problem - given a known or discovered module, how do we
> discover what classes are within that module in order to scan them for
> annotations?       We cannot find any API via Module nor the jrt URLHandler
> that gives us access to a list of classes?
>
> We can get a lists/sets of Packages,Exports etc. but nothing we can see
> gives us the actual classes/resources contained in that module, nor the
> location of the source jar file that we could open and thus determine the
> contents ourselves.
>
> So some guidance of how we are meant to do discovery of annotations,
> fragments etc in a modularized deployment would be very useful.
>
For the class path scanning then I assume you can use the 
java.class.path property, the type of the application or system class 
loader should not be interesting.

For the module path or other locations containing packaged modules then 
you create a ModuleFinder and invoke its findAll method to get a module 
reference to every module. Then you can use a ModuleReader to open the 
module. ModuleReader defines `list` and other methods to access its 
resources. My guess is this is where you will end up in that I suspect 
you are selecting the candidate modules to resolve and these modules 
will be instantiated in a child layer.

If you are really just looking to get at the contents of the modules in 
the boot layer (the modules that are resolved at startup) then you'll do 
something like this:

 ???? ModuleLayer.boot().configuration().modules().stream()
 ??????????? .map(ResolvedModule::reference)
 ??????????? .forEach(mref -> {
 ??????????????? System.out.println(mref.descriptor().name());
 ??????????????? try (ModuleReader reader = mref.open()) {
 ??????????????????? reader.list().forEach(System.out::println);
 ??????????????? } catch (IOException ioe) {
 ??????????????????? throw new UncheckedIOException(ioe);
 ??????????????? }
 ??????????? });

This code fragment processes the configuration for the boot layer, 
opening up each module and listing the names of the resources in every 
module.

You mention the jrt protocol handler. That is to support URL 
connections. There is also a jrt file system provider for tools that 
need to scan the contents of the current or other run-time image. I 
suspect you won't need this for what you are doing.

-Alan.


From Alan.Bateman at oracle.com  Fri Sep 15 07:27:52 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 15 Sep 2017 08:27:52 +0100
Subject: Scanning multi version jars?
In-Reply-To: <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
Message-ID: <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>



On 15/09/2017 03:09, Greg Wilkins wrote:
>
> Alan,
>
> thanks for correcting me on the API of JarFile - I can see it kind of 
> works, but in a very bizarre way (it gives different content for 
> entries obtained via the enumerator vs the getJarEntry API, even 
> though both entries report the same name).? But I'll discuss that 
> elsewhere.
This is something that was discussed on core-libs-dev on a number of 
occasions. The summary is that JarFile needs a new API for this, 
versionedStream() was suggested, but it was kicked down the road for 
later in order deal with the fallout from adding MR JARs.

Since you have access to the code then look at 
jdk.internal.util.jar.VersionedStream for an example code of what I 
think you are looking for.

-Alan

From gregw at webtide.com  Fri Sep 15 07:40:41 2017
From: gregw at webtide.com (Greg Wilkins)
Date: Fri, 15 Sep 2017 17:40:41 +1000
Subject: Scanning modules?
In-Reply-To: <90300f2c-7b23-aed8-3e89-c8cbc0614e96@oracle.com>
References: <CAAPGdfELUwkW07QEz6vu79zZ4yGRwYsSvFpygfYsnhdgH_oPmA@mail.gmail.com>
 <90300f2c-7b23-aed8-3e89-c8cbc0614e96@oracle.com>
Message-ID: <CAAPGdfEKhep0FfigVbfw+Zim+KbvJVBRC0OPngAFx271u07pHg@mail.gmail.com>

Alan & Christian,

thanks for that - I totally missed ResolvedModule.open().list()

So now a supplementary question... are ModuleReaders MR aware?  Ie if the
jar of the module is MR, will the resolved module only contain classes that
are appropriate for the current Runtime?

thanks in advance.

cheers


On 15 September 2017 at 17:15, Alan Bateman <Alan.Bateman at oracle.com> wrote:

> On 15/09/2017 05:01, Greg Wilkins wrote:
>
>> All,
>> another question on scanning, but more jigsaw related than my other
>> question.
>>
>> App containers have to scan jars deployed in EARs, WARs etc. for
>> annotations, fragments, resources etc.  However, many containers also
>> offer
>> features to be able to optionally scan jars that are on the system
>> classpath.
>>
>> For example, Jetty deploys the JSTL jar on the system classpath, but adds
>> it to the annotation scanning for every webapp deployed.  Similarly when
>> running in embedded mode, where all jars are on the system class path, we
>> have been able to find and scan those that matched a pattern.
>>
>> However, with java9 modules, we have two problems if a jar like jstl.jar
>> is
>> deployed on at runtime.
>>
>> Firstly, the system loader is no longer a URLClassLoader, so we cannot
>> scan
>> to list of provided jar files looking for potential matches of known jars
>> that we wish to scan.
>>
>> We can work around that by loading a known class from the known jars and
>> asking for its location (via it's protection domain).   For a normal jar,
>> this gives us a file URI, which we can then use as the basis for a scan.
>>
>> However, I expect that eventually JSTL will be converted to a module, and
>> then the location will just be something like jrt://apache.jstl.
>> Which
>> gives us our second problem - given a known or discovered module, how do
>> we
>> discover what classes are within that module in order to scan them for
>> annotations?       We cannot find any API via Module nor the jrt
>> URLHandler
>> that gives us access to a list of classes?
>>
>> We can get a lists/sets of Packages,Exports etc. but nothing we can see
>> gives us the actual classes/resources contained in that module, nor the
>> location of the source jar file that we could open and thus determine the
>> contents ourselves.
>>
>> So some guidance of how we are meant to do discovery of annotations,
>> fragments etc in a modularized deployment would be very useful.
>>
>> For the class path scanning then I assume you can use the java.class.path
> property, the type of the application or system class loader should not be
> interesting.
>
> For the module path or other locations containing packaged modules then
> you create a ModuleFinder and invoke its findAll method to get a module
> reference to every module. Then you can use a ModuleReader to open the
> module. ModuleReader defines `list` and other methods to access its
> resources. My guess is this is where you will end up in that I suspect you
> are selecting the candidate modules to resolve and these modules will be
> instantiated in a child layer.
>
> If you are really just looking to get at the contents of the modules in
> the boot layer (the modules that are resolved at startup) then you'll do
> something like this:
>
>      ModuleLayer.boot().configuration().modules().stream()
>             .map(ResolvedModule::reference)
>             .forEach(mref -> {
>                 System.out.println(mref.descriptor().name());
>                 try (ModuleReader reader = mref.open()) {
>                     reader.list().forEach(System.out::println);
>                 } catch (IOException ioe) {
>                     throw new UncheckedIOException(ioe);
>                 }
>             });
>
> This code fragment processes the configuration for the boot layer, opening
> up each module and listing the names of the resources in every module.
>
> You mention the jrt protocol handler. That is to support URL connections.
> There is also a jrt file system provider for tools that need to scan the
> contents of the current or other run-time image. I suspect you won't need
> this for what you are doing.
>
> -Alan.
>
>


-- 
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From Alan.Bateman at oracle.com  Fri Sep 15 08:26:03 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 15 Sep 2017 09:26:03 +0100
Subject: Scanning modules?
In-Reply-To: <CAAPGdfEKhep0FfigVbfw+Zim+KbvJVBRC0OPngAFx271u07pHg@mail.gmail.com>
References: <CAAPGdfELUwkW07QEz6vu79zZ4yGRwYsSvFpygfYsnhdgH_oPmA@mail.gmail.com>
 <90300f2c-7b23-aed8-3e89-c8cbc0614e96@oracle.com>
 <CAAPGdfEKhep0FfigVbfw+Zim+KbvJVBRC0OPngAFx271u07pHg@mail.gmail.com>
Message-ID: <e7a55d86-3f32-959f-6ab5-99044e35ee6b@oracle.com>

On 15/09/2017 08:40, Greg Wilkins wrote:
> Alan & Christian,
>
> thanks for that - I totally missed ResolvedModule.open().list()
>
> So now a supplementary question... are ModuleReaders MR aware?? Ie if 
> the jar of the module is MR, will the resolved module only contain 
> classes that are appropriate for the current Runtime?
>
Yes.

From alexander.udalov at jetbrains.com  Fri Sep 15 12:14:21 2017
From: alexander.udalov at jetbrains.com (Alexander Udalov)
Date: Fri, 15 Sep 2017 15:14:21 +0300
Subject: "exports" directive does not open module's resources for dependent
 modules
Message-ID: <CAPMW=6X3REeduurBZ=KN5kfEYo9Yunopa=TrO93+_ysooybiJA@mail.gmail.com>

It looks like exporting a package from a (non-open) module with an
"exports" directive is not enough to ensure that
Module.getResourceAsStream on that module would load resources from
the module. Surprisingly, adding an "opens" directive for the same
package to the module declaration allows resources to be found.

I've put up a simple project to demonstrate this issue:
https://github.com/udalov/jigsaw-resources-are-not-exported

I'm wondering whether this is a bug or expected behavior? I would
expect that exporting a package would also open it not only for uses
of reflection, but for resource loading. I can't find anything related
to resources in the JLS, but p.7.7 "Module Declarations" gives a clear
impression that the "exports" directive gives a superset of
functionality of the "opens" directive in this regard.

Looking at the implementation of Module.getResourceAsStream, I see
that it calls Module.isOpen(String, Module) which returns false in
case the package name is exported by an "exports" directive. Which
raises another question: is Module.isOpen supposed to return whether
the package has been explicitly opened by an "opens" directive (or via
being in an open module)? Or is it supposed to also return true if the
package has been opened as a consequence of the fact that it's
exported (e.g. by an "exports" directive)? If the former is true, I
suppose there may be other usages of Module.isOpen, including in the
JDK, which (incorrectly) do not also check if the package name is
exported with Module.isExported.

The real-world case where I met this issue is in kotlin.stdlib where
we store metadata of Kotlin declarations in resource files per package
(only of those declarations which are not representable by JVM class
files), and kotlin.reflect (which is another module) reaches out to
those resource files in kotlin.stdlib. To make them reachable thus
along with every "exports XXX" in kotlin.stdlib's module declaration
I'm adding "opens XXX to kotlin.reflect", which looks cumbersome.

Thanks!

Alexander

From stephen.felts at oracle.com  Fri Sep 15 12:32:25 2017
From: stephen.felts at oracle.com (Stephen Felts)
Date: Fri, 15 Sep 2017 12:32:25 +0000 (UTC)
Subject: "exports" directive does not open module's resources for
 dependent modules
In-Reply-To: <CAPMW=6X3REeduurBZ=KN5kfEYo9Yunopa=TrO93+_ysooybiJA@mail.gmail.com>
References: <CAPMW=6X3REeduurBZ=KN5kfEYo9Yunopa=TrO93+_ysooybiJA@mail.gmail.com>
Message-ID: <5ad05adc-79e0-4b76-82f0-e73012e91c24@default>

"opens" is a superset of "exports".


-----Original Message-----
From: Alexander Udalov [mailto:alexander.udalov at jetbrains.com] 
Sent: Friday, September 15, 2017 8:14 AM
To: jigsaw-dev <jigsaw-dev at openjdk.java.net>
Subject: "exports" directive does not open module's resources for dependent modules

It looks like exporting a package from a (non-open) module with an "exports" directive is not enough to ensure that Module.getResourceAsStream on that module would load resources from the module. Surprisingly, adding an "opens" directive for the same package to the module declaration allows resources to be found.

I've put up a simple project to demonstrate this issue:
https://github.com/udalov/jigsaw-resources-are-not-exported

I'm wondering whether this is a bug or expected behavior? I would expect that exporting a package would also open it not only for uses of reflection, but for resource loading. I can't find anything related to resources in the JLS, but p.7.7 "Module Declarations" gives a clear impression that the "exports" directive gives a superset of functionality of the "opens" directive in this regard.

Looking at the implementation of Module.getResourceAsStream, I see that it calls Module.isOpen(String, Module) which returns false in case the package name is exported by an "exports" directive. Which raises another question: is Module.isOpen supposed to return whether the package has been explicitly opened by an "opens" directive (or via being in an open module)? Or is it supposed to also return true if the package has been opened as a consequence of the fact that it's exported (e.g. by an "exports" directive)? If the former is true, I suppose there may be other usages of Module.isOpen, including in the JDK, which (incorrectly) do not also check if the package name is exported with Module.isExported.

The real-world case where I met this issue is in kotlin.stdlib where we store metadata of Kotlin declarations in resource files per package (only of those declarations which are not representable by JVM class files), and kotlin.reflect (which is another module) reaches out to those resource files in kotlin.stdlib. To make them reachable thus along with every "exports XXX" in kotlin.stdlib's module declaration I'm adding "opens XXX to kotlin.reflect", which looks cumbersome.

Thanks!

Alexander

From Alan.Bateman at oracle.com  Fri Sep 15 12:52:50 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 15 Sep 2017 13:52:50 +0100
Subject: "exports" directive does not open module's resources for
 dependent modules
In-Reply-To: <CAPMW=6X3REeduurBZ=KN5kfEYo9Yunopa=TrO93+_ysooybiJA@mail.gmail.com>
References: <CAPMW=6X3REeduurBZ=KN5kfEYo9Yunopa=TrO93+_ysooybiJA@mail.gmail.com>
Message-ID: <346543f6-a46a-ae1a-9a2a-1186a296d96b@oracle.com>

On 15/09/2017 13:14, Alexander Udalov wrote:
> It looks like exporting a package from a (non-open) module with an
> "exports" directive is not enough to ensure that
> Module.getResourceAsStream on that module would load resources from
> the module. Surprisingly, adding an "opens" directive for the same
> package to the module declaration allows resources to be found.
>
> I've put up a simple project to demonstrate this issue:
> https://github.com/udalov/jigsaw-resources-are-not-exported
>
> I'm wondering whether this is a bug or expected behavior?
Yes, this is expected behavior. The javadoc has all details. In your 
example, if kotlin.reflect is using Class.getResourceXXX or 
Module.getResourceAsStream to locate a non .class resource in module X 
then module X needs to open the package with the resource to at least 
kotlin.reflect.


> I would
> expect that exporting a package would also open it not only for uses
> of reflection, but for resource loading. I can't find anything related
> to resources in the JLS, but p.7.7 "Module Declarations" gives a clear
> impression that the "exports" directive gives a superset of
> functionality of the "opens" directive in this regard.
I think you need to re-read JLS 7.7.2. As `opens` doesn't grant any 
access at compile-time then you think of it as being a subset of 
`exports` in this phase.? On the other hand, `opens` grants access to 
the public classes/members and additionally reflective access to all 
members of all classes at run-time so you can think of it as a superset 
in this phase. The resource APIs just build on this.

-Alan

From sander.mak at luminis.eu  Fri Sep 15 12:55:15 2017
From: sander.mak at luminis.eu (Sander Mak)
Date: Fri, 15 Sep 2017 12:55:15 +0000
Subject: "exports" directive does not open module's resources for
 dependent modules
In-Reply-To: <5ad05adc-79e0-4b76-82f0-e73012e91c24@default>
References: <CAPMW=6X3REeduurBZ=KN5kfEYo9Yunopa=TrO93+_ysooybiJA@mail.gmail.com>
 <5ad05adc-79e0-4b76-82f0-e73012e91c24@default>
Message-ID: <69B0B898-8C8E-48AB-9B35-EE558CAFA01F@luminis.eu>


> On 15 Sep 2017, at 14:32, Stephen Felts <stephen.felts at oracle.com> wrote:
> 
> "opens" is a superset of "exports".

Only when qualified with 'at run-time'.


Sander

From alexander.udalov at jetbrains.com  Fri Sep 15 13:17:23 2017
From: alexander.udalov at jetbrains.com (Alexander Udalov)
Date: Fri, 15 Sep 2017 16:17:23 +0300
Subject: "exports" directive does not open module's resources for
 dependent modules
In-Reply-To: <346543f6-a46a-ae1a-9a2a-1186a296d96b@oracle.com>
References: <CAPMW=6X3REeduurBZ=KN5kfEYo9Yunopa=TrO93+_ysooybiJA@mail.gmail.com>
 <346543f6-a46a-ae1a-9a2a-1186a296d96b@oracle.com>
Message-ID: <CAPMW=6Xy4e-4s7OZvQHXouaqAkO_tnp0hbU1TAapdmjRaNC3dg@mail.gmail.com>

Thank you, I've re-read JLS 7.7.2 and understand now that it behaves
as expected. I'll continue exporting and opening every package then.

Alexander

On Fri, Sep 15, 2017 at 3:52 PM, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> On 15/09/2017 13:14, Alexander Udalov wrote:
>>
>> It looks like exporting a package from a (non-open) module with an
>> "exports" directive is not enough to ensure that
>> Module.getResourceAsStream on that module would load resources from
>> the module. Surprisingly, adding an "opens" directive for the same
>> package to the module declaration allows resources to be found.
>>
>> I've put up a simple project to demonstrate this issue:
>> https://github.com/udalov/jigsaw-resources-are-not-exported
>>
>> I'm wondering whether this is a bug or expected behavior?
>
> Yes, this is expected behavior. The javadoc has all details. In your
> example, if kotlin.reflect is using Class.getResourceXXX or
> Module.getResourceAsStream to locate a non .class resource in module X then
> module X needs to open the package with the resource to at least
> kotlin.reflect.
>
>
>> I would
>> expect that exporting a package would also open it not only for uses
>> of reflection, but for resource loading. I can't find anything related
>> to resources in the JLS, but p.7.7 "Module Declarations" gives a clear
>> impression that the "exports" directive gives a superset of
>> functionality of the "opens" directive in this regard.
>
> I think you need to re-read JLS 7.7.2. As `opens` doesn't grant any access
> at compile-time then you think of it as being a subset of `exports` in this
> phase.  On the other hand, `opens` grants access to the public
> classes/members and additionally reflective access to all members of all
> classes at run-time so you can think of it as a superset in this phase. The
> resource APIs just build on this.
>
> -Alan

From sander.mak at luminis.eu  Fri Sep 15 13:29:03 2017
From: sander.mak at luminis.eu (Sander Mak)
Date: Fri, 15 Sep 2017 13:29:03 +0000
Subject: "exports" directive does not open module's resources for
 dependent modules
In-Reply-To: <CAPMW=6Xy4e-4s7OZvQHXouaqAkO_tnp0hbU1TAapdmjRaNC3dg@mail.gmail.com>
References: <CAPMW=6X3REeduurBZ=KN5kfEYo9Yunopa=TrO93+_ysooybiJA@mail.gmail.com>
 <346543f6-a46a-ae1a-9a2a-1186a296d96b@oracle.com>
 <CAPMW=6Xy4e-4s7OZvQHXouaqAkO_tnp0hbU1TAapdmjRaNC3dg@mail.gmail.com>
Message-ID: <07848A07-C040-4C30-9CA2-C0F4FCA5906C@luminis.eu>


> On 15 Sep 2017, at 15:17, Alexander Udalov <alexander.udalov at jetbrains.com> wrote:
> 
> Thank you, I've re-read JLS 7.7.2 and understand now that it behaves
> as expected. I'll continue exporting and opening every package then.

If you open *every* package, you could create an open module as well. Might be less annoying, at the risk of accidentally opening new packages added latre to the module as well even though they might not need to be open.


Sander


From gregw at webtide.com  Fri Sep 15 21:58:36 2017
From: gregw at webtide.com (Greg Wilkins)
Date: Sat, 16 Sep 2017 07:58:36 +1000
Subject: Scanning multi version jars?
In-Reply-To: <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
Message-ID: <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>

Alan,

I had a quick look at `jdk.internal.util.jar.VersionedStream` and have the
following comments:

   - The style of the API is fine - pass in a JarFile and get a
   Stream<JarEntry>.
   - It might be better to have a Stream<VersionedJarEntry> which includes
   a method to query the actual version of each entry.
   - I think the stream needs to handle inner classes and only include them
   if their matching outerclass is available at the same version.  So for
   example a base Foo$Bar.class will only be included if the stream includes a
   base Foo.class, and it will not be included if the Foo.class is version 9
   or above.  Likewise a version 9 Foo$Bar.class will only be included in the
   stream if the stream also includes a version 9 Foo.class, and will not  be
   included if the stream has a version 10 or above Foo.class

If you think this last point is possible, then I'll move the discussion
back the EE expert groups to try to get an agreement on the exact stream
code that will be used in the mid term until it is available in the JRE
lib, at which time the specs should be amended to say they will defer the
decision of which classes to scan the JRE lib so they will be future proof
for any changes in java 10, 11 etc.

cheers




On 15 September 2017 at 17:27, Alan Bateman <Alan.Bateman at oracle.com> wrote:

>
>
> On 15/09/2017 03:09, Greg Wilkins wrote:
>
>>
>> Alan,
>>
>> thanks for correcting me on the API of JarFile - I can see it kind of
>> works, but in a very bizarre way (it gives different content for entries
>> obtained via the enumerator vs the getJarEntry API, even though both
>> entries report the same name).  But I'll discuss that elsewhere.
>>
> This is something that was discussed on core-libs-dev on a number of
> occasions. The summary is that JarFile needs a new API for this,
> versionedStream() was suggested, but it was kicked down the road for later
> in order deal with the fallout from adding MR JARs.
>
> Since you have access to the code then look at
> jdk.internal.util.jar.VersionedStream for an example code of what I think
> you are looking for.
>
> -Alan
>



-- 
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From stephen.felts at oracle.com  Sat Sep 16 00:29:03 2017
From: stephen.felts at oracle.com (Stephen Felts)
Date: Sat, 16 Sep 2017 00:29:03 +0000 (UTC)
Subject: Scanning multi version jars?
In-Reply-To: <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
 <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
Message-ID: <6e97133d-fa73-4098-bc93-8c1126debff2@default>

FWIW I tracked down the MR jar file that I was having trouble with.  It's the stand-alone JAXWS jar file com.sun.xml.ws.jaxws-rt.jar.
Focusing on the problem class, the jar contains
com/sun/xml/ws/util/xml/XmlCatalogUtil$1.class
com/sun/xml/ws/util/xml/XmlCatalogUtil.class
META-INF/versions/9/com/sun/xml/ws/util/xml/XmlCatalogUtil.class

The inner class XmlCatalogUtil$1.class is not used by the JDK9 version of the outer class.  Further, XmlCatalogUtil$1.class has class/method references that will not be resolved on JDK9.
If the stream includes the inner class, it's possible that whatever is processing it will fail.

In the use case that I had for processing jar files, we generated a collection of all class names in the jar file, and then removed class file names as Greg proposed. 
In the above example, com/sun/xml/ws/util/xml/XmlCatalogUtil$1.class and com/sun/xml/ws/util/xml/XmlCatalogUtil.class are removed.
Processing of the class files can only proceed after we generate/modify the entire list for the jar.
It ignores the possibility that a class outside the jar could be referencing something in the inner class that is public.


-----Original Message-----
From: Greg Wilkins [mailto:gregw at webtide.com] 
Sent: Friday, September 15, 2017 5:59 PM
To: Alan Bateman <alan.bateman at oracle.com>
Cc: jigsaw-dev <jigsaw-dev at openjdk.java.net>; core-libs-dev at openjdk.java.net
Subject: Re: Scanning multi version jars?

Alan,

I had a quick look at `jdk.internal.util.jar.VersionedStream` and have the following comments:

   - The style of the API is fine - pass in a JarFile and get a
   Stream<JarEntry>.
   - It might be better to have a Stream<VersionedJarEntry> which includes
   a method to query the actual version of each entry.
   - I think the stream needs to handle inner classes and only include them
   if their matching outerclass is available at the same version.  So for
   example a base Foo$Bar.class will only be included if the stream includes a
   base Foo.class, and it will not be included if the Foo.class is version 9
   or above.  Likewise a version 9 Foo$Bar.class will only be included in the
   stream if the stream also includes a version 9 Foo.class, and will not  be
   included if the stream has a version 10 or above Foo.class

If you think this last point is possible, then I'll move the discussion back the EE expert groups to try to get an agreement on the exact stream code that will be used in the mid term until it is available in the JRE lib, at which time the specs should be amended to say they will defer the decision of which classes to scan the JRE lib so they will be future proof for any changes in java 10, 11 etc.

cheers




On 15 September 2017 at 17:27, Alan Bateman <Alan.Bateman at oracle.com> wrote:

>
>
> On 15/09/2017 03:09, Greg Wilkins wrote:
>
>>
>> Alan,
>>
>> thanks for correcting me on the API of JarFile - I can see it kind of 
>> works, but in a very bizarre way (it gives different content for 
>> entries obtained via the enumerator vs the getJarEntry API, even 
>> though both entries report the same name).  But I'll discuss that elsewhere.
>>
> This is something that was discussed on core-libs-dev on a number of 
> occasions. The summary is that JarFile needs a new API for this,
> versionedStream() was suggested, but it was kicked down the road for 
> later in order deal with the fallout from adding MR JARs.
>
> Since you have access to the code then look at 
> jdk.internal.util.jar.VersionedStream for an example code of what I 
> think you are looking for.
>
> -Alan
>



--
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From gregw at webtide.com  Sat Sep 16 06:39:09 2017
From: gregw at webtide.com (Greg Wilkins)
Date: Sat, 16 Sep 2017 16:39:09 +1000
Subject: Scanning multi version jars?
In-Reply-To: <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
 <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
Message-ID: <CAAPGdfF8oVv9QBsgpc+eVXrZzrAnyL3+=ZUmfXfLRbZ8Hz6xHg@mail.gmail.com>

Alen et al,

here is a VersionedJarFile implementation that filters out inappropriate
inner classes:

https://gist.github.com/gregw/8f305e369d0b769e9c3fe791a0634b13

cheers

On 16 September 2017 at 07:58, Greg Wilkins <gregw at webtide.com> wrote:

>
> Alan,
>
> I had a quick look at `jdk.internal.util.jar.VersionedStream` and have
> the following comments:
>
>    - The style of the API is fine - pass in a JarFile and get a
>    Stream<JarEntry>.
>    - It might be better to have a Stream<VersionedJarEntry> which
>    includes a method to query the actual version of each entry.
>    - I think the stream needs to handle inner classes and only include
>    them if their matching outerclass is available at the same version.  So for
>    example a base Foo$Bar.class will only be included if the stream includes a
>    base Foo.class, and it will not be included if the Foo.class is version 9
>    or above.  Likewise a version 9 Foo$Bar.class will only be included in the
>    stream if the stream also includes a version 9 Foo.class, and will not  be
>    included if the stream has a version 10 or above Foo.class
>
> If you think this last point is possible, then I'll move the discussion
> back the EE expert groups to try to get an agreement on the exact stream
> code that will be used in the mid term until it is available in the JRE
> lib, at which time the specs should be amended to say they will defer the
> decision of which classes to scan the JRE lib so they will be future proof
> for any changes in java 10, 11 etc.
>
> cheers
>
>
>
>
> On 15 September 2017 at 17:27, Alan Bateman <Alan.Bateman at oracle.com>
> wrote:
>
>>
>>
>> On 15/09/2017 03:09, Greg Wilkins wrote:
>>
>>>
>>> Alan,
>>>
>>> thanks for correcting me on the API of JarFile - I can see it kind of
>>> works, but in a very bizarre way (it gives different content for entries
>>> obtained via the enumerator vs the getJarEntry API, even though both
>>> entries report the same name).  But I'll discuss that elsewhere.
>>>
>> This is something that was discussed on core-libs-dev on a number of
>> occasions. The summary is that JarFile needs a new API for this,
>> versionedStream() was suggested, but it was kicked down the road for later
>> in order deal with the fallout from adding MR JARs.
>>
>> Since you have access to the code then look at
>> jdk.internal.util.jar.VersionedStream for an example code of what I
>> think you are looking for.
>>
>> -Alan
>>
>
>
>
> --
> Greg Wilkins <gregw at webtide.com> CTO http://webtide.com
>



-- 
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From Alan.Bateman at oracle.com  Sun Sep 17 19:27:47 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Sun, 17 Sep 2017 20:27:47 +0100
Subject: Scanning multi version jars?
In-Reply-To: <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
 <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
Message-ID: <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>

On 15/09/2017 22:58, Greg Wilkins wrote:
> :
>
>   * I think the stream needs to handle inner classes and only include
>     them if their matching outerclass is available at the same
>     version.? So for example a base Foo$Bar.class will only be
>     included if the stream includes a base Foo.class, and it will not
>     be included if the Foo.class is version 9 or above.? Likewise a
>     version 9 Foo$Bar.class will only be included in the stream if the
>     stream also includes a version 9 Foo.class, and will not be
>     included if the stream has a version 10 or above Foo.class
>
> If you think this last point is possible, then I'll move the 
> discussion back the EE expert groups to try to get an agreement on the 
> exact stream code that will be used in the mid term until it is 
> available in the JRE lib, at which time the specs should be amended to 
> say they will defer the decision of which classes to scan the JRE lib 
> so they will be future proof for any changes in java 10, 11 etc.
>
I don't think this should be pushed down to the JarFile API. The JarFile 
API provides the base API for accessing JAR files and should not be 
concerned with the semantics or relationship between entries. I agree 
that annotation scanning tools and libraries need to do additional work 
to deal with orphaned or menacing inner classes in a MR JAR but it's not 
too different to arranging a class path with a JAR file containing the 
"classes for JDK 9" ahead of a JAR file containing the version of the 
library that runs on JDK 8. I do think that further checks could be done 
by the `jar` tool to identify issues at packaging time.

-Alan

From gregw at webtide.com  Mon Sep 18 00:30:55 2017
From: gregw at webtide.com (Greg Wilkins)
Date: Mon, 18 Sep 2017 10:30:55 +1000
Subject: Scanning multi version jars?
In-Reply-To: <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
 <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
 <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>
Message-ID: <CAAPGdfGPEH5wAu2r7YVY9AZOa5P3pfye+AKF+mctpK2bNupQKg@mail.gmail.com>

Alan,

I can sympathise somewhat with that point of view, however the counter
point is that the semantics of MR jars is something that has come from a
new feature released by the JVM that was perhaps a little under specified
with regards to inner classes - probably because the JVM itself does not
need to interpret that semantic when classloader and it is only an issue
when scanning is involved.

Now that the feature has been released and is being used, it is a bit of a
tall ask to expect the disparate tool vendors, spec groups, container
implementors and general developers to come up with a common semantic
interpretation of MR jars that contain inner classes.  Specially with the
EE spec groups a bit pre-occupied with their move to eclipse.

I think my own interpretation is common sense and I'm advocating it to be
adopted by the servlet specification. But who is to say that the CDI groups
might disagree and come up with another interpretation (it's not like the
two groups can even decide on how to interpret @Resource the same way :)

If core-libs were to signal their intention to provide an implementation of
a particular semantic of inner classes in MR jars, then that would be a
great unifying action that would guide the disparate groups to a common
interpretation of the semantic  that was missing from the original
specification.

regards







On 18 September 2017 at 05:27, Alan Bateman <Alan.Bateman at oracle.com> wrote:

> On 15/09/2017 22:58, Greg Wilkins wrote:
>
> :
>
>    - I think the stream needs to handle inner classes and only include
>    them if their matching outerclass is available at the same version.  So for
>    example a base Foo$Bar.class will only be included if the stream includes a
>    base Foo.class, and it will not be included if the Foo.class is version 9
>    or above.  Likewise a version 9 Foo$Bar.class will only be included in the
>    stream if the stream also includes a version 9 Foo.class, and will not  be
>    included if the stream has a version 10 or above Foo.class
>
> If you think this last point is possible, then I'll move the discussion
> back the EE expert groups to try to get an agreement on the exact stream
> code that will be used in the mid term until it is available in the JRE
> lib, at which time the specs should be amended to say they will defer the
> decision of which classes to scan the JRE lib so they will be future proof
> for any changes in java 10, 11 etc.
>
> I don't think this should be pushed down to the JarFile API. The JarFile
> API provides the base API for accessing JAR files and should not be
> concerned with the semantics or relationship between entries. I agree that
> annotation scanning tools and libraries need to do additional work to deal
> with orphaned or menacing inner classes in a MR JAR but it's not too
> different to arranging a class path with a JAR file containing the "classes
> for JDK 9" ahead of a JAR file containing the version of the library that
> runs on JDK 8. I do think that further checks could be done by the `jar`
> tool to identify issues at packaging time.
>
> -Alan
>



-- 
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From paul.sandoz at oracle.com  Mon Sep 18 17:04:59 2017
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 18 Sep 2017 10:04:59 -0700
Subject: Scanning multi version jars?
In-Reply-To: <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
 <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
 <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>
Message-ID: <7570FD98-FD63-481D-A30A-72071CDF6050@oracle.com>

I agree with Alan here, we should not be pushing a semantic understanding of inner classes into JarFile. 

I do sympathise with the case of annotation class scanning, which has always tunnelled through the class loader view to directly get at class file bytes possibly dealing with various URI schemes, since that is currently the only effective way of accessing the required information in an efficient manner.

As Alan mentioned we should add a traversable versioned view of a JarFile, returning a Stream, from which it should be possible to filter according to certain semantics.

Paul.


> On 17 Sep 2017, at 12:27, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> 
> On 15/09/2017 22:58, Greg Wilkins wrote:
>> :
>> 
>>  * I think the stream needs to handle inner classes and only include
>>    them if their matching outerclass is available at the same
>>    version.  So for example a base Foo$Bar.class will only be
>>    included if the stream includes a base Foo.class, and it will not
>>    be included if the Foo.class is version 9 or above.  Likewise a
>>    version 9 Foo$Bar.class will only be included in the stream if the
>>    stream also includes a version 9 Foo.class, and will not be
>>    included if the stream has a version 10 or above Foo.class
>> 
>> If you think this last point is possible, then I'll move the discussion back the EE expert groups to try to get an agreement on the exact stream code that will be used in the mid term until it is available in the JRE lib, at which time the specs should be amended to say they will defer the decision of which classes to scan the JRE lib so they will be future proof for any changes in java 10, 11 etc.
>> 
> I don't think this should be pushed down to the JarFile API. The JarFile API provides the base API for accessing JAR files and should not be concerned with the semantics or relationship between entries. I agree that annotation scanning tools and libraries need to do additional work to deal with orphaned or menacing inner classes in a MR JAR but it's not too different to arranging a class path with a JAR file containing the "classes for JDK 9" ahead of a JAR file containing the version of the library that runs on JDK 8. I do think that further checks could be done by the `jar` tool to identify issues at packaging time.
> 
> -Alan


From gregw at webtide.com  Tue Sep 19 00:18:43 2017
From: gregw at webtide.com (Greg Wilkins)
Date: Tue, 19 Sep 2017 10:18:43 +1000
Subject: Scanning multi version jars?
In-Reply-To: <7570FD98-FD63-481D-A30A-72071CDF6050@oracle.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
 <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
 <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>
 <7570FD98-FD63-481D-A30A-72071CDF6050@oracle.com>
Message-ID: <CAAPGdfH5EEOPMkB2ND1EJXZ3d+Ka9vjYU3jnu3JRifkXVorgBQ@mail.gmail.com>

Paul,

yeh... I guess I concede it's not JarFiles job... as much as that would
make things easier for containers to reach agreement:(

However, can we at least look at having a new default method on JarEntry to
query the version. Without that, containers don't have the information
available to perform the semantic filtering required and thus will not be
able to use the stream API and will have to work from an unversioned stream.

regards

On 19 September 2017 at 03:04, Paul Sandoz <paul.sandoz at oracle.com> wrote:

> I agree with Alan here, we should not be pushing a semantic understanding
> of inner classes into JarFile.
>
> I do sympathise with the case of annotation class scanning, which has
> always tunnelled through the class loader view to directly get at class
> file bytes possibly dealing with various URI schemes, since that is
> currently the only effective way of accessing the required information in
> an efficient manner.
>
> As Alan mentioned we should add a traversable versioned view of a JarFile,
> returning a Stream, from which it should be possible to filter according to
> certain semantics.
>
> Paul.
>
>
> > On 17 Sep 2017, at 12:27, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> >
> > On 15/09/2017 22:58, Greg Wilkins wrote:
> >> :
> >>
> >>  * I think the stream needs to handle inner classes and only include
> >>    them if their matching outerclass is available at the same
> >>    version.  So for example a base Foo$Bar.class will only be
> >>    included if the stream includes a base Foo.class, and it will not
> >>    be included if the Foo.class is version 9 or above.  Likewise a
> >>    version 9 Foo$Bar.class will only be included in the stream if the
> >>    stream also includes a version 9 Foo.class, and will not be
> >>    included if the stream has a version 10 or above Foo.class
> >>
> >> If you think this last point is possible, then I'll move the discussion
> back the EE expert groups to try to get an agreement on the exact stream
> code that will be used in the mid term until it is available in the JRE
> lib, at which time the specs should be amended to say they will defer the
> decision of which classes to scan the JRE lib so they will be future proof
> for any changes in java 10, 11 etc.
> >>
> > I don't think this should be pushed down to the JarFile API. The JarFile
> API provides the base API for accessing JAR files and should not be
> concerned with the semantics or relationship between entries. I agree that
> annotation scanning tools and libraries need to do additional work to deal
> with orphaned or menacing inner classes in a MR JAR but it's not too
> different to arranging a class path with a JAR file containing the "classes
> for JDK 9" ahead of a JAR file containing the version of the library that
> runs on JDK 8. I do think that further checks could be done by the `jar`
> tool to identify issues at packaging time.
> >
> > -Alan
>
>


-- 
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From stephen.felts at oracle.com  Tue Sep 19 01:05:34 2017
From: stephen.felts at oracle.com (Stephen Felts)
Date: Mon, 18 Sep 2017 18:05:34 -0700 (PDT)
Subject: Scanning multi version jars?
In-Reply-To: <CAAPGdfH5EEOPMkB2ND1EJXZ3d+Ka9vjYU3jnu3JRifkXVorgBQ@mail.gmail.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
 <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
 <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>
 <7570FD98-FD63-481D-A30A-72071CDF6050@oracle.com>
 <CAAPGdfH5EEOPMkB2ND1EJXZ3d+Ka9vjYU3jnu3JRifkXVorgBQ@mail.gmail.com>
Message-ID: <ab055e07-0258-4876-99c8-2569f6e6623f@default>

A versioned file name, JarEntry.getName(), starts with "META-INF/versions/".
The version is the following string up to the next "/".
The version can be parsed with Runtime.Version.parse().
If not a versioned class file name, then use Jarfile.baseVersion().
That should be sufficient to get the version for any JarEntry.

If it needs to run on pre-JDK9, this needs a lot of reflection.

IMO Having a method that behaves as described below is likely to be needed for many use cases and it would be good if someone wrote it and put it in a well-known, public jar file.



-----Original Message-----
From: Greg Wilkins [mailto:gregw at webtide.com] 
Sent: Monday, September 18, 2017 8:19 PM
To: Paul Sandoz <paul.sandoz at oracle.com>
Cc: jigsaw-dev <jigsaw-dev at openjdk.java.net>; core-libs-dev at openjdk.java.net
Subject: Re: Scanning multi version jars?

Paul,

yeh... I guess I concede it's not JarFiles job... as much as that would make things easier for containers to reach agreement:(

However, can we at least look at having a new default method on JarEntry to query the version. Without that, containers don't have the information available to perform the semantic filtering required and thus will not be able to use the stream API and will have to work from an unversioned stream.

regards

On 19 September 2017 at 03:04, Paul Sandoz <paul.sandoz at oracle.com> wrote:

> I agree with Alan here, we should not be pushing a semantic 
> understanding of inner classes into JarFile.
>
> I do sympathise with the case of annotation class scanning, which has 
> always tunnelled through the class loader view to directly get at 
> class file bytes possibly dealing with various URI schemes, since that 
> is currently the only effective way of accessing the required 
> information in an efficient manner.
>
> As Alan mentioned we should add a traversable versioned view of a 
> JarFile, returning a Stream, from which it should be possible to 
> filter according to certain semantics.
>
> Paul.
>
>
> > On 17 Sep 2017, at 12:27, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> >
> > On 15/09/2017 22:58, Greg Wilkins wrote:
> >> :
> >>
> >>  * I think the stream needs to handle inner classes and only include
> >>    them if their matching outerclass is available at the same
> >>    version.  So for example a base Foo$Bar.class will only be
> >>    included if the stream includes a base Foo.class, and it will not
> >>    be included if the Foo.class is version 9 or above.  Likewise a
> >>    version 9 Foo$Bar.class will only be included in the stream if the
> >>    stream also includes a version 9 Foo.class, and will not be
> >>    included if the stream has a version 10 or above Foo.class
> >>
> >> If you think this last point is possible, then I'll move the 
> >> discussion
> back the EE expert groups to try to get an agreement on the exact 
> stream code that will be used in the mid term until it is available in 
> the JRE lib, at which time the specs should be amended to say they 
> will defer the decision of which classes to scan the JRE lib so they 
> will be future proof for any changes in java 10, 11 etc.
> >>
> > I don't think this should be pushed down to the JarFile API. The 
> > JarFile
> API provides the base API for accessing JAR files and should not be 
> concerned with the semantics or relationship between entries. I agree 
> that annotation scanning tools and libraries need to do additional 
> work to deal with orphaned or menacing inner classes in a MR JAR but 
> it's not too different to arranging a class path with a JAR file 
> containing the "classes for JDK 9" ahead of a JAR file containing the 
> version of the library that runs on JDK 8. I do think that further 
> checks could be done by the `jar` tool to identify issues at packaging time.
> >
> > -Alan
>
>


--
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From gregw at webtide.com  Tue Sep 19 01:32:48 2017
From: gregw at webtide.com (Greg Wilkins)
Date: Tue, 19 Sep 2017 11:32:48 +1000
Subject: Scanning multi version jars?
In-Reply-To: <ab055e07-0258-4876-99c8-2569f6e6623f@default>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
 <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
 <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>
 <7570FD98-FD63-481D-A30A-72071CDF6050@oracle.com>
 <CAAPGdfH5EEOPMkB2ND1EJXZ3d+Ka9vjYU3jnu3JRifkXVorgBQ@mail.gmail.com>
 <ab055e07-0258-4876-99c8-2569f6e6623f@default>
Message-ID: <CAAPGdfH_ZdnS_UYV+kXNdeQ4wNTZAMu8UfHYVsAHtK+BUVzwOA@mail.gmail.com>

Stephen,

It is not the case that the getName() always returns the path starting with
"META-INF/versions/". Specifically, if the entry is obtained from
getJarEntry() API (and not from the enumerator), then the name is that of
the unversioned file, but the metadata and contents obtained using the
jarEntry are for the versioned entry.

For example the following code:


JarFile jarFile = new JarFile(new File("/tmp/example.jar"),
                              false,
                              JarFile.OPEN_READ,
                              Runtime.version());
JarEntry entry = jarFile.getJarEntry("org/example/OnlyIn9.class");
System.err.printf("%s ->
%s%n",entry.getName(),IO.toString(jarFile.getInputStream(entry)));

when run against a jar where the class files contain just the text of their
full path produces the following output:

org/example/OnlyIn9.class -> META-INF/versions/9/org/example/OnlyIn9.class


There is nothing in the public API of the JarEntry so obtained that
indicates that it the versioned entry, nor can I distinguish it from an
entry obtained by iteration that may report the same name (if the entry was
also in the base), although at least equals does return false.

Moreover, the proposed stream API as represented by the current
implementation of jdk.internal.util.jar.VersionedStream, applies some
filtering based on the versioning and then converts it's enumerated
JarEntry instances to opaquely versioned JarEntry instances by calling
map(jf::getJarEntry),which thus hides the version information and makes any
additional filtering based on version impossible by any users of that
stream.

regards


On 19 September 2017 at 11:05, Stephen Felts <stephen.felts at oracle.com>
wrote:

> A versioned file name, JarEntry.getName(), starts with
> "META-INF/versions/".
> The version is the following string up to the next "/".
> The version can be parsed with Runtime.Version.parse().
> If not a versioned class file name, then use Jarfile.baseVersion().
> That should be sufficient to get the version for any JarEntry.
>
> If it needs to run on pre-JDK9, this needs a lot of reflection.
>
> IMO Having a method that behaves as described below is likely to be needed
> for many use cases and it would be good if someone wrote it and put it in a
> well-known, public jar file.
>
>
>
> -----Original Message-----
> From: Greg Wilkins [mailto:gregw at webtide.com]
> Sent: Monday, September 18, 2017 8:19 PM
> To: Paul Sandoz <paul.sandoz at oracle.com>
> Cc: jigsaw-dev <jigsaw-dev at openjdk.java.net>;
> core-libs-dev at openjdk.java.net
> Subject: Re: Scanning multi version jars?
>
> Paul,
>
> yeh... I guess I concede it's not JarFiles job... as much as that would
> make things easier for containers to reach agreement:(
>
> However, can we at least look at having a new default method on JarEntry
> to query the version. Without that, containers don't have the information
> available to perform the semantic filtering required and thus will not be
> able to use the stream API and will have to work from an unversioned stream.
>
> regards
>
> On 19 September 2017 at 03:04, Paul Sandoz <paul.sandoz at oracle.com> wrote:
>
> > I agree with Alan here, we should not be pushing a semantic
> > understanding of inner classes into JarFile.
> >
> > I do sympathise with the case of annotation class scanning, which has
> > always tunnelled through the class loader view to directly get at
> > class file bytes possibly dealing with various URI schemes, since that
> > is currently the only effective way of accessing the required
> > information in an efficient manner.
> >
> > As Alan mentioned we should add a traversable versioned view of a
> > JarFile, returning a Stream, from which it should be possible to
> > filter according to certain semantics.
> >
> > Paul.
> >
> >
> > > On 17 Sep 2017, at 12:27, Alan Bateman <Alan.Bateman at oracle.com>
> wrote:
> > >
> > > On 15/09/2017 22:58, Greg Wilkins wrote:
> > >> :
> > >>
> > >>  * I think the stream needs to handle inner classes and only include
> > >>    them if their matching outerclass is available at the same
> > >>    version.  So for example a base Foo$Bar.class will only be
> > >>    included if the stream includes a base Foo.class, and it will not
> > >>    be included if the Foo.class is version 9 or above.  Likewise a
> > >>    version 9 Foo$Bar.class will only be included in the stream if the
> > >>    stream also includes a version 9 Foo.class, and will not be
> > >>    included if the stream has a version 10 or above Foo.class
> > >>
> > >> If you think this last point is possible, then I'll move the
> > >> discussion
> > back the EE expert groups to try to get an agreement on the exact
> > stream code that will be used in the mid term until it is available in
> > the JRE lib, at which time the specs should be amended to say they
> > will defer the decision of which classes to scan the JRE lib so they
> > will be future proof for any changes in java 10, 11 etc.
> > >>
> > > I don't think this should be pushed down to the JarFile API. The
> > > JarFile
> > API provides the base API for accessing JAR files and should not be
> > concerned with the semantics or relationship between entries. I agree
> > that annotation scanning tools and libraries need to do additional
> > work to deal with orphaned or menacing inner classes in a MR JAR but
> > it's not too different to arranging a class path with a JAR file
> > containing the "classes for JDK 9" ahead of a JAR file containing the
> > version of the library that runs on JDK 8. I do think that further
> > checks could be done by the `jar` tool to identify issues at packaging
> time.
> > >
> > > -Alan
> >
> >
>
>
> --
> Greg Wilkins <gregw at webtide.com> CTO http://webtide.com
>



-- 
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From stephen.felts at oracle.com  Tue Sep 19 03:41:19 2017
From: stephen.felts at oracle.com (Stephen Felts)
Date: Mon, 18 Sep 2017 20:41:19 -0700 (PDT)
Subject: Scanning multi version jars?
In-Reply-To: <CAAPGdfH_ZdnS_UYV+kXNdeQ4wNTZAMu8UfHYVsAHtK+BUVzwOA@mail.gmail.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
 <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
 <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>
 <7570FD98-FD63-481D-A30A-72071CDF6050@oracle.com>
 <CAAPGdfH5EEOPMkB2ND1EJXZ3d+Ka9vjYU3jnu3JRifkXVorgBQ@mail.gmail.com>
 <ab055e07-0258-4876-99c8-2569f6e6623f@default>
 <CAAPGdfH_ZdnS_UYV+kXNdeQ4wNTZAMu8UfHYVsAHtK+BUVzwOA@mail.gmail.com>
Message-ID: <a873b5eb-ad25-4b21-9127-58eb699b801f@default>

Thanks for the clarification ? I overstated the ?any JarEntry?.

I didn?t look at VersionedStream so I now understand the limitations you mention.

?

In my case, it?s necessary to look at all files in the jar file to do the elimination of unneeded ordinary/inner classes so JarInputStream getNextJarEntry() can be used.

By using the versioned JarFile constructor, getting the JarEntry returns the right one for processing.? If I needed further filtering on the file names, I?d need to return the real file names.

?

Maybe the use case isn?t so universal or well defined.

?

?????????????????????????????????????????????????????????????????????????????????????????????????????????????? 

?

From: Greg Wilkins [mailto:gregw at webtide.com] 
Sent: Monday, September 18, 2017 9:33 PM
To: Stephen Felts <stephen.felts at oracle.com>
Cc: Paul Sandoz <paul.sandoz at oracle.com>; jigsaw-dev <jigsaw-dev at openjdk.java.net>; core-libs-dev at openjdk.java.net
Subject: Re: Scanning multi version jars?

?

Stephen,

?

It is not the case that the getName() always returns the path starting with "META-INF/versions/". Specifically, if the entry is obtained from getJarEntry() API (and not from the enumerator), then the name is that of the unversioned file, but the metadata and contents obtained using the jarEntry are for the versioned entry.

?

For example the following code:

?

JarFile jarFile = new JarFile(new File("/tmp/example.jar"),
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? false,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? JarFile.OPEN_READ,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Runtime.version());
JarEntry entry = jarFile.getJarEntry("org/example/OnlyIn9.class");
System.err.printf("%s -> %s%n",entry.getName(),IO.toString(jarFile.getInputStream(entry)));

when run against a jar where the class files contain just the text of their full path produces the following output:

?

org/example/OnlyIn9.class -> META-INF/versions/9/org/example/OnlyIn9.class

?

There is nothing in the public API of the JarEntry so obtained that indicates that it the versioned entry, nor can I distinguish it from an entry obtained by iteration that may report the same name (if the entry was also in the base), although at least equals does return false.

?

Moreover, the proposed stream API as represented by the current implementation of jdk.internal.util.jar.VersionedStream, applies some filtering based on the versioning and then converts it's enumerated JarEntry instances to opaquely versioned JarEntry instances by calling map(jf::getJarEntry),which thus hides the version information and makes any additional filtering based on version impossible by any users of that stream.

?

regards

?

?

On 19 September 2017 at 11:05, Stephen Felts <HYPERLINK "mailto:stephen.felts at oracle.com"stephen.felts at oracle.com> wrote:

A versioned file name, JarEntry.getName(), starts with "META-INF/versions/".
The version is the following string up to the next "/".
The version can be parsed with Runtime.Version.parse().
If not a versioned class file name, then use Jarfile.baseVersion().
That should be sufficient to get the version for any JarEntry.

If it needs to run on pre-JDK9, this needs a lot of reflection.

IMO Having a method that behaves as described below is likely to be needed for many use cases and it would be good if someone wrote it and put it in a well-known, public jar file.



-----Original Message-----
From: Greg Wilkins [mailto:HYPERLINK "mailto:gregw at webtide.com"gregw at webtide.com]
Sent: Monday, September 18, 2017 8:19 PM
To: Paul Sandoz <HYPERLINK "mailto:paul.sandoz at oracle.com"paul.sandoz at oracle.com>
Cc: jigsaw-dev <HYPERLINK "mailto:jigsaw-dev at openjdk.java.net"jigsaw-dev at openjdk.java.net>; HYPERLINK "mailto:core-libs-dev at openjdk.java.net"core-libs-dev at openjdk.java.net
Subject: Re: Scanning multi version jars?

Paul,

yeh... I guess I concede it's not JarFiles job... as much as that would make things easier for containers to reach agreement:(

However, can we at least look at having a new default method on JarEntry to query the version. Without that, containers don't have the information available to perform the semantic filtering required and thus will not be able to use the stream API and will have to work from an unversioned stream.

regards

On 19 September 2017 at 03:04, Paul Sandoz <HYPERLINK "mailto:paul.sandoz at oracle.com"paul.sandoz at oracle.com> wrote:

> I agree with Alan here, we should not be pushing a semantic
> understanding of inner classes into JarFile.
>
> I do sympathise with the case of annotation class scanning, which has
> always tunnelled through the class loader view to directly get at
> class file bytes possibly dealing with various URI schemes, since that
> is currently the only effective way of accessing the required
> information in an efficient manner.
>
> As Alan mentioned we should add a traversable versioned view of a
> JarFile, returning a Stream, from which it should be possible to
> filter according to certain semantics.
>
> Paul.
>
>
> > On 17 Sep 2017, at 12:27, Alan Bateman <HYPERLINK "mailto:Alan.Bateman at oracle.com"Alan.Bateman at oracle.com> wrote:
> >
> > On 15/09/2017 22:58, Greg Wilkins wrote:
> >> :
> >>
> >>? * I think the stream needs to handle inner classes and only include
> >>? ? them if their matching outerclass is available at the same
> >>? ? version.? So for example a base Foo$Bar.class will only be
> >>? ? included if the stream includes a base Foo.class, and it will not
> >>? ? be included if the Foo.class is version 9 or above.? Likewise a
> >>? ? version 9 Foo$Bar.class will only be included in the stream if the
> >>? ? stream also includes a version 9 Foo.class, and will not be
> >>? ? included if the stream has a version 10 or above Foo.class
> >>
> >> If you think this last point is possible, then I'll move the
> >> discussion
> back the EE expert groups to try to get an agreement on the exact
> stream code that will be used in the mid term until it is available in
> the JRE lib, at which time the specs should be amended to say they
> will defer the decision of which classes to scan the JRE lib so they
> will be future proof for any changes in java 10, 11 etc.
> >>
> > I don't think this should be pushed down to the JarFile API. The
> > JarFile
> API provides the base API for accessing JAR files and should not be
> concerned with the semantics or relationship between entries. I agree
> that annotation scanning tools and libraries need to do additional
> work to deal with orphaned or menacing inner classes in a MR JAR but
> it's not too different to arranging a class path with a JAR file
> containing the "classes for JDK 9" ahead of a JAR file containing the
> version of the library that runs on JDK 8. I do think that further
> checks could be done by the `jar` tool to identify issues at packaging time.
> >
> > -Alan
>
>


--
Greg Wilkins <HYPERLINK "mailto:gregw at webtide.com"gregw at webtide.com> CTO http://webtide.com





?

-- 

Greg Wilkins <HYPERLINK "mailto:gregw at webtide.com"gregw at webtide.com> CTO http://webtide.com

From gregw at webtide.com  Tue Sep 19 04:37:37 2017
From: gregw at webtide.com (Greg Wilkins)
Date: Tue, 19 Sep 2017 14:37:37 +1000
Subject: Scanning multi version jars?
In-Reply-To: <a873b5eb-ad25-4b21-9127-58eb699b801f@default>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
 <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
 <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>
 <7570FD98-FD63-481D-A30A-72071CDF6050@oracle.com>
 <CAAPGdfH5EEOPMkB2ND1EJXZ3d+Ka9vjYU3jnu3JRifkXVorgBQ@mail.gmail.com>
 <ab055e07-0258-4876-99c8-2569f6e6623f@default>
 <CAAPGdfH_ZdnS_UYV+kXNdeQ4wNTZAMu8UfHYVsAHtK+BUVzwOA@mail.gmail.com>
 <a873b5eb-ad25-4b21-9127-58eb699b801f@default>
Message-ID: <CAAPGdfGMRv-FPQw-pecPFiSedDp+SyeUV5FUuX-UDRubtbu=UA@mail.gmail.com>

Stephen,

I think the use-case can be pretty well defined.

There should be an enumeration/iterator/stream available that provides the
contents of a jar file as it would be seen/interpreted by the JVMs
classloader.    So if the classloader is doing any processing to handle
versioned classes/resources, then we need an iterator that implements the
exact same logic.

Which raises an interesting point....   with the multi versioned jar I have
used as an example, which contains:

   - org/example/Foo.class
   - org/example/Foo$Bar.class
   - META-INF/versions/9/org/example/Foo.class

What does the classloader do when asked to load "org.example.Foo$Bar" ?
 If it loads it OK, then the JarFile enumerator/iterator/stream should also
return it.   If it throws a ClassNotFoundException, then the
JarFile enumerator/iterator/stream should skip it.

Currently the classloader will happily return a resource for a base inner
class even if its outerclass does not refer to it, so that suggests that
the iteration should also not process out the inappropriate inner classes.
However I think it could be argued that the loader should not load it.

Eitherway, there should be an iteration available that is entirely
consistent with what the classloader does.

regards




On 19 September 2017 at 13:41, Stephen Felts <stephen.felts at oracle.com>
wrote:

> Thanks for the clarification ? I overstated the ?any JarEntry?.
>
> I didn?t look at VersionedStream so I now understand the limitations you
> mention.
>
>
>
> In my case, it?s necessary to look at all files in the jar file to do the
> elimination of unneeded ordinary/inner classes so JarInputStream getNextJarEntry()
> can be used.
>
> By using the versioned JarFile constructor, getting the JarEntry returns
> the right one for processing.  If I needed further filtering on the file
> names, I?d need to return the real file names.
>
>
>
> Maybe the use case isn?t so universal or well defined.
>
>
>
>
>
>
>
>
> *From:* Greg Wilkins [mailto:gregw at webtide.com]
> *Sent:* Monday, September 18, 2017 9:33 PM
> *To:* Stephen Felts <stephen.felts at oracle.com>
> *Cc:* Paul Sandoz <paul.sandoz at oracle.com>; jigsaw-dev <
> jigsaw-dev at openjdk.java.net>; core-libs-dev at openjdk.java.net
>
> *Subject:* Re: Scanning multi version jars?
>
>
>
> Stephen,
>
>
>
> It is not the case that the getName() always returns the path starting
> with "META-INF/versions/". Specifically, if the entry is obtained from
> getJarEntry() API (and not from the enumerator), then the name is that of
> the unversioned file, but the metadata and contents obtained using the
> jarEntry are for the versioned entry.
>
>
>
> For example the following code:
>
>
>
> JarFile jarFile = new JarFile(new File("/tmp/example.jar"),
>                               false,
>                               JarFile.OPEN_READ,
>                               Runtime.version());
> JarEntry entry = jarFile.getJarEntry("org/example/OnlyIn9.class");
> System.err.printf("%s -> %s%n",entry.getName(),IO.toString(jarFile.
> getInputStream(entry)));
>
> when run against a jar where the class files contain just the text of
> their full path produces the following output:
>
>
>
> org/example/OnlyIn9.class -> META-INF/versions/9/org/example/OnlyIn9.class
>
>
>
> There is nothing in the public API of the JarEntry so obtained that
> indicates that it the versioned entry, nor can I distinguish it from an
> entry obtained by iteration that may report the same name (if the entry was
> also in the base), although at least equals does return false.
>
>
>
> Moreover, the proposed stream API as represented by the current
> implementation of jdk.internal.util.jar.VersionedStream, applies some
> filtering based on the versioning and then converts it's enumerated
> JarEntry instances to opaquely versioned JarEntry instances by calling
> map(jf::getJarEntry),which thus hides the version information and makes
> any additional filtering based on version impossible by any users of that
> stream.
>
>
>
> regards
>
>
>
>
>
> On 19 September 2017 at 11:05, Stephen Felts <stephen.felts at oracle.com>
> wrote:
>
> A versioned file name, JarEntry.getName(), starts with
> "META-INF/versions/".
> The version is the following string up to the next "/".
> The version can be parsed with Runtime.Version.parse().
> If not a versioned class file name, then use Jarfile.baseVersion().
> That should be sufficient to get the version for any JarEntry.
>
> If it needs to run on pre-JDK9, this needs a lot of reflection.
>
> IMO Having a method that behaves as described below is likely to be needed
> for many use cases and it would be good if someone wrote it and put it in a
> well-known, public jar file.
>
>
>
> -----Original Message-----
> From: Greg Wilkins [mailto:gregw at webtide.com]
> Sent: Monday, September 18, 2017 8:19 PM
> To: Paul Sandoz <paul.sandoz at oracle.com>
> Cc: jigsaw-dev <jigsaw-dev at openjdk.java.net>;
> core-libs-dev at openjdk.java.net
> Subject: Re: Scanning multi version jars?
>
> Paul,
>
> yeh... I guess I concede it's not JarFiles job... as much as that would
> make things easier for containers to reach agreement:(
>
> However, can we at least look at having a new default method on JarEntry
> to query the version. Without that, containers don't have the information
> available to perform the semantic filtering required and thus will not be
> able to use the stream API and will have to work from an unversioned stream.
>
> regards
>
> On 19 September 2017 at 03:04, Paul Sandoz <paul.sandoz at oracle.com> wrote:
>
> > I agree with Alan here, we should not be pushing a semantic
> > understanding of inner classes into JarFile.
> >
> > I do sympathise with the case of annotation class scanning, which has
> > always tunnelled through the class loader view to directly get at
> > class file bytes possibly dealing with various URI schemes, since that
> > is currently the only effective way of accessing the required
> > information in an efficient manner.
> >
> > As Alan mentioned we should add a traversable versioned view of a
> > JarFile, returning a Stream, from which it should be possible to
> > filter according to certain semantics.
> >
> > Paul.
> >
> >
> > > On 17 Sep 2017, at 12:27, Alan Bateman <Alan.Bateman at oracle.com>
> wrote:
> > >
> > > On 15/09/2017 22:58, Greg Wilkins wrote:
> > >> :
> > >>
> > >>  * I think the stream needs to handle inner classes and only include
> > >>    them if their matching outerclass is available at the same
> > >>    version.  So for example a base Foo$Bar.class will only be
> > >>    included if the stream includes a base Foo.class, and it will not
> > >>    be included if the Foo.class is version 9 or above.  Likewise a
> > >>    version 9 Foo$Bar.class will only be included in the stream if the
> > >>    stream also includes a version 9 Foo.class, and will not be
> > >>    included if the stream has a version 10 or above Foo.class
> > >>
> > >> If you think this last point is possible, then I'll move the
> > >> discussion
> > back the EE expert groups to try to get an agreement on the exact
> > stream code that will be used in the mid term until it is available in
> > the JRE lib, at which time the specs should be amended to say they
> > will defer the decision of which classes to scan the JRE lib so they
> > will be future proof for any changes in java 10, 11 etc.
> > >>
> > > I don't think this should be pushed down to the JarFile API. The
> > > JarFile
> > API provides the base API for accessing JAR files and should not be
> > concerned with the semantics or relationship between entries. I agree
> > that annotation scanning tools and libraries need to do additional
> > work to deal with orphaned or menacing inner classes in a MR JAR but
> > it's not too different to arranging a class path with a JAR file
> > containing the "classes for JDK 9" ahead of a JAR file containing the
> > version of the library that runs on JDK 8. I do think that further
> > checks could be done by the `jar` tool to identify issues at packaging
> time.
> > >
> > > -Alan
> >
> >
>
>
> --
> Greg Wilkins <gregw at webtide.com> CTO http://webtide.com
>
>
>
>
>
> --
>
> Greg Wilkins <gregw at webtide.com> CTO http://webtide.com
>



-- 
Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From forax at univ-mlv.fr  Tue Sep 19 07:15:22 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Tue, 19 Sep 2017 09:15:22 +0200 (CEST)
Subject: Scanning multi version jars?
In-Reply-To: <CAAPGdfGMRv-FPQw-pecPFiSedDp+SyeUV5FUuX-UDRubtbu=UA@mail.gmail.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>
 <7570FD98-FD63-481D-A30A-72071CDF6050@oracle.com>
 <CAAPGdfH5EEOPMkB2ND1EJXZ3d+Ka9vjYU3jnu3JRifkXVorgBQ@mail.gmail.com>
 <ab055e07-0258-4876-99c8-2569f6e6623f@default>
 <CAAPGdfH_ZdnS_UYV+kXNdeQ4wNTZAMu8UfHYVsAHtK+BUVzwOA@mail.gmail.com>
 <a873b5eb-ad25-4b21-9127-58eb699b801f@default>
 <CAAPGdfGMRv-FPQw-pecPFiSedDp+SyeUV5FUuX-UDRubtbu=UA@mail.gmail.com>
Message-ID: <484658822.1202607.1505805322702.JavaMail.zimbra@u-pem.fr>

Hi Greg,
the notion of inner classes do not exist at runtime (apart if you explicitly ask by reflection).

The compiler desugar inner classes to several classes, so the compiler needs attributes (InnerClasses and EnclosingMethod) in the classfile to be able to reconstruct the java source file view of the world when it sees already compiled classes.

But the VM/runtime doesn't need those attributes, the VM doesn't care about inner classes. There is a comment at the end of section 4.7.6 of the JVMS that explain that the VM does not check the InnerClasses attribute at runtime "Oracle's Java Virtual Machine implementation does not check the consistency of an InnerClasses attribute against a class file representing a class or interface referenced by the attribute." 

So to answer to your question, the classloader do not care about inner classes.

Note that in a close future with the introduction of nestmates [1], it will be another story, but we have at least to wait 18.9 for that.

R?mi
[1] http://openjdk.java.net/jeps/181

----- Mail original -----
> De: "Greg Wilkins" <gregw at webtide.com>
> ?: "Stephen Felts" <stephen.felts at oracle.com>
> Cc: "jigsaw-dev" <jigsaw-dev at openjdk.java.net>, "core-libs-dev" <core-libs-dev at openjdk.java.net>
> Envoy?: Mardi 19 Septembre 2017 06:37:37
> Objet: Re: Scanning multi version jars?

> Stephen,
> 
> I think the use-case can be pretty well defined.
> 
> There should be an enumeration/iterator/stream available that provides the
> contents of a jar file as it would be seen/interpreted by the JVMs
> classloader.    So if the classloader is doing any processing to handle
> versioned classes/resources, then we need an iterator that implements the
> exact same logic.
> 
> Which raises an interesting point....   with the multi versioned jar I have
> used as an example, which contains:
> 
>   - org/example/Foo.class
>   - org/example/Foo$Bar.class
>   - META-INF/versions/9/org/example/Foo.class
> 
> What does the classloader do when asked to load "org.example.Foo$Bar" ?
> If it loads it OK, then the JarFile enumerator/iterator/stream should also
> return it.   If it throws a ClassNotFoundException, then the
> JarFile enumerator/iterator/stream should skip it.
> 
> Currently the classloader will happily return a resource for a base inner
> class even if its outerclass does not refer to it, so that suggests that
> the iteration should also not process out the inappropriate inner classes.
> However I think it could be argued that the loader should not load it.
> 
> Eitherway, there should be an iteration available that is entirely
> consistent with what the classloader does.
> 
> regards
> 
> 
> 
> 
> On 19 September 2017 at 13:41, Stephen Felts <stephen.felts at oracle.com>
> wrote:
> 
>> Thanks for the clarification ? I overstated the ?any JarEntry?.
>>
>> I didn?t look at VersionedStream so I now understand the limitations you
>> mention.
>>
>>
>>
>> In my case, it?s necessary to look at all files in the jar file to do the
>> elimination of unneeded ordinary/inner classes so JarInputStream
>> getNextJarEntry()
>> can be used.
>>
>> By using the versioned JarFile constructor, getting the JarEntry returns
>> the right one for processing.  If I needed further filtering on the file
>> names, I?d need to return the real file names.
>>
>>
>>
>> Maybe the use case isn?t so universal or well defined.
>>
>>
>>
>>
>>
>>
>>
>>
>> *From:* Greg Wilkins [mailto:gregw at webtide.com]
>> *Sent:* Monday, September 18, 2017 9:33 PM
>> *To:* Stephen Felts <stephen.felts at oracle.com>
>> *Cc:* Paul Sandoz <paul.sandoz at oracle.com>; jigsaw-dev <
>> jigsaw-dev at openjdk.java.net>; core-libs-dev at openjdk.java.net
>>
>> *Subject:* Re: Scanning multi version jars?
>>
>>
>>
>> Stephen,
>>
>>
>>
>> It is not the case that the getName() always returns the path starting
>> with "META-INF/versions/". Specifically, if the entry is obtained from
>> getJarEntry() API (and not from the enumerator), then the name is that of
>> the unversioned file, but the metadata and contents obtained using the
>> jarEntry are for the versioned entry.
>>
>>
>>
>> For example the following code:
>>
>>
>>
>> JarFile jarFile = new JarFile(new File("/tmp/example.jar"),
>>                               false,
>>                               JarFile.OPEN_READ,
>>                               Runtime.version());
>> JarEntry entry = jarFile.getJarEntry("org/example/OnlyIn9.class");
>> System.err.printf("%s -> %s%n",entry.getName(),IO.toString(jarFile.
>> getInputStream(entry)));
>>
>> when run against a jar where the class files contain just the text of
>> their full path produces the following output:
>>
>>
>>
>> org/example/OnlyIn9.class -> META-INF/versions/9/org/example/OnlyIn9.class
>>
>>
>>
>> There is nothing in the public API of the JarEntry so obtained that
>> indicates that it the versioned entry, nor can I distinguish it from an
>> entry obtained by iteration that may report the same name (if the entry was
>> also in the base), although at least equals does return false.
>>
>>
>>
>> Moreover, the proposed stream API as represented by the current
>> implementation of jdk.internal.util.jar.VersionedStream, applies some
>> filtering based on the versioning and then converts it's enumerated
>> JarEntry instances to opaquely versioned JarEntry instances by calling
>> map(jf::getJarEntry),which thus hides the version information and makes
>> any additional filtering based on version impossible by any users of that
>> stream.
>>
>>
>>
>> regards
>>
>>
>>
>>
>>
>> On 19 September 2017 at 11:05, Stephen Felts <stephen.felts at oracle.com>
>> wrote:
>>
>> A versioned file name, JarEntry.getName(), starts with
>> "META-INF/versions/".
>> The version is the following string up to the next "/".
>> The version can be parsed with Runtime.Version.parse().
>> If not a versioned class file name, then use Jarfile.baseVersion().
>> That should be sufficient to get the version for any JarEntry.
>>
>> If it needs to run on pre-JDK9, this needs a lot of reflection.
>>
>> IMO Having a method that behaves as described below is likely to be needed
>> for many use cases and it would be good if someone wrote it and put it in a
>> well-known, public jar file.
>>
>>
>>
>> -----Original Message-----
>> From: Greg Wilkins [mailto:gregw at webtide.com]
>> Sent: Monday, September 18, 2017 8:19 PM
>> To: Paul Sandoz <paul.sandoz at oracle.com>
>> Cc: jigsaw-dev <jigsaw-dev at openjdk.java.net>;
>> core-libs-dev at openjdk.java.net
>> Subject: Re: Scanning multi version jars?
>>
>> Paul,
>>
>> yeh... I guess I concede it's not JarFiles job... as much as that would
>> make things easier for containers to reach agreement:(
>>
>> However, can we at least look at having a new default method on JarEntry
>> to query the version. Without that, containers don't have the information
>> available to perform the semantic filtering required and thus will not be
>> able to use the stream API and will have to work from an unversioned stream.
>>
>> regards
>>
>> On 19 September 2017 at 03:04, Paul Sandoz <paul.sandoz at oracle.com> wrote:
>>
>> > I agree with Alan here, we should not be pushing a semantic
>> > understanding of inner classes into JarFile.
>> >
>> > I do sympathise with the case of annotation class scanning, which has
>> > always tunnelled through the class loader view to directly get at
>> > class file bytes possibly dealing with various URI schemes, since that
>> > is currently the only effective way of accessing the required
>> > information in an efficient manner.
>> >
>> > As Alan mentioned we should add a traversable versioned view of a
>> > JarFile, returning a Stream, from which it should be possible to
>> > filter according to certain semantics.
>> >
>> > Paul.
>> >
>> >
>> > > On 17 Sep 2017, at 12:27, Alan Bateman <Alan.Bateman at oracle.com>
>> wrote:
>> > >
>> > > On 15/09/2017 22:58, Greg Wilkins wrote:
>> > >> :
>> > >>
>> > >>  * I think the stream needs to handle inner classes and only include
>> > >>    them if their matching outerclass is available at the same
>> > >>    version.  So for example a base Foo$Bar.class will only be
>> > >>    included if the stream includes a base Foo.class, and it will not
>> > >>    be included if the Foo.class is version 9 or above.  Likewise a
>> > >>    version 9 Foo$Bar.class will only be included in the stream if the
>> > >>    stream also includes a version 9 Foo.class, and will not be
>> > >>    included if the stream has a version 10 or above Foo.class
>> > >>
>> > >> If you think this last point is possible, then I'll move the
>> > >> discussion
>> > back the EE expert groups to try to get an agreement on the exact
>> > stream code that will be used in the mid term until it is available in
>> > the JRE lib, at which time the specs should be amended to say they
>> > will defer the decision of which classes to scan the JRE lib so they
>> > will be future proof for any changes in java 10, 11 etc.
>> > >>
>> > > I don't think this should be pushed down to the JarFile API. The
>> > > JarFile
>> > API provides the base API for accessing JAR files and should not be
>> > concerned with the semantics or relationship between entries. I agree
>> > that annotation scanning tools and libraries need to do additional
>> > work to deal with orphaned or menacing inner classes in a MR JAR but
>> > it's not too different to arranging a class path with a JAR file
>> > containing the "classes for JDK 9" ahead of a JAR file containing the
>> > version of the library that runs on JDK 8. I do think that further
>> > checks could be done by the `jar` tool to identify issues at packaging
>> time.
>> > >
>> > > -Alan
>> >
>> >
>>
>>
>> --
>> Greg Wilkins <gregw at webtide.com> CTO http://webtide.com
>>
>>
>>
>>
>>
>> --
>>
>> Greg Wilkins <gregw at webtide.com> CTO http://webtide.com
>>
> 
> 
> 
> --
> Greg Wilkins <gregw at webtide.com> CTO http://webtide.com

From Alan.Bateman at oracle.com  Tue Sep 19 07:34:18 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 19 Sep 2017 08:34:18 +0100
Subject: Scanning multi version jars?
In-Reply-To: <CAAPGdfGMRv-FPQw-pecPFiSedDp+SyeUV5FUuX-UDRubtbu=UA@mail.gmail.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
 <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
 <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>
 <7570FD98-FD63-481D-A30A-72071CDF6050@oracle.com>
 <CAAPGdfH5EEOPMkB2ND1EJXZ3d+Ka9vjYU3jnu3JRifkXVorgBQ@mail.gmail.com>
 <ab055e07-0258-4876-99c8-2569f6e6623f@default>
 <CAAPGdfH_ZdnS_UYV+kXNdeQ4wNTZAMu8UfHYVsAHtK+BUVzwOA@mail.gmail.com>
 <a873b5eb-ad25-4b21-9127-58eb699b801f@default>
 <CAAPGdfGMRv-FPQw-pecPFiSedDp+SyeUV5FUuX-UDRubtbu=UA@mail.gmail.com>
Message-ID: <cb013a55-b025-71c7-1c3c-2607d95b1c6c@oracle.com>

On 19/09/2017 05:37, Greg Wilkins wrote:
> :
>
> Which raises an interesting point....   with the multi versioned jar I have
> used as an example, which contains:
>
>     - org/example/Foo.class
>     - org/example/Foo$Bar.class
>     - META-INF/versions/9/org/example/Foo.class
>
> What does the classloader do when asked to load "org.example.Foo$Bar" ?
>   If it loads it OK, then the JarFile enumerator/iterator/stream should also
> return it.   If it throws a ClassNotFoundException, then the
> JarFile enumerator/iterator/stream should skip it.
A class loader that loads from a JAR file will just map 
"org.example.Foo$Bar" to entry "org/example/Foo$Bar.class" and attempt 
to define the class bytes to VM.? It shouldn't care if the entry comes 
from the base or a versioned section. It also shouldn't care if the 
class name looks like it might have been compiled from an inner class.

The one case where a custom class loader does need to know more is when 
it loading resources (findResource/findResources implementation 
usually). For that case then the returned URL needs to locate the right 
resource and so may encode a path to a resource in a versioned section. 
You'll see URLClassLoader does the right thing, as does the built-in 
class loaders for cases where you have modular MR JARs on the module 
path. There were a few threads on core-libs-dev discussing whether to 
add a getRealName method but in the end it was kicked down the road to 
re-examine later.

-Alan

From sadhak001 at gmail.com  Tue Sep 19 21:55:16 2017
From: sadhak001 at gmail.com (Mani Sarkar)
Date: Tue, 19 Sep 2017 21:55:16 +0000
Subject: Become an early Java 9 expert: AJUG + vJUG + JUGs Worldwide Hackday
 Feedback on JDK 9 EA
Message-ID: <CAGHtMWmqeVt5ORGGuYo7EDp-7whA2Dogos_hd3SGh1s_y2_c-Q@mail.gmail.com>

Hi all,

My apologies for the delayed message, as everyone is getting ready for
JavaOne and also busy with working on JDK 9 RC1 feedback since last month.

Last month (19th August 2017) AJUG and a number of JUGs worldwide with the
help and support from vJUG, re-ran the  "Become an early Java 9 expert"
hackday.

You can find the feedback gathered in http://bit.ly/J9HackDay-AJUG-feedback,
we have been trailing JDK 9 EA b181 (RC1).

I hope you will be able to make good use of the feedback and post your
responses in the doc, I could also post the individual issues as separate
messages if they become discussion worthy.

Like the previous time, points that need addressing are marked with a
rightwards arrowhead ( ? ) - we have had three categories of feedbacks
namely *Potential bugs, Suggestions* and *Queries* about the different
aspects of Jigsaw & Java 9.

Please let me know if any other individuals or lists need to be informed
about this.

Thanks.

Cheers,
Mani
-- 
@theNeomatrix369 <http://twitter.com/theNeomatrix369>*  |  **Blog
<http://neomatrix369.wordpress.com/>**  |  *@adoptopenjdk | Dev communities
*Meet-a-Project - *MutabilityDetector
<https://github.com/MutabilityDetector>*  |  **Bitbucket
<https://bitbucket.org/neomatrix369>* * |  **Github
<https://github.com/neomatrix369>* * |  **LinkedIn
<http://uk.linkedin.com/pub/mani-sarkar/71/a77/39b>*
*Come to Devoxx UK 2018:* http://www.devoxx.co.uk/

*Don't chase success, rather aim for "Excellence", and success will come
chasing after you!*

From sadhak001 at gmail.com  Tue Sep 19 21:57:50 2017
From: sadhak001 at gmail.com (Mani Sarkar)
Date: Tue, 19 Sep 2017 21:57:50 +0000
Subject: Become an early Java 9 expert: AJUG + vJUG + JUGs Worldwide
 Hackday Feedback on JDK 9 EA
In-Reply-To: <CAGHtMWmqeVt5ORGGuYo7EDp-7whA2Dogos_hd3SGh1s_y2_c-Q@mail.gmail.com>
References: <CAGHtMWmqeVt5ORGGuYo7EDp-7whA2Dogos_hd3SGh1s_y2_c-Q@mail.gmail.com>
Message-ID: <CAGHtMWn+Zb0PFdqj0V=Wp1c_kQEr4EhrZR6HkzTvejOasu=JpQ@mail.gmail.com>

Adding hubert.pan3 at gmail.com, vaslabsco at gmail.com, mgrand at markgrand.com to
the conversation.

On Tue, 19 Sep 2017 at 22:55 Mani Sarkar <sadhak001 at gmail.com> wrote:

> Hi all,
>
> My apologies for the delayed message, as everyone is getting ready for
> JavaOne and also busy with working on JDK 9 RC1 feedback since last month.
>
> Last month (19th August 2017) AJUG and a number of JUGs worldwide with
> the help and support from vJUG, re-ran the  "Become an early Java 9 expert"
> hackday.
>
> You can find the feedback gathered in
> http://bit.ly/J9HackDay-AJUG-feedback, we have been trailing JDK 9 EA
> b181 (RC1).
>
> I hope you will be able to make good use of the feedback and post your
> responses in the doc, I could also post the individual issues as separate
> messages if they become discussion worthy.
>
> Like the previous time, points that need addressing are marked with a
> rightwards arrowhead ( ? ) - we have had three categories of feedbacks
> namely *Potential bugs, Suggestions* and *Queries* about the different
> aspects of Jigsaw & Java 9.
>
> Please let me know if any other individuals or lists need to be informed
> about this.
>
> Thanks.
>
> Cheers,
> Mani
> --
> @theNeomatrix369 <http://twitter.com/theNeomatrix369>*  |  **Blog
> <http://neomatrix369.wordpress.com/>**  |  *@adoptopenjdk | Dev
> communities
> *Meet-a-Project - *MutabilityDetector
> <https://github.com/MutabilityDetector>*  |  **Bitbucket
> <https://bitbucket.org/neomatrix369>* * |  **Github
> <https://github.com/neomatrix369>* * |  **LinkedIn
> <http://uk.linkedin.com/pub/mani-sarkar/71/a77/39b>*
> *Come to Devoxx UK 2018:* http://www.devoxx.co.uk/
>
> *Don't chase success, rather aim for "Excellence", and success will come
> chasing after you!*
>
-- 
@theNeomatrix369 <http://twitter.com/theNeomatrix369>*  |  **Blog
<http://neomatrix369.wordpress.com/>**  |  *@adoptopenjdk | Dev communities
*Meet-a-Project - *MutabilityDetector
<https://github.com/MutabilityDetector>*  |  **Bitbucket
<https://bitbucket.org/neomatrix369>* * |  **Github
<https://github.com/neomatrix369>* * |  **LinkedIn
<http://uk.linkedin.com/pub/mani-sarkar/71/a77/39b>*
*Come to Devoxx UK 2018:* http://www.devoxx.co.uk/

*Don't chase success, rather aim for "Excellence", and success will come
chasing after you!*

From alex.buckley at oracle.com  Wed Sep 20 01:54:26 2017
From: alex.buckley at oracle.com (Alex Buckley)
Date: Tue, 19 Sep 2017 18:54:26 -0700
Subject: Become an early Java 9 expert: AJUG + vJUG + JUGs Worldwide
 Hackday Feedback on JDK 9 EA
In-Reply-To: <CAGHtMWmqeVt5ORGGuYo7EDp-7whA2Dogos_hd3SGh1s_y2_c-Q@mail.gmail.com>
References: <CAGHtMWmqeVt5ORGGuYo7EDp-7whA2Dogos_hd3SGh1s_y2_c-Q@mail.gmail.com>
Message-ID: <59C1CA52.6030504@oracle.com>

Hi Mani,

On 9/19/2017 2:55 PM, Mani Sarkar wrote:
> Last month (19th August 2017) AJUG and a number of JUGs worldwide with the
> help and support from vJUG, re-ran the  "Become an early Java 9 expert"
> hackday.

Thank you AJUG and vJUG!

> You can find the feedback gathered in http://bit.ly/J9HackDay-AJUG-feedback,
> we have been trailing JDK 9 EA b181 (RC1).

A lot of feedback seems to boil down to "JDK command line tools are not 
so easy to use; I want my IDE!". I don't mean to make light of people's 
usability issues, but the module-related paths and flags in JDK 9 tools 
tend to operate along similar lines as the paths and flags in JDK 8 
tools -- it's just that a lot of people haven't set ANY paths and flags 
for a long time.

I see there were some more open-ended questions and this one in 
particular caught my attention:

-----
Do I need to convert a legacy Java program to use named modules in order 
to take advantage of the smaller images that jlink can create?

Mani: You will have to convert your applications to use Java 9?s modules 
system in order to take advantage of JLink fully, although please play 
around with older legacy apps to see what JLink produces (most likely 
the whole JDK and not modularised pieces).

Simon: create an empty module with module-info.java and handcraft the 
dependencies using requires and have jlink compile it. This is 
experimental, would need to be tested to see how it works.
-----

The direct answer to the question is "No, you do not need to convert a 
legacy Java program to use named modules in order to take advantage of 
the smaller images that jlink can create."

The Java runtime that's present in even the smallest image (just 
java.base) still lets you to put your pre-existing JAR files on the 
classpath and run them with java -cp. You do not need to turn your JAR 
files into named modules. Even as traditional JAR files, they have 
access to all the APIs that you would expect from such an image. 
(Obviously if your JAR files try to use Swing on an image built from 
just java.base, that won't work.) The reduced footprint and security 
surface of the smaller image is plainly an advantage from jlink.

Alex

From scolebourne at joda.org  Wed Sep 20 12:43:47 2017
From: scolebourne at joda.org (Stephen Colebourne)
Date: Wed, 20 Sep 2017 13:43:47 +0100
Subject: java.lang.annotation.Generated
Message-ID: <CACzrW9CxoTMwwnGNa9JuL5vjRDDMRtS4HoPLM3BnYi8_DMdGtQ@mail.gmail.com>

As per this email:
 http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-February/011365.html
the idea was to add a new annotation `java.lang.annotation.Generated`
to replace the old problematic one.

Is it my imagination, or did this get forgotten?:
 http://download.java.net/java/jdk9/docs/api/java/lang/annotation/package-summary.html

Stephen

From Alan.Bateman at oracle.com  Wed Sep 20 12:49:54 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 20 Sep 2017 13:49:54 +0100
Subject: java.lang.annotation.Generated
In-Reply-To: <CACzrW9CxoTMwwnGNa9JuL5vjRDDMRtS4HoPLM3BnYi8_DMdGtQ@mail.gmail.com>
References: <CACzrW9CxoTMwwnGNa9JuL5vjRDDMRtS4HoPLM3BnYi8_DMdGtQ@mail.gmail.com>
Message-ID: <bdac8fe8-814d-e98f-9573-98f242d9ff4b@oracle.com>

On 20/09/2017 13:43, Stephen Colebourne wrote:
> As per this email:
>   http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-February/011365.html
> the idea was to add a new annotation `java.lang.annotation.Generated`
> to replace the old problematic one.
>
> Is it my imagination, or did this get forgotten?:
>   
The discussion/review moved to core-libs-dev and compiler-dev in March, 
here it is:

http://download.java.net/java/jdk9/docs/api/javax/annotation/processing/Generated.html

From david.lloyd at redhat.com  Wed Sep 20 12:52:59 2017
From: david.lloyd at redhat.com (David Lloyd)
Date: Wed, 20 Sep 2017 07:52:59 -0500
Subject: java.lang.annotation.Generated
In-Reply-To: <bdac8fe8-814d-e98f-9573-98f242d9ff4b@oracle.com>
References: <CACzrW9CxoTMwwnGNa9JuL5vjRDDMRtS4HoPLM3BnYi8_DMdGtQ@mail.gmail.com>
 <bdac8fe8-814d-e98f-9573-98f242d9ff4b@oracle.com>
Message-ID: <CANghgrRucPS+3vpJENsdTndGcsqXd8HLpEODSoGKiygHth8DsA@mail.gmail.com>

On Wed, Sep 20, 2017 at 7:49 AM, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> On 20/09/2017 13:43, Stephen Colebourne wrote:
>>
>> As per this email:
>>
>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-February/011365.html
>> the idea was to add a new annotation `java.lang.annotation.Generated`
>> to replace the old problematic one.
>>
>> Is it my imagination, or did this get forgotten?:
>>
>
> The discussion/review moved to core-libs-dev and compiler-dev in March, here
> it is:
>
> http://download.java.net/java/jdk9/docs/api/javax/annotation/processing/Generated.html

This would seem to presume that the @Generated annotation is only ever
produced by annotation processors...

-- 
- DML

From scolebourne at joda.org  Wed Sep 20 12:58:12 2017
From: scolebourne at joda.org (Stephen Colebourne)
Date: Wed, 20 Sep 2017 13:58:12 +0100
Subject: java.lang.annotation.Generated
In-Reply-To: <bdac8fe8-814d-e98f-9573-98f242d9ff4b@oracle.com>
References: <CACzrW9CxoTMwwnGNa9JuL5vjRDDMRtS4HoPLM3BnYi8_DMdGtQ@mail.gmail.com>
 <bdac8fe8-814d-e98f-9573-98f242d9ff4b@oracle.com>
Message-ID: <CACzrW9AR-OiztRfXqfNRSfjO4CxRbDtdiacy78DEEJuXQ32P1A@mail.gmail.com>

Ouch. Thats an unpleasant result. It should have gone in `java.lang.annotation`.
Stephen

On 20 September 2017 at 13:49, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> On 20/09/2017 13:43, Stephen Colebourne wrote:
>>
>> As per this email:
>>
>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-February/011365.html
>> the idea was to add a new annotation `java.lang.annotation.Generated`
>> to replace the old problematic one.
>>
>> Is it my imagination, or did this get forgotten?:
>>
>
> The discussion/review moved to core-libs-dev and compiler-dev in March, here
> it is:
>
> http://download.java.net/java/jdk9/docs/api/javax/annotation/processing/Generated.html

From r.spilker at gmail.com  Wed Sep 20 13:00:27 2017
From: r.spilker at gmail.com (Roel Spilker)
Date: Wed, 20 Sep 2017 15:00:27 +0200
Subject: java.lang.annotation.Generated
In-Reply-To: <CACzrW9AR-OiztRfXqfNRSfjO4CxRbDtdiacy78DEEJuXQ32P1A@mail.gmail.com>
References: <CACzrW9CxoTMwwnGNa9JuL5vjRDDMRtS4HoPLM3BnYi8_DMdGtQ@mail.gmail.com>
 <bdac8fe8-814d-e98f-9573-98f242d9ff4b@oracle.com>
 <CACzrW9AR-OiztRfXqfNRSfjO4CxRbDtdiacy78DEEJuXQ32P1A@mail.gmail.com>
Message-ID: <CAO91Z4xANyVsMmA_yC5gDpEmNiTJ_xbqZM8y9eWaQe2LUVicwg@mail.gmail.com>

And possibly have retention "class"

On Sep 20, 2017 14:59, "Stephen Colebourne" <scolebourne at joda.org> wrote:

> Ouch. Thats an unpleasant result. It should have gone in
> `java.lang.annotation`.
> Stephen
>
> On 20 September 2017 at 13:49, Alan Bateman <Alan.Bateman at oracle.com>
> wrote:
> > On 20/09/2017 13:43, Stephen Colebourne wrote:
> >>
> >> As per this email:
> >>
> >> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-
> February/011365.html
> >> the idea was to add a new annotation `java.lang.annotation.Generated`
> >> to replace the old problematic one.
> >>
> >> Is it my imagination, or did this get forgotten?:
> >>
> >
> > The discussion/review moved to core-libs-dev and compiler-dev in March,
> here
> > it is:
> >
> > http://download.java.net/java/jdk9/docs/api/javax/annotation/processing/
> Generated.html
>

From scolebourne at joda.org  Wed Sep 20 13:04:50 2017
From: scolebourne at joda.org (Stephen Colebourne)
Date: Wed, 20 Sep 2017 14:04:50 +0100
Subject: java.lang.annotation.Generated
In-Reply-To: <CACzrW9AR-OiztRfXqfNRSfjO4CxRbDtdiacy78DEEJuXQ32P1A@mail.gmail.com>
References: <CACzrW9CxoTMwwnGNa9JuL5vjRDDMRtS4HoPLM3BnYi8_DMdGtQ@mail.gmail.com>
 <bdac8fe8-814d-e98f-9573-98f242d9ff4b@oracle.com>
 <CACzrW9AR-OiztRfXqfNRSfjO4CxRbDtdiacy78DEEJuXQ32P1A@mail.gmail.com>
Message-ID: <CACzrW9BmZJ-tAnPqhS2O5tbGDQpm9mhprpaA-510aVM5A++DGg@mail.gmail.com>

And just noting that the thread in March did not call out the fact
that the webrev did not correspond to the original agreed change from
Mark/jigsaw-dev. The only way you'd have spotted the package was
javax.annotation.processing and not java.lang.annotation is by
clicking on the webrev itself. Sorry, but I'm not impressed by this.

Stephen


On 20 September 2017 at 13:58, Stephen Colebourne <scolebourne at joda.org> wrote:
> Ouch. Thats an unpleasant result. It should have gone in `java.lang.annotation`.
> Stephen
>
> On 20 September 2017 at 13:49, Alan Bateman <Alan.Bateman at oracle.com> wrote:
>> On 20/09/2017 13:43, Stephen Colebourne wrote:
>>>
>>> As per this email:
>>>
>>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-February/011365.html
>>> the idea was to add a new annotation `java.lang.annotation.Generated`
>>> to replace the old problematic one.
>>>
>>> Is it my imagination, or did this get forgotten?:
>>>
>>
>> The discussion/review moved to core-libs-dev and compiler-dev in March, here
>> it is:
>>
>> http://download.java.net/java/jdk9/docs/api/javax/annotation/processing/Generated.html

From Alan.Bateman at oracle.com  Wed Sep 20 13:22:58 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 20 Sep 2017 14:22:58 +0100
Subject: java.lang.annotation.Generated
In-Reply-To: <CACzrW9AR-OiztRfXqfNRSfjO4CxRbDtdiacy78DEEJuXQ32P1A@mail.gmail.com>
References: <CACzrW9CxoTMwwnGNa9JuL5vjRDDMRtS4HoPLM3BnYi8_DMdGtQ@mail.gmail.com>
 <bdac8fe8-814d-e98f-9573-98f242d9ff4b@oracle.com>
 <CACzrW9AR-OiztRfXqfNRSfjO4CxRbDtdiacy78DEEJuXQ32P1A@mail.gmail.com>
Message-ID: <400afcf4-7be4-3127-fbf4-70e4e046e846@oracle.com>

On 20/09/2017 13:58, Stephen Colebourne wrote:
> Ouch. Thats an unpleasant result. It should have gone in `java.lang.annotation`.
> Stephen
>
That was the original suggestion but that package is more for 
annotations types that are used as meta annotations (@Native should have 
gone elsewhere but we can't change that now).

compiler-dev would be a better list to discuss this. As I recall, 
javax.annotation.processing was chosen in preference to 
java.lang.reflect or a new package for annotations.

-Alan.



From scolebourne at joda.org  Wed Sep 20 14:38:11 2017
From: scolebourne at joda.org (Stephen Colebourne)
Date: Wed, 20 Sep 2017 15:38:11 +0100
Subject: java.lang.annotation.Generated
In-Reply-To: <400afcf4-7be4-3127-fbf4-70e4e046e846@oracle.com>
References: <CACzrW9CxoTMwwnGNa9JuL5vjRDDMRtS4HoPLM3BnYi8_DMdGtQ@mail.gmail.com>
 <bdac8fe8-814d-e98f-9573-98f242d9ff4b@oracle.com>
 <CACzrW9AR-OiztRfXqfNRSfjO4CxRbDtdiacy78DEEJuXQ32P1A@mail.gmail.com>
 <400afcf4-7be4-3127-fbf4-70e4e046e846@oracle.com>
Message-ID: <CACzrW9Bq=QYRe4JC=dGy_5MUGBn+ukpfwdiJj5Rpf6nK5UitjA@mail.gmail.com>

On 20 September 2017 at 14:22, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> That was the original suggestion but that package is more for annotations
> types that are used as meta annotations (@Native should have gone elsewhere
> but we can't change that now).

A point acknowledged by
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-February/011365.html
and yet it still accepted it as the best location!

Basically, by not being in java.base it isn't useful to the people who
need it. No-one is going to pull in the java.compiler module just to
get this annotation. Jut like the original thread, I think the way
that this is used has been lost.

Stephen

From michael.rasmussen at zeroturnaround.com  Wed Sep 20 15:50:49 2017
From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen)
Date: Wed, 20 Sep 2017 18:50:49 +0300
Subject: java.lang.annotation.Generated
In-Reply-To: <CACzrW9CxoTMwwnGNa9JuL5vjRDDMRtS4HoPLM3BnYi8_DMdGtQ@mail.gmail.com>
References: <CACzrW9CxoTMwwnGNa9JuL5vjRDDMRtS4HoPLM3BnYi8_DMdGtQ@mail.gmail.com>
Message-ID: <CAPMkJWZ4Upe2bhXEwyESAn1TXY7KaQYx2xGhY47mCvAvt95F4g@mail.gmail.com>

This one?
http://download.java.net/java/jdk9/docs/api/javax/annotation/processing/Generated.html

/Michael

On Sep 20, 2017 18:44, "Stephen Colebourne" <scolebourne at joda.org> wrote:

> As per this email:
>  http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-
> February/011365.html
> the idea was to add a new annotation `java.lang.annotation.Generated`
> to replace the old problematic one.
>
> Is it my imagination, or did this get forgotten?:
>  http://download.java.net/java/jdk9/docs/api/java/lang/
> annotation/package-summary.html
>
> Stephen
>

From gunnar at hibernate.org  Thu Sep 21 20:04:01 2017
From: gunnar at hibernate.org (Gunnar Morling)
Date: Thu, 21 Sep 2017 22:04:01 +0200
Subject: Moving to Java 9 - module-info.class not found for <module-name>
 module
In-Reply-To: <686786261.2446055.1504352599651.JavaMail.zimbra@u-pem.fr>
References: <CAGe7fv36O6JsYERF=KB1sS=71wWvdHutHwhebkfpnhTKGjQpNQ@mail.gmail.com>
 <59A9CBC8.9000306@oracle.com>
 <1618721687.2391736.1504303158586.JavaMail.zimbra@u-pem.fr>
 <CAGe7fv2odN3Z1gF+xJ+tbXbSYXCBh2=dmCGEGrQfxYaru5ig0Q@mail.gmail.com>
 <686786261.2446055.1504352599651.JavaMail.zimbra@u-pem.fr>
Message-ID: <CADGJaX_602+mNBaiGM+_BNJ+xTZbzj29SJ_2zi+adFUDmQv4fg@mail.gmail.com>

> If you are in a hacky mood, there is a simple solution to transform an
automatic module to an explicit one,
> create a module-info.java, compile it and update the jar of the automatic
module with the module-info.class

I'm working on a Maven plug-in, ModiTect, which largely automates that
task. E.g. here is an example which adds the required descriptors to
Undertow (a web server) as well as its dependencies and then creates a
runtime image from that:


https://github.com/moditect/moditect/blob/master/integrationtest/undertow/pom.xml

Needless to say, that having the upstream project provide explicit modules
is the favourable approach.



2017-09-02 13:43 GMT+02:00 <forax at univ-mlv.fr>:

> > De: "Rahman USTA" <rahman.usta.88 at gmail.com>
> > ?: "Remi Forax" <forax at univ-mlv.fr>
> > Cc: "Alex Buckley" <alex.buckley at oracle.com>, "jigsaw-dev"
> > <jigsaw-dev at openjdk.java.net>
> > Envoy?: Samedi 2 Septembre 2017 13:30:24
> > Objet: Re: Moving to Java 9 - module-info.class not found for
> <module-name>
> > module
>
> > Thank you, Remi and Alex.
> > I think automatic modules can be supported by jlink. It could be a
> transition
> > feature like --illegal-access imho.
>
> The whole idea of jlink is that unlike classical Java it works under the
> assumption that the world is closed i.e. all the bytecodes that will be
> loaded is known, so it can do "whole world" optimizations like i know all
> the lambda forms that will be used by the string concatenation because i
> know all the call sites that does string concatenation so i can
> pre-generate them.
> If you have an automatic module, it can have dependencies on jars from the
> classpath, the close world assumption does not hold.
>
> If you are in a hacky mood, there is a simple solution to transform an
> automatic module to an explicit one, create a module-info.java, compile it
> and update the jar of the automatic module with the module-info.class or
> better ask the maintainer of the jar to do the work for you :)
>
> > Kind regards.
>
> regards,
> R?mi
>
> > 2017-09-02 0:59 GMT+03:00 Remi Forax < [ mailto:forax at univ-mlv.fr |
> > forax at univ-mlv.fr ] > :
>
> >> As Alex said,
> >> jackson.databind is not an explicit module.
>
> >> You should report that issue to their bugtracker.
>
> >> cheers,
> >> R?mi
>
> >> ----- Mail original -----
> >>> De: "Alex Buckley" < [ mailto:alex.buckley at oracle.com |
> alex.buckley at oracle.com
> >> > ] >
> >>> ?: "jigsaw-dev" < [ mailto:jigsaw-dev at openjdk.java.net |
> >> > jigsaw-dev at openjdk.java.net ] >
> >> > Envoy?: Vendredi 1 Septembre 2017 23:06:16
> >>> Objet: Re: Moving to Java 9 - module-info.class not found for
> <module-name>
> >> > module
>
> >> > On 9/1/2017 1:21 PM, Rahman USTA wrote:
> >> >> java --module-path
> >> >> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency
> --add-modules
> >> >> terminalfx -m terminalfx/com.terminalfx.AppStarter
>
> >> > (You shouldn't need the --add-modules, since terminalfx is already the
> >> > main module.)
>
> >> >> It works normally. Then, I want to generate a jlink image with the
> >> >> following script
>
> >> >> jlink --module-path
> >> >> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency
> --add-modules
> >> >> terminalfx --launcher terminalfx=terminalfx/com.
> terminalfx.AppStarter
> >> >> --output target/release
>
> >> >> However it gives me the following error;
>
> >> >> Error: module-info.class not found for jackson.databind module
>
> >> > I suspect jackson.databind is an automatic module. jlink does not
> >> > support linking of automatic modules because they can rely on the
> >> > arbitrary content of the classpath, which goes against the idea of a
> >> > self-contained Java runtime.
>
> >> > Alex
>
> > --
> > Rahman USTA
> > Istanbul JUG
> > [ https://github.com/rahmanusta | https://github.com/rahmanusta ]
>

From sanne at redhat.com  Thu Sep 21 22:24:21 2017
From: sanne at redhat.com (Sanne Grinovero)
Date: Thu, 21 Sep 2017 23:24:21 +0100
Subject: java.lang.annotation.Generated
In-Reply-To: <CAPMkJWZ4Upe2bhXEwyESAn1TXY7KaQYx2xGhY47mCvAvt95F4g@mail.gmail.com>
References: <CACzrW9CxoTMwwnGNa9JuL5vjRDDMRtS4HoPLM3BnYi8_DMdGtQ@mail.gmail.com>
 <CAPMkJWZ4Upe2bhXEwyESAn1TXY7KaQYx2xGhY47mCvAvt95F4g@mail.gmail.com>
Message-ID: <CAA5_-WaotWHJsuDsdCNvR_7_xgqM+U79_T0Xm0er2u+DS_SRHw@mail.gmail.com>

For people looking for an example solution, this is what we ended up doing
in Hibernate projects after we had shared these pain points in February:
add an explicit dependency to the Maven artifact
`javax.annotation:jsr250-api:1.0`.

We use an explicit phase for running annotation processors via the
maven-processor-plugin, so that it doesn't pollute published dependency
definitions.

https://github.com/hibernate/hibernate-search/blob/master/pom.xml#L996-L1001

HTH

Thanks,
Sanne


On Wed, Sep 20, 2017 at 4:50 PM, Michael Rasmussen <
michael.rasmussen at zeroturnaround.com> wrote:

> This one?
> http://download.java.net/java/jdk9/docs/api/javax/annotation/processing/
> Generated.html
>
> /Michael
>
> On Sep 20, 2017 18:44, "Stephen Colebourne" <scolebourne at joda.org> wrote:
>
> > As per this email:
> >  http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-
> > February/011365.html
> > the idea was to add a new annotation `java.lang.annotation.Generated`
> > to replace the old problematic one.
> >
> > Is it my imagination, or did this get forgotten?:
> >  http://download.java.net/java/jdk9/docs/api/java/lang/
> > annotation/package-summary.html
> >
> > Stephen
> >
>

From khmarbaise at gmx.de  Fri Sep 22 07:41:28 2017
From: khmarbaise at gmx.de (Karl Heinz Marbaise)
Date: Fri, 22 Sep 2017 09:41:28 +0200
Subject: module-info file location
Message-ID: <899dcbc2-4b15-306e-6f7a-58a32beb5dc6@gmx.de>

Hi,

I have seen an example of a multi version jar file which contained the 
mudule-info.class at the following location:

META-INF/versions/9/module-info.class

based on the information I have I would have assumed that this
is only allowed having the module-info.class file at the root of the jar.

So the question is: Is it allowed to have a module-info.class file in 
that location or is it only allowed having only a singe 
module-info.class file at the root level of a jar file?



Kind regards
Karl Heinz Marbaise

From alex.buckley at oracle.com  Fri Sep 22 07:54:10 2017
From: alex.buckley at oracle.com (Alex Buckley)
Date: Fri, 22 Sep 2017 00:54:10 -0700 (PDT)
Subject: module-info file location
Message-ID: <822492fb-d48b-474e-808c-de950a8c694a@default>

See "Modular multi-release JAR files" in http://openjdk.java.net/jeps/238

----- khmarbaise at gmx.de wrote:

> From: khmarbaise at gmx.de
> To: jigsaw-dev at openjdk.java.net
> Sent: Friday, September 22, 2017 12:43:38 AM GMT -08:00 US/Canada Pacific
> Subject: module-info file location
>
> Hi,
> 
> I have seen an example of a multi version jar file which contained the
> mudule-info.class at the following location:
> 
> META-INF/versions/9/module-info.class
> 
> based on the information I have I would have assumed that this
> is only allowed having the module-info.class file at the root of the
> jar.
> 
> So the question is: Is it allowed to have a module-info.class file in
> that location or is it only allowed having only a singe 
> module-info.class file at the root level of a jar file?
> 
> 
> 
> Kind regards
> Karl Heinz Marbaise

From Alan.Bateman at oracle.com  Fri Sep 22 07:57:36 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 22 Sep 2017 08:57:36 +0100
Subject: module-info file location
In-Reply-To: <899dcbc2-4b15-306e-6f7a-58a32beb5dc6@gmx.de>
References: <899dcbc2-4b15-306e-6f7a-58a32beb5dc6@gmx.de>
Message-ID: <9141eb7c-4353-46c4-8b3f-e2d8e33f603d@oracle.com>

On 22/09/2017 08:41, Karl Heinz Marbaise wrote:
> Hi,
>
> I have seen an example of a multi version jar file which contained the 
> mudule-info.class at the following location:
>
> META-INF/versions/9/module-info.class
>
> based on the information I have I would have assumed that this
> is only allowed having the module-info.class file at the root of the jar.
>
> So the question is: Is it allowed to have a module-info.class file in 
> that location or is it only allowed having only a singe 
> module-info.class file at the root level of a jar file?
(Ignoring your Reply-to header as it looks like a marketing address).

The "Modular multi-release JAR file" section in JEP 238 [1] has the 
details.

-Alan

[1] http://openjdk.java.net/jeps/238#Modular-multi-release-JAR-files



From jay.a at outlook.in  Fri Sep 22 09:44:55 2017
From: jay.a at outlook.in (Jayaprakash Artanareeswaran)
Date: Fri, 22 Sep 2017 09:44:55 +0000
Subject: ModuleElement#getEnclosedElements() is underdocumented
Message-ID: <DM5PR06MB3020F48E49ACBD9579222DC487670@DM5PR06MB3020.namprd06.prod.outlook.com>

Hello,


There's very little documentation on the newly added API - javax.lang.model.element.ModuleElement.getEnclosedElements(). All it says is this:


  "Returns the packages within this module."


Specifically, it doesn't mention when a package is considered to be enclosed in a module. For e.g., consider this case, where a Java module contains and exports a package with a single CU that has nothing but a package declaration statement.


module.one
  package.one
    package.one.one
      CU.java
        --
        // Nothing but a package declaration
        package package.one.one;
        --

Now during compilation of these sources, calling getEnclosedElements() on "module.one" returns a list that contains "package.one.one". So far so good. But compilation doesn't produce any .class files nor does it create a folder for "package.one.one". If you invoke the compiler again with the binaries and call getEnclosedElements(), Javac returns a list identical to our first iteration. Consistent alright, but in binary form, the package "package.one.one" doesn't point to any physical location nor does it contain any types. Is it enough that the package is found to be exported by the module?


What are the rules we must adhere to in order to implement this API?


Thanks,

Jay


From peter.levart at gmail.com  Fri Sep 22 14:11:16 2017
From: peter.levart at gmail.com (Peter Levart)
Date: Fri, 22 Sep 2017 16:11:16 +0200
Subject: Scanning multi version jars?
In-Reply-To: <cb013a55-b025-71c7-1c3c-2607d95b1c6c@oracle.com>
References: <CAAPGdfEuOnVK-5d3_7PLETB385mbq2aPqPqFSjr-6tbJXKDaOg@mail.gmail.com>
 <c25ef954-10f7-035b-b821-e712f8341cbf@oracle.com>
 <521049DD-F550-4A6C-B0BB-1FE6AED69A78@oracle.com>
 <7d4cddeb-4235-3a12-1ed9-cd5333275289@oracle.com>
 <CAAPGdfHB2fm-9JXms3ocTnEbN2VbJj6Nq7G4OQH6_-ohnA5Seg@mail.gmail.com>
 <6d8d2f5c-3aef-1c58-58cd-699dea012b90@oracle.com>
 <CAAPGdfHFnJ8P+pOt8ZUmQk1x06E4cKkwBCydEdn575G8uk6JnQ@mail.gmail.com>
 <00476a40-04bc-b1b5-678f-01f0ec4a07a9@oracle.com>
 <7570FD98-FD63-481D-A30A-72071CDF6050@oracle.com>
 <CAAPGdfH5EEOPMkB2ND1EJXZ3d+Ka9vjYU3jnu3JRifkXVorgBQ@mail.gmail.com>
 <ab055e07-0258-4876-99c8-2569f6e6623f@default>
 <CAAPGdfH_ZdnS_UYV+kXNdeQ4wNTZAMu8UfHYVsAHtK+BUVzwOA@mail.gmail.com>
 <a873b5eb-ad25-4b21-9127-58eb699b801f@default>
 <CAAPGdfGMRv-FPQw-pecPFiSedDp+SyeUV5FUuX-UDRubtbu=UA@mail.gmail.com>
 <cb013a55-b025-71c7-1c3c-2607d95b1c6c@oracle.com>
Message-ID: <951c84eb-30a5-14c5-5923-8ff61dd6f100@gmail.com>



On 09/19/2017 09:34 AM, Alan Bateman wrote:
> On 19/09/2017 05:37, Greg Wilkins wrote:
>> :
>>
>> Which raises an interesting point....?? with the multi versioned jar 
>> I have
>> used as an example, which contains:
>>
>> ??? - org/example/Foo.class
>> ??? - org/example/Foo$Bar.class
>> ??? - META-INF/versions/9/org/example/Foo.class
>>
>> What does the classloader do when asked to load "org.example.Foo$Bar" ?
>> ? If it loads it OK, then the JarFile enumerator/iterator/stream 
>> should also
>> return it.?? If it throws a ClassNotFoundException, then the
>> JarFile enumerator/iterator/stream should skip it.
> A class loader that loads from a JAR file will just map 
> "org.example.Foo$Bar" to entry "org/example/Foo$Bar.class" and attempt 
> to define the class bytes to VM.? It shouldn't care if the entry comes 
> from the base or a versioned section. It also shouldn't care if the 
> class name looks like it might have been compiled from an inner class.
>
> The one case where a custom class loader does need to know more is 
> when it loading resources (findResource/findResources implementation 
> usually). For that case then the returned URL needs to locate the 
> right resource and so may encode a path to a resource in a versioned 
> section. You'll see URLClassLoader does the right thing, as does the 
> built-in class loaders for cases where you have modular MR JARs on the 
> module path. There were a few threads on core-libs-dev discussing 
> whether to add a getRealName method but in the end it was kicked down 
> the road to re-examine later.
>
> -Alan

Just a though,

The only problem with ClassLoader API is that it doesn't have any means 
to enumerate all the resources it is able to resolve. If it had such 
API, the code for scanning would be much simpler.

But how would such API look like if we know that resolving works by 
delegation, might work by lazily generating the resource with a 
particular path on the fly, etc... Perhaps simply by ignoring the 
delegation and lazy generation and just return a stream of resources 
that a particular ClassLoader instance is responsible for and are backed 
by real bytes in some repository which has a means of enumeration (like 
filesystem, jar file, etc...)

Regards, Peter


From sadhak001 at gmail.com  Fri Sep 22 14:21:04 2017
From: sadhak001 at gmail.com (Mani Sarkar)
Date: Fri, 22 Sep 2017 14:21:04 +0000
Subject: Become an early Java 9 expert: AJUG + vJUG + JUGs Worldwide
 Hackday Feedback on JDK 9 EA
In-Reply-To: <59C1CA52.6030504@oracle.com>
References: <CAGHtMWmqeVt5ORGGuYo7EDp-7whA2Dogos_hd3SGh1s_y2_c-Q@mail.gmail.com>
 <59C1CA52.6030504@oracle.com>
Message-ID: <CAGHtMWkZdsC_4scD-Uk51o-aGdeqA4wBuwtKzJbixFDM9YdEYA@mail.gmail.com>

Thanks Alex for the response, and clarifying the question further with your
answer.

There a few issues reported as well, would you want to comment on them,
should I put them on a separate thread for further discussions?

Cheers,
Mani


On Wed, 20 Sep 2017 at 02:54 Alex Buckley <alex.buckley at oracle.com> wrote:

> Hi Mani,
>
> On 9/19/2017 2:55 PM, Mani Sarkar wrote:
> > Last month (19th August 2017) AJUG and a number of JUGs worldwide with
> the
> > help and support from vJUG, re-ran the  "Become an early Java 9 expert"
> > hackday.
>
> Thank you AJUG and vJUG!
>
> > You can find the feedback gathered in
> http://bit.ly/J9HackDay-AJUG-feedback,
> > we have been trailing JDK 9 EA b181 (RC1).
>
> A lot of feedback seems to boil down to "JDK command line tools are not
> so easy to use; I want my IDE!". I don't mean to make light of people's
> usability issues, but the module-related paths and flags in JDK 9 tools
> tend to operate along similar lines as the paths and flags in JDK 8
> tools -- it's just that a lot of people haven't set ANY paths and flags
> for a long time.
>
> I see there were some more open-ended questions and this one in
> particular caught my attention:
>
> -----
> Do I need to convert a legacy Java program to use named modules in order
> to take advantage of the smaller images that jlink can create?
>
> Mani: You will have to convert your applications to use Java 9?s modules
> system in order to take advantage of JLink fully, although please play
> around with older legacy apps to see what JLink produces (most likely
> the whole JDK and not modularised pieces).
>
> Simon: create an empty module with module-info.java and handcraft the
> dependencies using requires and have jlink compile it. This is
> experimental, would need to be tested to see how it works.
> -----
>
> The direct answer to the question is "No, you do not need to convert a
> legacy Java program to use named modules in order to take advantage of
> the smaller images that jlink can create."
>
> The Java runtime that's present in even the smallest image (just
> java.base) still lets you to put your pre-existing JAR files on the
> classpath and run them with java -cp. You do not need to turn your JAR
> files into named modules. Even as traditional JAR files, they have
> access to all the APIs that you would expect from such an image.
> (Obviously if your JAR files try to use Swing on an image built from
> just java.base, that won't work.) The reduced footprint and security
> surface of the smaller image is plainly an advantage from jlink.
>
> Alex
>
-- 
@theNeomatrix369 <http://twitter.com/theNeomatrix369>*  |  **Blog
<http://neomatrix369.wordpress.com/>**  |  *@adoptopenjdk | Dev communities
*Meet-a-Project - *MutabilityDetector
<https://github.com/MutabilityDetector>*  |  **Bitbucket
<https://bitbucket.org/neomatrix369>* * |  **Github
<https://github.com/neomatrix369>* * |  **LinkedIn
<http://uk.linkedin.com/pub/mani-sarkar/71/a77/39b>*
*Come to Devoxx UK 2018:* http://www.devoxx.co.uk/

*Don't chase success, rather aim for "Excellence", and success will come
chasing after you!*

From mark.reinhold at oracle.com  Fri Sep 22 20:28:18 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Fri, 22 Sep 2017 13:28:18 -0700 (PDT)
Subject: Project Jigsaw: Complete!
Message-ID: <20170922202818.34110C9EBD@eggemoggin.niobe.net>

As you all probably know by now, yesterday we shipped the GA release of
JDK 9, and so at last the Jigsaw is complete.  I wrote a bit more about
this milestone on my blog:

  https://mreinhold.org/blog/jigsaw-complete

I've updated the project page (http://openjdk.java.net/projects/jigsaw/),
and also all the JEPs, to reflect the final state of things.  If anything
is missing or incorrect, please let me know.  (I'm still working on text
to describe the JDK-specific class-file attributes `ModuleHashes` and
`ModuleResolution`.)

My thanks to everyone who contributed their experiences, insights, and
suggestions here on jigsaw-dev.  That feedback was critical to the final
result, which lays a strong foundation for the future of Java.

- Mark

From alexjecan88 at yahoo.com  Tue Sep 26 13:37:18 2017
From: alexjecan88 at yahoo.com (Alexandru Jecan)
Date: Tue, 26 Sep 2017 15:37:18 +0200
Subject: Compile Jigsaw project with Intellij IDEA supported structure
Message-ID: <000e01d336cc$93a52ba0$baef82e0$@yahoo.com>

Hello,

 

In order to make my Jigsaw project be supported by Intellij IDEA 2017.2, I need a structure like this, according to IDEA Video:  https://www.youtube.com/watch?v=WL48zkLvK3I (00:10):

 

com.myModule.m1

                            /src

                                  /module-info.java

                                 /com.myModule.m2

                                                                  /Main.java

 

com.myModule.m2

                            /src

                                  /module-info.java

                                 /com.myModule.m2

                                                                  /Main.java

 

 

How do I perform multi-module compilation in this case using the --module-source-path option?

 

As far as I know, to perform multi-module compilation, the source code directory should have the same name like the name of the module.

In this case, I have the ?src? folder there which does not comply to this rule.

 

 

Best regards

 

Alexandru Jecan


From jay.a at outlook.in  Tue Sep 26 18:08:24 2017
From: jay.a at outlook.in (Jayaprakash Artanareeswaran)
Date: Tue, 26 Sep 2017 18:08:24 +0000
Subject: ModuleElement#getEnclosedElements() is underdocumented
In-Reply-To: <DM5PR06MB3020F48E49ACBD9579222DC487670@DM5PR06MB3020.namprd06.prod.outlook.com>
References: <DM5PR06MB3020F48E49ACBD9579222DC487670@DM5PR06MB3020.namprd06.prod.outlook.com>
Message-ID: <DM5PR06MB3020690CAB2B26A694EE72EF877B0@DM5PR06MB3020.namprd06.prod.outlook.com>

After Mark's announcement about project Jigsaw's completion, I don't see any further discussion on this mailing list. Particularly, there's not been any response to the question I posted on September 22.


So, is this still the right medium to raise any Java 9 related questions?


Regards,

Jay

________________________________
From: Jayaprakash Artanareeswaran
Sent: Friday, September 22, 2017 9:44 AM
To: jigsaw-dev at openjdk.java.net
Subject: ModuleElement#getEnclosedElements() is underdocumented


Hello,


There's very little documentation on the newly added API - javax.lang.model.element.ModuleElement.getEnclosedElements(). All it says is this:


  "Returns the packages within this module."


Specifically, it doesn't mention when a package is considered to be enclosed in a module. For e.g., consider this case, where a Java module contains and exports a package with a single CU that has nothing but a package declaration statement.


module.one
  package.one
    package.one.one
      CU.java
        --
        // Nothing but a package declaration
        package package.one.one;
        --

Now during compilation of these sources, calling getEnclosedElements() on "module.one" returns a list that contains "package.one.one". So far so good. But compilation doesn't produce any .class files nor does it create a folder for "package.one.one". If you invoke the compiler again with the binaries and call getEnclosedElements(), Javac returns a list identical to our first iteration. Consistent alright, but in binary form, the package "package.one.one" doesn't point to any physical location nor does it contain any types. Is it enough that the package is found to be exported by the module?


What are the rules we must adhere to in order to implement this API?


Thanks,

Jay


From sander.mak at luminis.eu  Tue Sep 26 19:37:14 2017
From: sander.mak at luminis.eu (Sander Mak)
Date: Tue, 26 Sep 2017 19:37:14 +0000
Subject: ServiceLoader and ModuleLayer
Message-ID: <C6D023DB-A45C-41FA-B011-938798483D7D@luminis.eu>

I'm currently running into an issue that behaves unexpected as far as I can see. Let's say there are two service types, A and B. The module `main` in the boot layer has a uses constraint on A. Module `main` instantiates a new ModuleLayer with the following code:

      ModuleFinder finder = ModuleFinder.of(dir.toPath());

      ModuleLayer parent = ModuleLayer.boot();
      Configuration cf = parent.configuration()
        .resolveAndBind(finder, ModuleFinder.of(), Set.of());

      ClassLoader scl = ClassLoader.getSystemClassLoader();
      ModuleLayer layer = parent.defineModulesWithOneLoader(cf, scl);

      // Now use new A providers from the layer:
      ServiceLoader.load(layer, A.class).forEach(...)

When `dir` contains a single provider module that provides an implementation of A, this works fine. What doesn't work, is if I have a directory with a provider module providing an A implementation, where this A implementation in turn has a uses constraint on B. When I check `layer.modules()`, I can see that the B provider modules do get resolved into the layer (they're also in `dir`). However, `ServiceLoader.load(B.class)`, which is part of the A service implementation code, returns no instances. How can I make sure the B service providers are bound as well within the layer?

To answer my own question, after some thinking and experimentation I found out that the A implementation should use the following code:

     ServiceLoader.load(getClass().getModule().getLayer(), B.class)

So... should I start writing all my ServiceLoader calls this way from now on, if I want to make sure my modules work regardless of whether they're loaded in the boot layer or another layer? Wouldn't it be more logical for ServiceLoader to always work from the current layer (hm, that would probably break the current classloader based contract)? Am I missing another option?


Sander




From Alan.Bateman at oracle.com  Tue Sep 26 20:44:17 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 26 Sep 2017 21:44:17 +0100
Subject: ServiceLoader and ModuleLayer
In-Reply-To: <C6D023DB-A45C-41FA-B011-938798483D7D@luminis.eu>
References: <C6D023DB-A45C-41FA-B011-938798483D7D@luminis.eu>
Message-ID: <12dc301f-66e7-5b16-9c6f-ca1fcb08834c@oracle.com>

On 26/09/2017 20:37, Sander Mak wrote:
> I'm currently running into an issue that behaves unexpected as far as I can see. Let's say there are two service types, A and B. The module `main` in the boot layer has a uses constraint on A. Module `main` instantiates a new ModuleLayer with the following code:
>
>        ModuleFinder finder = ModuleFinder.of(dir.toPath());
>
>        ModuleLayer parent = ModuleLayer.boot();
>        Configuration cf = parent.configuration()
>          .resolveAndBind(finder, ModuleFinder.of(), Set.of());
>
>        ClassLoader scl = ClassLoader.getSystemClassLoader();
>        ModuleLayer layer = parent.defineModulesWithOneLoader(cf, scl);
>
>        // Now use new A providers from the layer:
>        ServiceLoader.load(layer, A.class).forEach(...)
>
> When `dir` contains a single provider module that provides an implementation of A, this works fine.
Yes, you've provided the child layer as the starting point so it will 
load the A providers in that layer, then the A providers in the parent 
layer. An alternative would be to specify the class loader for any 
module in the layer (doesn't matter which one although there is only one 
in your example).

> What doesn't work, is if I have a directory with a provider module providing an A implementation, where this A implementation in turn has a uses constraint on B. When I check `layer.modules()`, I can see that the B provider modules do get resolved into the layer (they're also in `dir`). However, `ServiceLoader.load(B.class)`, which is part of the A service implementation code, returns no instances. How can I make sure the B service providers are bound as well within the layer?
The 1-arg load method uses the TCCL as the starting point and it's 
probably the application class loader in your case. In container 
environments where there is a TCCL per application then the 1-arg load 
method works well.

>
> To answer my own question, after some thinking and experimentation I found out that the A implementation should use the following code:
>
>       ServiceLoader.load(getClass().getModule().getLayer(), B.class)
>
> So... should I start writing all my ServiceLoader calls this way from now on, if I want to make sure my modules work regardless of whether they're loaded in the boot layer or another layer? Wouldn't it be more logical for ServiceLoader to always work from the current layer (hm, that would probably break the current classloader based contract)? Am I missing another option?
There's any notion of "current layer" and not clear that it would be 
useful. To explain why, assume that the A service type in your example 
is one of the service types that the java.xml module uses. Now suppose 
that some code in the child layer invokes an API in the java.xml module 
to parse a document. In that scenario you have code in the boot layer 
loading service providers in a child layer. If the XML code only used 
the "current layer" then it would never load providers in child layers.

However to your question, then specifying a starting point (be it layer 
or class loader) may be needed to allow libraries be used in complicated 
environments.

-Alan.

From trisha.gee at gmail.com  Wed Sep 27 12:09:06 2017
From: trisha.gee at gmail.com (Trisha Gee)
Date: Wed, 27 Sep 2017 12:09:06 +0000
Subject: Compile Jigsaw project with Intellij IDEA supported structure
In-Reply-To: <mailman.57.1506513602.3756.jigsaw-dev@openjdk.java.net>
References: <mailman.57.1506513602.3756.jigsaw-dev@openjdk.java.net>
Message-ID: <CABFwzdNG6-YzHiEGyK5BCwfVPZ-rZpNqhPKpPqabQWazu27gMg@mail.gmail.com>

Hi Alexandru,

The structure in IntelliJ IDEA isn't as strict as that, it doesn't require
you to have a "src" folder.  For example you could have:

com.myModule.m1
    /module-info.java
    /com.myModule.m2
        /Main.java

com.myModule.m2
    /module-info.java
    /com.myModule.m2
        /Main.java

You just need to mark com.myModule.m1 and com.myModule.m2 as your sources
root instead of having a src folder as the sources root.

https://www.jetbrains.com/help/idea/configuring-content-roots.html#d85322e277

Trisha

---------- Forwarded message ---------
Date: Tue, 26 Sep 2017 15:37:18 +0200
From: "Alexandru Jecan" <alexjecan88 at yahoo.com>
To: <jigsaw-dev at openjdk.java.net>
Subject: Compile Jigsaw project with Intellij IDEA supported structure
Message-ID: <000e01d336cc$93a52ba0$baef82e0$@yahoo.com>
Content-Type: text/plain;       charset="utf-8"

Hello,



In order to make my Jigsaw project be supported by Intellij IDEA 2017.2, I
need a structure like this, according to IDEA Video:
https://www.youtube.com/watch?v=WL48zkLvK3I (00:10):



com.myModule.m1

                            /src

                                  /module-info.java

                                 /com.myModule.m2

                                                                  /Main.java



com.myModule.m2

                            /src

                                  /module-info.java

                                 /com.myModule.m2

                                                                  /Main.java





How do I perform multi-module compilation in this case using the
--module-source-path option?



As far as I know, to perform multi-module compilation, the source code
directory should have the same name like the name of the module.

In this case, I have the ?src? folder there which does not comply to this
rule.





Best regards



Alexandru Jecan

From forax at univ-mlv.fr  Wed Sep 27 12:37:00 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Wed, 27 Sep 2017 12:37:00 +0000
Subject: Compile Jigsaw project with Intellij IDEA supported structure
In-Reply-To: <CABFwzdNG6-YzHiEGyK5BCwfVPZ-rZpNqhPKpPqabQWazu27gMg@mail.gmail.com>
References: <mailman.57.1506513602.3756.jigsaw-dev@openjdk.java.net>
 <CABFwzdNG6-YzHiEGyK5BCwfVPZ-rZpNqhPKpPqabQWazu27gMg@mail.gmail.com>
Message-ID: <EB5F9AC6-AA8A-4800-B23F-27F6CA4ED477@univ-mlv.fr>

And you can use the same trick with Eclipse too that said the Eclipse compiler does not support the multi-module layout (only one module by project).

R?mi


On September 27, 2017 2:09:06 PM GMT+02:00, Trisha Gee <trisha.gee at gmail.com> wrote:
>Hi Alexandru,
>
>The structure in IntelliJ IDEA isn't as strict as that, it doesn't
>require
>you to have a "src" folder.  For example you could have:
>
>com.myModule.m1
>    /module-info.java
>    /com.myModule.m2
>        /Main.java
>
>com.myModule.m2
>    /module-info.java
>    /com.myModule.m2
>        /Main.java
>
>You just need to mark com.myModule.m1 and com.myModule.m2 as your
>sources
>root instead of having a src folder as the sources root.
>
>https://www.jetbrains.com/help/idea/configuring-content-roots.html#d85322e277
>
>Trisha
>
>---------- Forwarded message ---------
>Date: Tue, 26 Sep 2017 15:37:18 +0200
>From: "Alexandru Jecan" <alexjecan88 at yahoo.com>
>To: <jigsaw-dev at openjdk.java.net>
>Subject: Compile Jigsaw project with Intellij IDEA supported structure
>Message-ID: <000e01d336cc$93a52ba0$baef82e0$@yahoo.com>
>Content-Type: text/plain;       charset="utf-8"
>
>Hello,
>
>
>
>In order to make my Jigsaw project be supported by Intellij IDEA
>2017.2, I
>need a structure like this, according to IDEA Video:
>https://www.youtube.com/watch?v=WL48zkLvK3I (00:10):
>
>
>
>com.myModule.m1
>
>                            /src
>
>                                  /module-info.java
>
>                                 /com.myModule.m2
>
>                                                             /Main.java
>
>
>
>com.myModule.m2
>
>                            /src
>
>                                  /module-info.java
>
>                                 /com.myModule.m2
>
>                                                             /Main.java
>
>
>
>
>
>How do I perform multi-module compilation in this case using the
>--module-source-path option?
>
>
>
>As far as I know, to perform multi-module compilation, the source code
>directory should have the same name like the name of the module.
>
>In this case, I have the ?src? folder there which does not comply to
>this
>rule.
>
>
>
>
>
>Best regards
>
>
>
>Alexandru Jecan

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.