Scanning multi version jars?

Tue Sep 19 03:41:19 UTC 2017

Thanks for the clarification – I overstated the “any JarEntry”.

I didn’t look at VersionedStream so I now understand the limitations you mention.

In my case, it’s necessary to look at all files in the jar file to do the elimination of unneeded ordinary/inner classes so JarInputStream getNextJarEntry() can be used.

By using the versioned JarFile constructor, getting the JarEntry returns the right one for processing.  If I needed further filtering on the file names, I’d need to return the real file names.

Maybe the use case isn’t so universal or well defined.

From: Greg Wilkins [mailto:gregw at webtide.com] 
Sent: Monday, September 18, 2017 9:33 PM
To: Stephen Felts <stephen.felts at oracle.com>
Cc: Paul Sandoz <paul.sandoz at oracle.com>; jigsaw-dev <jigsaw-dev at openjdk.java.net>; core-libs-dev at openjdk.java.net
Subject: Re: Scanning multi version jars?

Stephen,

It is not the case that the getName() always returns the path starting with "META-INF/versions/". Specifically, if the entry is obtained from getJarEntry() API (and not from the enumerator), then the name is that of the unversioned file, but the metadata and contents obtained using the jarEntry are for the versioned entry.

For example the following code:

JarFile jarFile = new JarFile(new File("/tmp/example.jar"),
                              false,
                              JarFile.OPEN_READ,
                              Runtime.version());
JarEntry entry = jarFile.getJarEntry("org/example/OnlyIn9.class");
System.err.printf("%s -> %s%n",entry.getName(),IO.toString(jarFile.getInputStream(entry)));

when run against a jar where the class files contain just the text of their full path produces the following output:

org/example/OnlyIn9.class -> META-INF/versions/9/org/example/OnlyIn9.class

There is nothing in the public API of the JarEntry so obtained that indicates that it the versioned entry, nor can I distinguish it from an entry obtained by iteration that may report the same name (if the entry was also in the base), although at least equals does return false.

Moreover, the proposed stream API as represented by the current implementation of jdk.internal.util.jar.VersionedStream, applies some filtering based on the versioning and then converts it's enumerated JarEntry instances to opaquely versioned JarEntry instances by calling map(jf::getJarEntry),which thus hides the version information and makes any additional filtering based on version impossible by any users of that stream.

regards

On 19 September 2017 at 11:05, Stephen Felts <HYPERLINK "mailto:stephen.felts at oracle.com"stephen.felts at oracle.com> wrote:

A versioned file name, JarEntry.getName(), starts with "META-INF/versions/".
The version is the following string up to the next "/".
The version can be parsed with Runtime.Version.parse().
If not a versioned class file name, then use Jarfile.baseVersion().
That should be sufficient to get the version for any JarEntry.

If it needs to run on pre-JDK9, this needs a lot of reflection.

IMO Having a method that behaves as described below is likely to be needed for many use cases and it would be good if someone wrote it and put it in a well-known, public jar file.

-----Original Message-----
From: Greg Wilkins [mailto:HYPERLINK "mailto:gregw at webtide.com"gregw at webtide.com]
Sent: Monday, September 18, 2017 8:19 PM
To: Paul Sandoz <HYPERLINK "mailto:paul.sandoz at oracle.com"paul.sandoz at oracle.com>
Cc: jigsaw-dev <HYPERLINK "mailto:jigsaw-dev at openjdk.java.net"jigsaw-dev at openjdk.java.net>; HYPERLINK "mailto:core-libs-dev at openjdk.java.net"core-libs-dev at openjdk.java.net
Subject: Re: Scanning multi version jars?

Paul,

yeh... I guess I concede it's not JarFiles job... as much as that would make things easier for containers to reach agreement:(

However, can we at least look at having a new default method on JarEntry to query the version. Without that, containers don't have the information available to perform the semantic filtering required and thus will not be able to use the stream API and will have to work from an unversioned stream.

regards

On 19 September 2017 at 03:04, Paul Sandoz <HYPERLINK "mailto:paul.sandoz at oracle.com"paul.sandoz at oracle.com> wrote:

> I agree with Alan here, we should not be pushing a semantic
> understanding of inner classes into JarFile.
>
> I do sympathise with the case of annotation class scanning, which has
> always tunnelled through the class loader view to directly get at
> class file bytes possibly dealing with various URI schemes, since that
> is currently the only effective way of accessing the required
> information in an efficient manner.
>
> As Alan mentioned we should add a traversable versioned view of a
> JarFile, returning a Stream, from which it should be possible to
> filter according to certain semantics.
>
> Paul.
>
>
> > On 17 Sep 2017, at 12:27, Alan Bateman <HYPERLINK "mailto:Alan.Bateman at oracle.com"Alan.Bateman at oracle.com> wrote:
> >
> > On 15/09/2017 22:58, Greg Wilkins wrote:
> >> :
> >>
> >>  * I think the stream needs to handle inner classes and only include
> >>    them if their matching outerclass is available at the same
> >>    version.  So for example a base Foo$Bar.class will only be
> >>    included if the stream includes a base Foo.class, and it will not
> >>    be included if the Foo.class is version 9 or above.  Likewise a
> >>    version 9 Foo$Bar.class will only be included in the stream if the
> >>    stream also includes a version 9 Foo.class, and will not be
> >>    included if the stream has a version 10 or above Foo.class
> >>
> >> If you think this last point is possible, then I'll move the
> >> discussion
> back the EE expert groups to try to get an agreement on the exact
> stream code that will be used in the mid term until it is available in
> the JRE lib, at which time the specs should be amended to say they
> will defer the decision of which classes to scan the JRE lib so they
> will be future proof for any changes in java 10, 11 etc.
> >>
> > I don't think this should be pushed down to the JarFile API. The
> > JarFile
> API provides the base API for accessing JAR files and should not be
> concerned with the semantics or relationship between entries. I agree
> that annotation scanning tools and libraries need to do additional
> work to deal with orphaned or menacing inner classes in a MR JAR but
> it's not too different to arranging a class path with a JAR file
> containing the "classes for JDK 9" ahead of a JAR file containing the
> version of the library that runs on JDK 8. I do think that further
> checks could be done by the `jar` tool to identify issues at packaging time.
> >
> > -Alan
>
>

--
Greg Wilkins <HYPERLINK "mailto:gregw at webtide.com"gregw at webtide.com> CTO http://webtide.com

-- 

Greg Wilkins <HYPERLINK "mailto:gregw at webtide.com"gregw at webtide.com> CTO http://webtide.com