RFR: 6957241: ClassLoader.getResources() returns only 1 instance when using jar indexing

[Alan Bateman]

On Thu, 2 Sep 2021 11:43:46 GMT, Alan Bateman <alanb at openjdk.org> wrote:

>> Using jarIndex for Hibench, there is an unexpected behavior with the exception "Exception in thread "main" org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "hdfs"".
>> 
>> After investigating it, it is related to the usage of ServiceLoader with JarIndex.
>> The below stack shows the issue with JDK11:
>> 
>> getResource:1016, URLClassPath$JarLoader (jdk.internal.loader)
>> getResource:937, URLClassPath$JarLoader (jdk.internal.loader)
>> findResource:912, URLClassPath$JarLoader (jdk.internal.loader)
>> next:341, URLClassPath$1 (jdk.internal.loader)
>> hasMoreElements:351, URLClassPath$1 (jdk.internal.loader)
>> hasNext:355, BuiltinClassLoader$1 (jdk.internal.loader)
>> hasMoreElements:363, BuiltinClassLoader$1 (jdk.internal.loader)
>> next:3032, CompoundEnumeration (java.lang)
>> hasMoreElements:3041, CompoundEnumeration (java.lang)
>> nextProviderClass:1203, ServiceLoader$LazyClassPathLookupIterator (java.util)
>> hasNextService:1221, ServiceLoader$LazyClassPathLookupIterator (java.util)
>> hasNext:1265, ServiceLoader$LazyClassPathLookupIterator (java.util)
>> hasNext:1300, ServiceLoader$2 (java.util)
>> hasNext:1385, ServiceLoader$3 (java.util)
>> 
>> The below API tries to get all the resources with the same name.
>> 
>> public Enumeration<URL> findResources(final String name,
>>                                      final boolean check) 
>>  ```
>> After using JarIndex, URLClassPath.findResources only returns 1 URL.
>> It is the same as the description in JDK-6957241.
>> 
>> The issue still exists in JDK18.
>> 
>> Root cause:
>> 
>> public Enumeration<URL> findResources(final String name,
>>                                      final boolean check) {
>>         return new Enumeration<>() {
>>             private int index = 0;
>>             private URL url = null;
>> 
>>             private boolean next() {
>>                 if (url != null) {
>>                     return true;
>>                 } else {
>>                     Loader loader;
>>                     while ((loader = getLoader(index++)) != null) {
>>                         url = loader.findResource(name, check);
>>                         if (url != null) {
>>                             return true;
>>                         }
>>                     }
>>                     return false;
>>                 }
>>             }
>> ...
>>         };
>>     }
>> 
>> With the JarIndex, there is only one loader which is corresponding to the jar with the index due to the implementation in JarLoader.getResource(final String name, boolean check, Set<String> visited).
>> 
>> Loaders corresponding to other jar packages will not appear in this while.
>> So it only returns 1 instance.
>> 
>> To solve the issue, I change the implementation "private boolean next()".
>> If the loader has index, traverse the index and get all the resource from the loader.
>
> @wxiang I think there is at least some support for removing the JAR indexing support rather than trying to fix findResources. The issue of what to do with the legacy JAR index mechanism came up during JDK 9 in the context of modular JARs and also Multi-Release JARs but it was too much to take on at the time. Would you be interested in working out the changes to remove it?

> @AlanBateman Sure, I am interested in it.

Great! I think there are several parts to this. The removal of the JAR index support from the URLClassLoader implementation, the `jar i` option, the JAR file spec, and the jar tool man page.

It would be good to create a patch for the removal to see if there are any issues. There will probably need to be some discussion on what to do with the jar tool. I suspect we will need to keep the code that updates the index when updating a JAR file that has an existing index, this means keeping JarIndex and maybe moving it to the jdk.jartool module. We can change `jar i` to print a warning when the tool is called to generate an index. Don't worry about the JAR spec and man page for now.

-------------

PR: https://git.openjdk.java.net/jdk/pull/5316