Jigsaw classpath mode support

[Paul Sandoz]

Hi Mandy,

This looks good to me, although i admit to not fully understanding the interactions between the Java layer and the VM layer.

IMHO, given that the test results are so good, this patch should be pushed early and iterated on to refine/improve. That way we all get exposure to it.

Paul.

--

Looks like the Library interface may need to be updated so that:

    public abstract URI findLocalResource(ModuleId mid, String rn)

returns a URI plus some meta-data e.g. size.

--

Definite room for optimisation in ClassPathContext, for example if we know the resource is a class or using some reverse lookup of resource path name.

--

FWIW in sun.misc.Launcher you can replace lines 215-224 with the two following lines:

  prepaths = urls.subList(0, index).toArray(new URL[index]);
  postpaths = urls.subList(index, urls.size()).toArray(new URL[urls.size() - index]);

Plus Vector could probably be replaced with List. 
 189         // keep prepaths and postpaths for native library search
 190         private static URL[] prepaths;
 191         private static URL[] postpaths;
 192         static private URL[] getExtURLs(File[] dirs) throws IOException {
 193             File defaultExtDir = EXTENSION.defaultPath();
 194             Vector<URL> urls = new Vector<URL>();
 195             int index = 0;
 196             for (int i = 0; i < dirs.length; i++) {
 197                 File dir = dirs[i];
 198                 String[] files = dir.list();
 199                 if (files != null) {
 200                     for (int j = 0; j < files.length; j++) {
 201                         if (!files[j].equals("meta-index")) {
 202                             File f = new File(dir, files[j]);
 203                             urls.add(getFileURL(f));
 204                         }
 205                     }
 206                 }
 207                 if (dir.exists() && defaultExtDir != null &&
 208                         Files.isSameFile(defaultExtDir.toPath(), dir.toPath())) {
 209                     // set index to urls that dirs[i] is the default ext dir
 210                     // so that prepaths and postpaths can be determined.
 211                     index = urls.size();
 212                     urls.add(getFileURL(dir));
 213                 }
 214             }
 215             URL[] ua = new URL[urls.size()];
 216             urls.copyInto(ua);
 217             prepaths = new URL[index];
 218             postpaths = new URL[urls.size() - index];
 219             for (int i=0, j=0; i < ua.length; i++) {
 220                 if (i < index)
 221                     prepaths[i] = ua[i];
 222                 else
 223                     postpaths[j++] = ua[i];
 224             }
 225             return ua;
 226         }

On Aug 7, 2012, at 7:47 PM, Mandy Chung <mandy.chung at oracle.com> wrote:

> I have implemented the "classpath mode" support running on
> a module image.
> 
> Webrev:
>  http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.00/
> 
> JDK has been modularized in the jigsaw repo.  There no longer
> has a rt.jar, tools.jar, or any jar file.  The system classes
> are now installed in a system module library as modules. This
> change will allow the bootstrap class loader, extension class
> loader, and system class loader to search classes and resources
> in the system module library as the default.  Some implementation
> details:
> 
> 1. VM used to maintain the default bootclasspath list and search
> classes in it.  Now it will call jigsaw native interface to find
> classes from the module library unless -Xbootclasspath is set
> to override the default bootclasspath.  The -Xbootclasspath/p
> and -Xbootclasspath/a will continue to work as it is today.
> Karen will be sending out the webrev for the hotspot change
> separately.
> 
> 2. tools.jar
> 
> For compatibility, -classpath tools.jar should continue
> to work. There are existing applications using URLClassLoader
> that loads classes from tools.jar (e.g. test harness) that
> also should continue to work.
> 
> To be able to determine if the path of the given tools.jar
> to either -classpath option or URL parameter to URLClassLoader
> is the same path as JAVA_HOME/lib/tools.jar in a lightweight,
> the module image for the full JDK will contain an empty tools.jar.
> This also allows applications that check for the existence of
> tools.jar to continue to work.
> 
> sun.misc.URLClassPath is the main implementation class to
> support URLClassLoader to search a file from the given URLs.
> Both ExtClassLoader and AppClassLoader are URLClassLoader.
> URLClassPath is extended to find resource file from the
> appropriate system modules if the given URL is
> JAVA_HOME/lib/tools.jar or JAVA_HOME/lib/ext.
> 
> 3. How to determine which modules belong to bootclasspath,
> ext directory, and tools.jar (and a few jar files in JDK/lib)
> at runtime
> 
> A configuration for the classpath mode support is loaded at
> startup rather than generated to minimize the startup cost.
> The idea is to have one ClassPathContext for bootclasspath,
> one for extension and one for the jdk tools.
> 
> In this prototype, I just define a new module "jdk.classpath"
> and install it in the module image.  It hardcodes the list
> of modules for each ClassPathContext for now.  One alternative
> is to extend jpkg to generate "jdk.classpath" config specifically
> for this classpath mode.  We'll explore this later.  But I
> think this is good enough for now.
> 
> 4. I made simple change in StoredConfiguration to reduce
> the footprint of the configuration stored on disk. There
> were several regression tests setting -Xmx failed due to
> the increase of the startup footprint without this change.
> 
> With storing the context names and package names only once,
> it reduces the size of the config with 35-40%.  I suspect
> the fast configuration will make similar change.
> 
> $ du -k `find . -name 'config'`
> 696     ./jdk.corba/8-ea/config
> 1304    ./jdk.classpath/8-ea/config
> 1232    ./jdk.tools/8-ea/config
> 604     ./jdk.rmi/8-ea/config
> 680     ./jdk.tools.base/8-ea/config
> 604     ./jdk.base/8-ea/config
> 1056    ./jdk.tools.jaxws/8-ea/config
> 676     ./jdk.devtools/8-ea/config
> 608     ./jdk.tools.jre/8-ea/config
> 
> $ du -k `find . -name 'config'`
> 444     ./jdk.corba/8-ea/config
> 792     ./jdk.classpath/8-ea/config
> 748     ./jdk.tools/8-ea/config
> 392     ./jdk.rmi/8-ea/config
> 436     ./jdk.tools.base/8-ea/config
> 392     ./jdk.base/8-ea/config
> 636     ./jdk.tools.jaxws/8-ea/config
> 436     ./jdk.devtools/8-ea/config
> 396     ./jdk.tools.jre/8-ea/config
> 
> 
> 5. I ran the jtreg regression tests and JCK api&  lang tests.
> The test results are looking real good.   Compared with the
> jigsaw repo as the baseline, there are 2 new JCK test failures
> that I have to investigate (passed: 23,685; failed: 138; error: 28)
> 
> There is a startup initialization issue due to the use of
> Files.isSameFile method and the default file system provider
> is replaced with user defined one.  I'm currently looking into
> a fix for it.
> 
> I'd like to get this webrev out for review early and I also
> expect any issue found will be minor that I will generate a
> separate patch on top of this as a separate review.
> 
> Open Issues:
> 1. sun.boot.class.path no longer represents the search path
> for the bootstrap class loader.  Any application depending
> on this system property may be impacted.
> 
> rmic is one example that uses this property value to find
> system classes.  There has been work to convert rmic to
> use the javax.tool APIs that will support a module image.
> Unless rmic becomes module-aware, it continues to use
> the launcher hack setting -Xbootclasspath/p as a workaround.
> 
> Mandy
>