Jigsaw classpath mode support
Mandy Chung
mandy.chung at oracle.com
Fri Aug 10 09:29:44 PDT 2012
Hi Paul,
Thanks for the review.
On 8/10/2012 3:10 AM, Paul Sandoz wrote:
> Hi Mandy,
>
> This looks good to me, although i admit to not fully understanding the interactions between the Java layer and the VM layer.
I'm not sure if there is any document describing the existing VM and
libs interactions. Karen and I will write up a document describing such
interaction and startup sequence and post it to the project page.
> IMHO, given that the test results are so good, this patch should be pushed early and iterated on to refine/improve. That way we all get exposure to it.
Yes and especially to find out any compatibility issue to be addressed.
>
> Paul.
>
> --
>
> Looks like the Library interface may need to be updated so that:
>
> public abstract URI findLocalResource(ModuleId mid, String rn)
>
> returns a URI plus some meta-data e.g. size.
That's one option I consider. I prefer to leave it as it is for now and
do this with other optimization.
> --
>
> Definite room for optimisation in ClassPathContext, for example if we know the resource is a class or using some reverse lookup of resource path name.
Right - that's one thing and I also consider returning the size of the
resource data is another. I should add a comment about the possible
optimization.
>
> --
>
> FWIW in sun.misc.Launcher you can replace lines 215-224 with the two following lines:
>
> prepaths = urls.subList(0, index).toArray(new URL[index]);
> postpaths = urls.subList(index, urls.size()).toArray(new URL[urls.size() - index]);
Fixed. Thanks for catching this.
> Plus Vector could probably be replaced with List.
FWIW. Vector is also used in ClassLoader and other jdk classes. It has
been on the list of technical debts to clean up but it has to be done
carefully since they are used during startup.
As in sun.misc.Launcher, I change it to use ArrayList since I don't see
any problem with it.
Thanks
Mandy
> 189 // keep prepaths and postpaths for native library search
> 190 private static URL[] prepaths;
> 191 private static URL[] postpaths;
> 192 static private URL[] getExtURLs(File[] dirs) throws IOException {
> 193 File defaultExtDir = EXTENSION.defaultPath();
> 194 Vector<URL> urls = new Vector<URL>();
> 195 int index = 0;
> 196 for (int i = 0; i< dirs.length; i++) {
> 197 File dir = dirs[i];
> 198 String[] files = dir.list();
> 199 if (files != null) {
> 200 for (int j = 0; j< files.length; j++) {
> 201 if (!files[j].equals("meta-index")) {
> 202 File f = new File(dir, files[j]);
> 203 urls.add(getFileURL(f));
> 204 }
> 205 }
> 206 }
> 207 if (dir.exists()&& defaultExtDir != null&&
> 208 Files.isSameFile(defaultExtDir.toPath(), dir.toPath())) {
> 209 // set index to urls that dirs[i] is the default ext dir
> 210 // so that prepaths and postpaths can be determined.
> 211 index = urls.size();
> 212 urls.add(getFileURL(dir));
> 213 }
> 214 }
> 215 URL[] ua = new URL[urls.size()];
> 216 urls.copyInto(ua);
> 217 prepaths = new URL[index];
> 218 postpaths = new URL[urls.size() - index];
> 219 for (int i=0, j=0; i< ua.length; i++) {
> 220 if (i< index)
> 221 prepaths[i] = ua[i];
> 222 else
> 223 postpaths[j++] = ua[i];
> 224 }
> 225 return ua;
> 226 }
>
> On Aug 7, 2012, at 7:47 PM, Mandy Chung<mandy.chung at oracle.com> wrote:
>
>> I have implemented the "classpath mode" support running on
>> a module image.
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.00/
>>
>> JDK has been modularized in the jigsaw repo. There no longer
>> has a rt.jar, tools.jar, or any jar file. The system classes
>> are now installed in a system module library as modules. This
>> change will allow the bootstrap class loader, extension class
>> loader, and system class loader to search classes and resources
>> in the system module library as the default. Some implementation
>> details:
>>
>> 1. VM used to maintain the default bootclasspath list and search
>> classes in it. Now it will call jigsaw native interface to find
>> classes from the module library unless -Xbootclasspath is set
>> to override the default bootclasspath. The -Xbootclasspath/p
>> and -Xbootclasspath/a will continue to work as it is today.
>> Karen will be sending out the webrev for the hotspot change
>> separately.
>>
>> 2. tools.jar
>>
>> For compatibility, -classpath tools.jar should continue
>> to work. There are existing applications using URLClassLoader
>> that loads classes from tools.jar (e.g. test harness) that
>> also should continue to work.
>>
>> To be able to determine if the path of the given tools.jar
>> to either -classpath option or URL parameter to URLClassLoader
>> is the same path as JAVA_HOME/lib/tools.jar in a lightweight,
>> the module image for the full JDK will contain an empty tools.jar.
>> This also allows applications that check for the existence of
>> tools.jar to continue to work.
>>
>> sun.misc.URLClassPath is the main implementation class to
>> support URLClassLoader to search a file from the given URLs.
>> Both ExtClassLoader and AppClassLoader are URLClassLoader.
>> URLClassPath is extended to find resource file from the
>> appropriate system modules if the given URL is
>> JAVA_HOME/lib/tools.jar or JAVA_HOME/lib/ext.
>>
>> 3. How to determine which modules belong to bootclasspath,
>> ext directory, and tools.jar (and a few jar files in JDK/lib)
>> at runtime
>>
>> A configuration for the classpath mode support is loaded at
>> startup rather than generated to minimize the startup cost.
>> The idea is to have one ClassPathContext for bootclasspath,
>> one for extension and one for the jdk tools.
>>
>> In this prototype, I just define a new module "jdk.classpath"
>> and install it in the module image. It hardcodes the list
>> of modules for each ClassPathContext for now. One alternative
>> is to extend jpkg to generate "jdk.classpath" config specifically
>> for this classpath mode. We'll explore this later. But I
>> think this is good enough for now.
>>
>> 4. I made simple change in StoredConfiguration to reduce
>> the footprint of the configuration stored on disk. There
>> were several regression tests setting -Xmx failed due to
>> the increase of the startup footprint without this change.
>>
>> With storing the context names and package names only once,
>> it reduces the size of the config with 35-40%. I suspect
>> the fast configuration will make similar change.
>>
>> $ du -k `find . -name 'config'`
>> 696 ./jdk.corba/8-ea/config
>> 1304 ./jdk.classpath/8-ea/config
>> 1232 ./jdk.tools/8-ea/config
>> 604 ./jdk.rmi/8-ea/config
>> 680 ./jdk.tools.base/8-ea/config
>> 604 ./jdk.base/8-ea/config
>> 1056 ./jdk.tools.jaxws/8-ea/config
>> 676 ./jdk.devtools/8-ea/config
>> 608 ./jdk.tools.jre/8-ea/config
>>
>> $ du -k `find . -name 'config'`
>> 444 ./jdk.corba/8-ea/config
>> 792 ./jdk.classpath/8-ea/config
>> 748 ./jdk.tools/8-ea/config
>> 392 ./jdk.rmi/8-ea/config
>> 436 ./jdk.tools.base/8-ea/config
>> 392 ./jdk.base/8-ea/config
>> 636 ./jdk.tools.jaxws/8-ea/config
>> 436 ./jdk.devtools/8-ea/config
>> 396 ./jdk.tools.jre/8-ea/config
>>
>>
>> 5. I ran the jtreg regression tests and JCK api& lang tests.
>> The test results are looking real good. Compared with the
>> jigsaw repo as the baseline, there are 2 new JCK test failures
>> that I have to investigate (passed: 23,685; failed: 138; error: 28)
>>
>> There is a startup initialization issue due to the use of
>> Files.isSameFile method and the default file system provider
>> is replaced with user defined one. I'm currently looking into
>> a fix for it.
>>
>> I'd like to get this webrev out for review early and I also
>> expect any issue found will be minor that I will generate a
>> separate patch on top of this as a separate review.
>>
>> Open Issues:
>> 1. sun.boot.class.path no longer represents the search path
>> for the bootstrap class loader. Any application depending
>> on this system property may be impacted.
>>
>> rmic is one example that uses this property value to find
>> system classes. There has been work to convert rmic to
>> use the javax.tool APIs that will support a module image.
>> Unless rmic becomes module-aware, it continues to use
>> the launcher hack setting -Xbootclasspath/p as a workaround.
>>
>> Mandy
>>
More information about the jigsaw-dev
mailing list