From mr at sun.com Mon Mar 1 13:03:15 2010 From: mr at sun.com (Mark Reinhold) Date: Mon, 01 Mar 2010 13:03:15 -0800 Subject: Jigsaw modules build Message-ID: <20100301210315.920E4513@eggemoggin.niobe.net> Thanks for the merge last Friday. I've pulled it into my working forest and am rebasing all my patches today. I'm pretty sure I already mentioned this verbally, but just in case not: Please make it your priority to update the modules build to use Dalibor's latest jpkg changes to generate module files for the JDK modules. Our goal is to create a minimal JRE base image into which additional modules can be installed. The legacy-mode effort can wait for now. There are some performance issues in the module-file code, but Dalibor is already working on them. If you run into any other problems, please let him know. Thanks, - Mark From Mandy.Chung at Sun.COM Mon Mar 1 14:16:13 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Mon, 01 Mar 2010 14:16:13 -0800 Subject: Jigsaw modules build In-Reply-To: <20100301210315.920E4513@eggemoggin.niobe.net> References: <20100301210315.920E4513@eggemoggin.niobe.net> Message-ID: <4B8C3CAD.6080808@Sun.com> Yes, I am working on the modules build to generate modules files in jmod format for jdk modules. Meanwhile, I'm also looking at a problem on linux-i586 that VM fails to start up (running java -m jdk.boot). *** glibc detected *** /net/woody/export/mchung/bundles/jigsaw-merge/linux-i586/bin/java: free(): invalid next size (fast): 0x090d1ad8 *** This seems to fail only on product VM. Mandy On 03/01/10 13:03, Mark Reinhold wrote: > Thanks for the merge last Friday. I've pulled it into my working forest > and am rebasing all my patches today. > > I'm pretty sure I already mentioned this verbally, but just in case not: > Please make it your priority to update the modules build to use Dalibor's > latest jpkg changes to generate module files for the JDK modules. Our > goal is to create a minimal JRE base image into which additional modules > can be installed. The legacy-mode effort can wait for now. > > There are some performance issues in the module-file code, but Dalibor is > already working on them. If you run into any other problems, please let > him know. > > Thanks, > - Mark From dalibor.topic at sun.com Mon Mar 1 15:45:50 2010 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Mon, 01 Mar 2010 23:45:50 +0000 Subject: hg: jigsaw/jigsaw/jdk: Use MessageDigest.isEqual to compare hashes Message-ID: <20100301234609.59CA341E1E@hg.openjdk.java.net> Changeset: 5e72d94a78c2 Author: robilad Date: 2010-03-02 00:44 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/5e72d94a78c2 Use MessageDigest.isEqual to compare hashes ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java From jonathan.gibbons at sun.com Mon Mar 1 17:56:11 2010 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Tue, 02 Mar 2010 01:56:11 +0000 Subject: hg: jigsaw/jigsaw/langtools: 34 new changesets Message-ID: <20100302015743.528DE41E42@hg.openjdk.java.net> Changeset: f0074aa48d4e Author: katleman Date: 2010-01-14 15:48 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/f0074aa48d4e Added tag jdk7-b79 for changeset ac5b4c5644ce ! .hgtags Changeset: 250a580ab046 Author: mikejwre Date: 2010-01-21 11:12 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/250a580ab046 Added tag jdk7-b80 for changeset f0074aa48d4e ! .hgtags Changeset: a84062774f0e Author: lana Date: 2010-01-15 15:37 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/a84062774f0e Merge Changeset: cfabfcf9f110 Author: lana Date: 2010-01-22 09:34 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/cfabfcf9f110 Merge Changeset: 47003a3622f6 Author: mikejwre Date: 2010-01-28 11:27 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/47003a3622f6 Added tag jdk7-b81 for changeset cfabfcf9f110 ! .hgtags Changeset: c9f4ae1f1480 Author: mikejwre Date: 2010-02-04 11:19 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/c9f4ae1f1480 Added tag jdk7-b82 for changeset 47003a3622f6 ! .hgtags Changeset: 2edcb5dc642d Author: mikejwre Date: 2010-02-12 13:25 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/2edcb5dc642d Added tag jdk7-b83 for changeset c9f4ae1f1480 ! .hgtags Changeset: f23b985beb78 Author: jjg Date: 2010-01-19 14:28 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/f23b985beb78 6917067: refactor type annotations code from TransTypes into new TypeAnnotations class Reviewed-by: jjg, darcy Contributed-by: mali at csail.mit.edu, mernst at cs.washington.edu + src/share/classes/com/sun/tools/javac/code/TypeAnnotations.java ! src/share/classes/com/sun/tools/javac/comp/TransTypes.java Changeset: 0eaf89e08564 Author: jjg Date: 2010-01-20 16:12 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/0eaf89e08564 6918127: improve handling of TypeAnnotationPosition fields Reviewed-by: jjg, darcy Contributed-by: mali at csail.mit.edu, mernst at cs.washington.edu ! src/share/classes/com/sun/tools/classfile/ExtendedAnnotation.java ! src/share/classes/com/sun/tools/javac/code/TypeAnnotationPosition.java ! src/share/classes/com/sun/tools/javac/jvm/Code.java ! src/share/classes/com/sun/tools/javac/jvm/Gen.java Changeset: da0e3e2dd3ef Author: jjg Date: 2010-01-26 11:15 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/da0e3e2dd3ef 6919944: incorrect position given for duplicate annotation value error Reviewed-by: darcy ! src/share/classes/com/sun/tools/javac/comp/Check.java ! test/tools/javac/typeAnnotations/failures/common/arrayclass/DuplicateAnnotationValue.java ! test/tools/javac/typeAnnotations/failures/common/arrayclass/DuplicateAnnotationValue.out ! test/tools/javac/typeAnnotations/failures/common/arrays/DuplicateAnnotationValue.java ! test/tools/javac/typeAnnotations/failures/common/arrays/DuplicateAnnotationValue.out ! test/tools/javac/typeAnnotations/failures/common/innertypeparams/DuplicateAnnotationValue.java ! test/tools/javac/typeAnnotations/failures/common/innertypeparams/DuplicateAnnotationValue.out ! test/tools/javac/typeAnnotations/failures/common/newarray/DuplicateAnnotationValue.java ! test/tools/javac/typeAnnotations/failures/common/newarray/DuplicateAnnotationValue.out ! test/tools/javac/typeAnnotations/failures/common/parambounds/DuplicateAnnotationValue.java ! test/tools/javac/typeAnnotations/failures/common/parambounds/DuplicateAnnotationValue.out ! test/tools/javac/typeAnnotations/failures/common/receiver/DuplicateAnnotationValue.java ! test/tools/javac/typeAnnotations/failures/common/receiver/DuplicateAnnotationValue.out ! test/tools/javac/typeAnnotations/failures/common/typeArgs/DuplicateAnnotationValue.java ! test/tools/javac/typeAnnotations/failures/common/typeArgs/DuplicateAnnotationValue.out ! test/tools/javac/typeAnnotations/failures/common/typeparams/DuplicateAnnotationValue.java ! test/tools/javac/typeAnnotations/failures/common/typeparams/DuplicateAnnotationValue.out ! test/tools/javac/typeAnnotations/failures/common/wildcards/DuplicateAnnotationValue.java ! test/tools/javac/typeAnnotations/failures/common/wildcards/DuplicateAnnotationValue.out Changeset: 59167312ed4e Author: jjg Date: 2010-01-26 11:23 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/59167312ed4e 6917130: should test that annotations that have been optimized away are not emitted to classfile Reviewed-by: jjg, darcy Contributed-by: mali at csail.mit.edu, mernst at cs.washington.edu + test/tools/javac/typeAnnotations/classfile/DeadCode.java Changeset: ff7a01f9eff3 Author: lana Date: 2010-01-27 14:46 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/ff7a01f9eff3 Merge Changeset: 699ecefbdd4e Author: jjg Date: 2010-01-29 16:06 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/699ecefbdd4e 6919889: assorted position errors in compiler syntax trees Reviewed-by: darcy ! src/share/classes/com/sun/tools/javac/code/Flags.java ! src/share/classes/com/sun/tools/javac/parser/JavacParser.java ! src/share/classes/com/sun/tools/javac/tree/TreeInfo.java ! src/share/classes/com/sun/tools/javac/tree/TreeMaker.java + test/tools/javac/T6654037.java ! test/tools/javac/generics/diamond/neg/Neg01.out ! test/tools/javac/generics/diamond/neg/Neg02.out ! test/tools/javac/generics/diamond/neg/Neg03.out ! test/tools/javac/generics/diamond/neg/Neg04.out + test/tools/javac/treepostests/TreePosTest.java Changeset: 8e638442522a Author: jjg Date: 2010-01-29 16:54 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/8e638442522a 6499119: Created package-info class file modeled improperly 6920317: package-info.java file has to be specified on the javac cmdline, else it will not be avail. Reviewed-by: darcy ! src/share/classes/com/sun/tools/javac/code/Symbol.java ! src/share/classes/com/sun/tools/javac/comp/Enter.java ! src/share/classes/com/sun/tools/javac/comp/Lower.java ! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java + test/tools/javac/processing/6499119/ClassProcessor.java + test/tools/javac/processing/6499119/package-info.java + test/tools/javac/processing/T6920317.java Changeset: 732510cc3538 Author: jjg Date: 2010-02-01 17:05 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/732510cc3538 6919986: [308] change size of type_index (of CLASS_EXTENDS and THROWS) from byte to short Reviewed-by: darcy, jjg Contributed-by: mali at csail.mit.edu, mernst at cs.washington.edu ! src/share/classes/com/sun/tools/classfile/ExtendedAnnotation.java ! src/share/classes/com/sun/tools/javac/code/TypeAnnotations.java ! src/share/classes/com/sun/tools/javac/jvm/ClassReader.java ! src/share/classes/com/sun/tools/javac/jvm/ClassWriter.java Changeset: b0a68258360a Author: jjg Date: 2010-02-02 10:56 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/b0a68258360a 6918625: handle annotations on array class literals Reviewed-by: jjg, darcy Contributed-by: mali at csail.mit.edu, mernst at cs.washington.edu ! src/share/classes/com/sun/tools/classfile/ClassWriter.java ! src/share/classes/com/sun/tools/javac/code/TypeAnnotationPosition.java ! src/share/classes/com/sun/tools/javap/AnnotationWriter.java + test/tools/javap/typeAnnotations/ArrayClassLiterals2.java Changeset: 41ed86f86585 Author: jjg Date: 2010-02-03 11:28 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/41ed86f86585 6922429: extend tree position test waiver Reviewed-by: darcy ! test/tools/javac/treepostests/TreePosTest.java Changeset: f65d652cb6af Author: jjg Date: 2010-02-03 11:33 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/f65d652cb6af 6922300: [308] populate the reference_info for type annotations targeting primitive class literals Reviewed-by: darcy, jjg Contributed-by: mali at csail.mit.edu, mernst at cs.washington.edu ! src/share/classes/com/sun/tools/javac/jvm/Gen.java Changeset: 4c844e609d81 Author: jjg Date: 2010-02-03 16:58 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/4c844e609d81 6921979: add test program to verify annotations are attached to nodes as expected Reviewed-by: darcy + test/tools/javac/treeannotests/AnnoTreeTests.java + test/tools/javac/treeannotests/DA.java + test/tools/javac/treeannotests/TA.java + test/tools/javac/treeannotests/Test.java + test/tools/javac/treeannotests/TestProcessor.java Changeset: 4b4e282a3146 Author: jjg Date: 2010-02-04 10:14 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/4b4e282a3146 6923080: TreeScanner.visitNewClass should scan tree.typeargs Reviewed-by: darcy ! src/share/classes/com/sun/tools/javac/code/TypeAnnotations.java ! src/share/classes/com/sun/tools/javac/comp/MemberEnter.java ! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java ! src/share/classes/com/sun/tools/javac/tree/TreeScanner.java + test/tools/javac/tree/T6923080.java + test/tools/javac/tree/TreeScannerTest.java Changeset: 56a46d079264 Author: lana Date: 2010-02-08 23:59 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/56a46d079264 Merge Changeset: d9cd5b8286e4 Author: lana Date: 2010-02-14 23:39 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/d9cd5b8286e4 Merge Changeset: 75d5bd12eb86 Author: mikejwre Date: 2010-02-18 13:31 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/75d5bd12eb86 Added tag jdk7-b84 for changeset d9cd5b8286e4 ! .hgtags Changeset: 7d9e3a15d2b3 Author: jjg Date: 2010-02-15 16:09 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/7d9e3a15d2b3 6926555: 6921979 breaks TreePosTest Reviewed-by: darcy ! test/tools/javac/treepostests/TreePosTest.java Changeset: af18e3956985 Author: darcy Date: 2010-02-15 18:20 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/af18e3956985 6634138: Source generated in last round not compiled Reviewed-by: jjg ! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java ! test/tools/javac/T6403466.java + test/tools/javac/processing/6634138/Dummy.java + test/tools/javac/processing/6634138/ExerciseDependency.java + test/tools/javac/processing/6634138/T6634138.java Changeset: fe17a9dbef03 Author: darcy Date: 2010-02-15 20:06 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/fe17a9dbef03 6926699: Annotation processing regression tests should typically return SourceVersion.latest Reviewed-by: jjg ! test/tools/javac/6341866/Anno.java ! test/tools/javac/T6406771.java ! test/tools/javac/T6411379.java ! test/tools/javac/T6423583.java ! test/tools/javac/T6855236.java ! test/tools/javac/api/6421111/T6421111.java ! test/tools/javac/api/6468404/T6468404.java ! test/tools/javac/api/T6412669.java ! test/tools/javac/enum/6424358/T6424358.java ! test/tools/javac/processing/6348499/A.java ! test/tools/javac/processing/6414633/A.java ! test/tools/javac/processing/6430209/T6430209.java ! test/tools/javac/processing/6430209/b6341534.java ! test/tools/javac/processing/T6439826.java ! test/tools/javac/processing/model/element/TypeParamBounds.java ! test/tools/javac/processing/model/type/MirroredTypeEx/OverEager.java ! test/tools/javac/processing/model/type/NoTypes.java ! test/tools/javac/processing/model/util/GetTypeElemBadArg.java ! test/tools/javac/processing/model/util/OverridesSpecEx.java Changeset: 631a273dac0f Author: darcy Date: 2010-02-15 20:17 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/631a273dac0f 6926703: apt tests should run with assertions enabled Reviewed-by: jjg ! src/share/classes/com/sun/mirror/util/SourceOrderDeclScanner.java Changeset: 16b9b7f45933 Author: darcy Date: 2010-02-17 14:30 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/16b9b7f45933 6927061: Refactor apt implemenation to use code from JSR 269 Reviewed-by: jjg ! src/share/classes/com/sun/mirror/util/SourceOrderDeclScanner.java ! src/share/classes/com/sun/tools/apt/comp/Apt.java ! src/share/classes/com/sun/tools/apt/main/Main.java ! src/share/classes/com/sun/tools/javac/file/Paths.java ! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java ! src/share/classes/com/sun/tools/javadoc/DocletInvoker.java Changeset: 67f0e05098fa Author: lana Date: 2010-02-17 10:25 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/67f0e05098fa Merge Changeset: 0fce6b64c258 Author: lana Date: 2010-02-17 16:29 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/0fce6b64c258 Merge Changeset: a3be81d385ee Author: jjg Date: 2010-02-18 15:41 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/a3be81d385ee 6927797: langtools/test/tools/javac/EarlyAssert.java fails when run with assertions enabled (-ea) Reviewed-by: darcy ! test/tools/javac/EarlyAssert.java + test/tools/javac/EarlyAssertWrapper.java Changeset: f25efdb55c99 Author: andrew Date: 2010-02-22 21:37 +0000 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/f25efdb55c99 6928623: Behaviour of VERBOSE=true on langtools build Summary: VERBOSE=true causes -diagnostics to be passed to ant rather than -debug Reviewed-by: jjg ! make/Makefile Changeset: 136bfc679462 Author: lana Date: 2010-02-23 10:17 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/136bfc679462 Merge Changeset: 94ceb4acdb8d Author: jjg Date: 2010-03-01 17:53 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/94ceb4acdb8d Merge ! .hgtags ! src/share/classes/com/sun/tools/classfile/ClassWriter.java ! src/share/classes/com/sun/tools/javac/code/Flags.java ! src/share/classes/com/sun/tools/javac/code/Symbol.java ! src/share/classes/com/sun/tools/javac/comp/Check.java ! src/share/classes/com/sun/tools/javac/comp/Enter.java ! src/share/classes/com/sun/tools/javac/comp/Lower.java ! src/share/classes/com/sun/tools/javac/comp/MemberEnter.java ! src/share/classes/com/sun/tools/javac/file/Paths.java ! src/share/classes/com/sun/tools/javac/jvm/ClassReader.java ! src/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/share/classes/com/sun/tools/javac/parser/JavacParser.java ! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java ! src/share/classes/com/sun/tools/javac/tree/TreeInfo.java ! src/share/classes/com/sun/tools/javac/tree/TreeMaker.java ! src/share/classes/com/sun/tools/javac/tree/TreeScanner.java + test/tools/javac/T6654037.java From Mandy.Chung at Sun.COM Mon Mar 1 19:13:44 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Mon, 01 Mar 2010 19:13:44 -0800 Subject: VM fix for booting from a module Message-ID: <4B8C8268.3000205@sun.com> Karen, Mark, I found two issues with the VM change for booting from a module. 1. sun.java.launcher.module.boot property added to the system property list but the memory is already freed. 2. resource path for the boot module in the bootclasspath was incorrect Webrev at: http://cr.openjdk.java.net/~mchung/jigsaw/hotspot-boot-fix/ Apparently, the memory problem caused by (1) only showed up in few linux-i586 machines but not all. Surprisingly, it didn't show up on solaris i586, windows, and linux-amd64 (at least on the machines I ran the tests). Can you review it? Thanks Mandy From mr at sun.com Mon Mar 1 20:53:42 2010 From: mr at sun.com (Mark Reinhold) Date: Mon, 01 Mar 2010 20:53:42 -0800 Subject: VM fix for booting from a module In-Reply-To: mandy.chung@sun.com; Mon, 01 Mar 2010 19:13:44 PST; <4B8C8268.3000205@sun.com> Message-ID: <20100302045342.E8453513@eggemoggin.niobe.net> > Date: Mon, 01 Mar 2010 19:13:44 -0800 > From: mandy.chung at sun.com > I found two issues with the VM change for booting from a module. > 1. sun.java.launcher.module.boot property added to the system property list but > the memory is already freed. Oh my, how embarrassing. (For me.) > 2. resource path for the boot module in the bootclasspath was incorrect Yep. > Webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/hotspot-boot-fix/ > > Apparently, the memory problem caused by (1) only showed up in few linux-i586 > machines but not all. Surprisingly, it didn't show up on solaris i586, > windows, and linux-amd64 (at least on the machines I ran the tests). > > Can you review it? The fix for (1) is obviously correct. The fix for (2) looks good, except you should use strncpy instead of strcpy so as to avoid buffer overflows. - Mark From Mandy.Chung at Sun.COM Mon Mar 1 22:08:55 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Mon, 01 Mar 2010 22:08:55 -0800 Subject: VM fix for booting from a module In-Reply-To: <20100302045342.E8453513@eggemoggin.niobe.net> References: <20100302045342.E8453513@eggemoggin.niobe.net> Message-ID: <4B8CAB77.4070804@sun.com> Mark Reinhold wrote: >> Webrev at: >> http://cr.openjdk.java.net/~mchung/jigsaw/hotspot-boot-fix/ >> >> Apparently, the memory problem caused by (1) only showed up in few linux-i586 >> machines but not all. Surprisingly, it didn't show up on solaris i586, >> windows, and linux-amd64 (at least on the machines I ran the tests). >> >> Can you review it? >> > > The fix for (1) is obviously correct. > > The fix for (2) looks good, except you should use strncpy instead of > strcpy so as to avoid buffer overflows. > Will fix that. Thanks. Mandy From Mandy.Chung at Sun.COM Mon Mar 1 22:50:58 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Mon, 01 Mar 2010 22:50:58 -0800 Subject: save/restore file permission in modules writing/reading Message-ID: <4B8CB552.40307@sun.com> The file permission of several files in the jdk is modified during the build. Upon installing a module, I would expect that the modules writing/reading would restore the file permission. Currently, native cmds on all platforms and native libs on windows are assumed to have execute permission. Would it be appropriate for the module file format to include the file permission info that can be saved/restored properly? Mandy From Karen.Kinnear at Sun.COM Tue Mar 2 05:50:37 2010 From: Karen.Kinnear at Sun.COM (Karen Kinnear) Date: Tue, 02 Mar 2010 08:50:37 -0500 Subject: VM fix for booting from a module In-Reply-To: <4B8C8268.3000205@sun.com> References: <4B8C8268.3000205@sun.com> Message-ID: <83C3FAFD-6F61-4797-A0E5-C79BF283D118@Sun.COM> Mandy, Looks good. Good catch on #1 - I'm so glad you figured it out. thanks, KAren p.s. sorry I didn't see your earlier email on this topic - I seem to be missing emails far too frequently recently On Mar 1, 2010, at 10:13 PM, Mandy Chung wrote: > Karen, Mark, > > I found two issues with the VM change for booting from a module. > 1. sun.java.launcher.module.boot property added to the system > property list but the memory is already freed. > 2. resource path for the boot module in the bootclasspath was > incorrect > > Webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/hotspot-boot-fix/ > > Apparently, the memory problem caused by (1) only showed up in few > linux-i586 machines but not all. Surprisingly, it didn't show up on > solaris i586, windows, and linux-amd64 (at least on the machines I > ran the tests). > > Can you review it? > > Thanks > Mandy From Mandy.Chung at Sun.COM Tue Mar 2 08:34:04 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Tue, 02 Mar 2010 08:34:04 -0800 Subject: Path names of native code files Message-ID: <4B8D3DFC.9020906@sun.com> Mark, Per the module-file format spec v4 [1]: "The path names of native-code ?les must not include more than one element." The jdk currently contains files that are in deeper directory hierachy e.g. lib/i386/client/libjvm.so lib/i386/server/libjvm.so What is the rationale behind the pathname limitation? For now, I can turn off this check but checks the length of the pathname instead. Mandy [1] http://cr.openjdk.java.net/~mr/jigsaw/notes/module-file-format/ From mr at sun.com Tue Mar 2 09:26:32 2010 From: mr at sun.com (Mark Reinhold) Date: Tue, 02 Mar 2010 09:26:32 -0800 Subject: save/restore file permission in modules writing/reading In-Reply-To: mandy.chung@sun.com; Mon, 01 Mar 2010 22:50:58 PST; <4B8CB552.40307@sun.com> Message-ID: <20100302172632.4DF38322@eggemoggin.niobe.net> > Date: Mon, 01 Mar 2010 22:50:58 -0800 > From: mandy.chung at sun.com > The file permission of several files in the jdk is modified during the build. > Upon installing a module, I would expect that the modules writing/reading would > restore the file permission. > > Currently, native cmds on all platforms and native libs on windows are assumed > to have execute permission. Would it be appropriate for the module file format > to include the file permission info that can be saved/restored properly? That's not necessary. (If it were, we'd then have to figure out how to encode file mode bits in a platform-independent way, etc. Ugh.) As the draft spec says, the module-file reader is responsible for setting the mode bits on executables and pseudo-executables (.DLLs on Windows) as necessary on the host system. There's already logic in the reader to do this. - Mark From mr at sun.com Tue Mar 2 09:39:33 2010 From: mr at sun.com (Mark Reinhold) Date: Tue, 02 Mar 2010 09:39:33 -0800 Subject: Path names of native code files In-Reply-To: mandy.chung@sun.com; Tue, 02 Mar 2010 08:34:04 PST; <4B8D3DFC.9020906@sun.com> Message-ID: <20100302173933.69286322@eggemoggin.niobe.net> > Date: Tue, 02 Mar 2010 08:34:04 -0800 > From: mandy.chung at sun.com > Per the module-file format spec v4 [1]: > > "The path names of native-code files must not include more than one element." > > The jdk currently contains files that are in deeper directory hierachy > e.g. lib/i386/client/libjvm.so > lib/i386/server/libjvm.so > > What is the rationale behind the pathname limitation? Simplicity. If we need multiple levels then let's make sure we have a really good reason for them. In the case of libjvm.so, for now those are in the base image, which we are not yet delivering as a module file, so they can stay where they are if that's easiest. For other cases we should eliminate the $ARCH element since module files will eventually contain architecture information for native code. That would leave just a few odd subdirectories (jli, xawt, headless) which I'm pretty sure we can eliminate by modifying runpaths and/or using dlopen(). - Mark From dalibor.topic at sun.com Tue Mar 2 11:22:12 2010 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Tue, 02 Mar 2010 19:22:12 +0000 Subject: hg: jigsaw/jigsaw/jdk: (jpkg) implement --fast for jmod files Message-ID: <20100302192257.B877841F5B@hg.openjdk.java.net> Changeset: a641e9d302c6 Author: robilad Date: 2010-03-02 20:17 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/a641e9d302c6 (jpkg) implement --fast for jmod files ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From Mandy.Chung at Sun.COM Tue Mar 2 14:17:00 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Tue, 02 Mar 2010 14:17:00 -0800 Subject: VM fix for booting from a module In-Reply-To: <83C3FAFD-6F61-4797-A0E5-C79BF283D118@Sun.COM> References: <4B8C8268.3000205@sun.com> <83C3FAFD-6F61-4797-A0E5-C79BF283D118@Sun.COM> Message-ID: <4B8D8E5C.8030805@Sun.com> Thanks. I clean up the fix for #2 to use memcpy and check for the length before copy. http://cr.openjdk.java.net/~mchung/jigsaw/hotspot-boot-fix.1/ Mandy On 03/02/10 05:50, Karen Kinnear wrote: > Mandy, > > Looks good. Good catch on #1 - I'm so glad you figured it out. > > thanks, > KAren > > p.s. sorry I didn't see your earlier email on this topic - I seem to be > missing emails far too frequently recently > > On Mar 1, 2010, at 10:13 PM, Mandy Chung wrote: > >> Karen, Mark, >> >> I found two issues with the VM change for booting from a module. >> 1. sun.java.launcher.module.boot property added to the system property >> list but the memory is already freed. >> 2. resource path for the boot module in the bootclasspath was incorrect >> >> Webrev at: >> http://cr.openjdk.java.net/~mchung/jigsaw/hotspot-boot-fix/ >> >> Apparently, the memory problem caused by (1) only showed up in few >> linux-i586 machines but not all. Surprisingly, it didn't show up on >> solaris i586, windows, and linux-amd64 (at least on the machines I ran >> the tests). >> >> Can you review it? >> >> Thanks >> Mandy > From Mandy.Chung at Sun.COM Tue Mar 2 15:50:20 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Tue, 02 Mar 2010 15:50:20 -0800 Subject: include header files, man pages, etc Message-ID: <4B8DA43C.5090409@Sun.com> The JDK has the following types of files that are not covered in the module-file format spec: 1. include header files 2. man pages I haven't dealt with the man pages yet but I think the man pages should go with the module of the tool it documents. Dalibor, Can you help add the support of the new section for the include header files? The section for man pages can come later. One other type of files is sample code. I would imagine that sample code would become several modules (not done in the modules build yet). Samples typically include sources, readme, and possibly makefile or ant script. It seems that there is no need to package samples in the module-file format. Perhaps it makes sense shipping the samples only in native packaging format? Thanks Mandy From Mandy.Chung at Sun.COM Tue Mar 2 18:16:50 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Tue, 02 Mar 2010 18:16:50 -0800 Subject: include header files, man pages, etc In-Reply-To: <4B8DA43C.5090409@Sun.com> References: <4B8DA43C.5090409@Sun.com> Message-ID: <4B8DC692.1000708@sun.com> Mandy Chung wrote: > The JDK has the following types of files that are not covered in the > module-file format spec: > > 1. include header files > 2. man pages > > I haven't dealt with the man pages yet but I think the man pages > should go > with the module of the tool it documents. > > Dalibor, > Can you help add the support of the new section for the include header > files? The section for man pages can come later. > Having a second thought, header files are for development use and they are not needed during runtime. It doesn't quite justify to add a section for include files in the module-file format. Another alternative is to to include all header files in the jdk base image that seems to be a reasonable option. Any thought? Mandy > One other type of files is sample code. I would imagine that sample code > would become several modules (not done in the modules build yet). > Samples typically include sources, readme, and possibly makefile or > ant script. > It seems that there is no need to package samples in the module-file > format. > Perhaps it makes sense shipping the samples only in native packaging > format? > > Thanks > Mandy From Karen.Kinnear at Sun.COM Wed Mar 3 05:41:47 2010 From: Karen.Kinnear at Sun.COM (Karen Kinnear) Date: Wed, 03 Mar 2010 08:41:47 -0500 Subject: VM fix for booting from a module In-Reply-To: <4B8D8E5C.8030805@Sun.com> References: <4B8C8268.3000205@sun.com> <83C3FAFD-6F61-4797-A0E5-C79BF283D118@Sun.COM> <4B8D8E5C.8030805@Sun.com> Message-ID: <05C93CCB-084F-48B9-BE0E-08D7B5871A94@Sun.COM> Thanks Mandy. Looks good thanks, Karen On Mar 2, 2010, at 5:17 PM, Mandy Chung wrote: > Thanks. I clean up the fix for #2 to use memcpy and check for the > length before copy. > http://cr.openjdk.java.net/~mchung/jigsaw/hotspot-boot-fix.1/ > > Mandy > > On 03/02/10 05:50, Karen Kinnear wrote: >> Mandy, >> Looks good. Good catch on #1 - I'm so glad you figured it out. >> thanks, >> KAren >> p.s. sorry I didn't see your earlier email on this topic - I seem >> to be missing emails far too frequently recently >> On Mar 1, 2010, at 10:13 PM, Mandy Chung wrote: >>> Karen, Mark, >>> >>> I found two issues with the VM change for booting from a module. >>> 1. sun.java.launcher.module.boot property added to the system >>> property list but the memory is already freed. >>> 2. resource path for the boot module in the bootclasspath was >>> incorrect >>> >>> Webrev at: >>> http://cr.openjdk.java.net/~mchung/jigsaw/hotspot-boot-fix/ >>> >>> Apparently, the memory problem caused by (1) only showed up in few >>> linux-i586 machines but not all. Surprisingly, it didn't show up >>> on solaris i586, windows, and linux-amd64 (at least on the >>> machines I ran the tests). >>> >>> Can you review it? >>> >>> Thanks >>> Mandy From Mandy.Chung at Sun.COM Wed Mar 3 08:55:40 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Wed, 03 Mar 2010 08:55:40 -0800 Subject: Review request for fix for legacy jdk image Message-ID: <4B8E948C.5010805@sun.com> The jdk modules build change causes two regressions in the legacy jdk images: 1. the modules build creates a module library under /lib/modules that gets copied to the legacy jdk image 2. the launcher runs in module mode as supposed to run in legacy mode Alan, Can you review the fix in the launcher and the modules build makefile: http://cr.openjdk.java.net/~mchung/jigsaw/launcher-in-legacy-image/ Thanks Mandy From mandy.chung at sun.com Wed Mar 3 12:29:37 2010 From: mandy.chung at sun.com (mandy.chung at sun.com) Date: Wed, 03 Mar 2010 20:29:37 +0000 Subject: hg: jigsaw/jigsaw/hotspot: Fix VM to properly boot from a module Message-ID: <20100303202940.9D72343C7A@hg.openjdk.java.net> Changeset: e04d24e3755c Author: mchung Date: 2010-03-03 08:17 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/e04d24e3755c Fix VM to properly boot from a module ! src/share/vm/runtime/arguments.cpp ! src/share/vm/runtime/os.cpp From Alan.Bateman at Sun.COM Wed Mar 3 12:56:58 2010 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 03 Mar 2010 20:56:58 +0000 Subject: Review request for fix for legacy jdk image In-Reply-To: <4B8E948C.5010805@sun.com> References: <4B8E948C.5010805@sun.com> Message-ID: <4B8ECD1A.8060909@sun.com> Mandy Chung wrote: > The jdk modules build change causes two regressions in the legacy jdk > images: > 1. the modules build creates a module library under > /lib/modules > that gets copied to the legacy jdk image > 2. the launcher runs in module mode as supposed to run in legacy mode > > Alan, > > Can you review the fix in the launcher and the modules build makefile: > http://cr.openjdk.java.net/~mchung/jigsaw/launcher-in-legacy-image/ > > Thanks > Mandy Looks fine except in the launcher where it would be better to use snprintf. -Alan. From Mandy.Chung at Sun.COM Wed Mar 3 13:12:45 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Wed, 03 Mar 2010 13:12:45 -0800 Subject: Review request for fix for legacy jdk image In-Reply-To: <4B8ECD1A.8060909@sun.com> References: <4B8E948C.5010805@sun.com> <4B8ECD1A.8060909@sun.com> Message-ID: <4B8ED0CD.9010902@Sun.com> On 03/03/10 12:56, Alan Bateman wrote: > Looks fine except in the launcher where it would be better to use snprintf. Right. I should have caught that :) Thanks. Mandy From ramon.garcia.f+java at gmail.com Thu Mar 4 07:34:42 2010 From: ramon.garcia.f+java at gmail.com (=?UTF-8?B?UmFtw7NuIEdhcmPDrWE=?=) Date: Thu, 4 Mar 2010 16:34:42 +0100 Subject: Is readonly file mapping a purpose of this project? Message-ID: It would save memory if the Java virtual machine were able to load a module file by mmap-ing it readonly and finding the Java class objects in it. In this way, several virtual machine instances using the same libraries would share memory. Is this among the features planned for this project? From mandy.chung at sun.com Thu Mar 4 09:52:10 2010 From: mandy.chung at sun.com (mandy.chung at sun.com) Date: Thu, 04 Mar 2010 17:52:10 +0000 Subject: hg: jigsaw/jigsaw/jdk: Launcher fix to run in legacy jdk image Message-ID: <20100304175254.946BA43DB1@hg.openjdk.java.net> Changeset: 3f11717f86c3 Author: mchung Date: 2010-03-04 09:50 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/3f11717f86c3 Launcher fix to run in legacy jdk image ! make/modules/Makefile ! src/share/bin/java.c ! src/windows/bin/java_md.h From Jonathan.Gibbons at Sun.COM Thu Mar 4 10:30:13 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Thu, 04 Mar 2010 10:30:13 -0800 Subject: hg: jigsaw/jigsaw/jdk: Launcher fix to run in legacy jdk image In-Reply-To: <20100304175254.946BA43DB1@hg.openjdk.java.net> References: <20100304175254.946BA43DB1@hg.openjdk.java.net> Message-ID: <4B8FFC35.5030302@sun.com> Mandy.Chung at Sun.COM wrote: > Changeset: 3f11717f86c3 > Author: mchung > Date: 2010-03-04 09:50 -0800 > URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/3f11717f86c3 > > Launcher fix to run in legacy jdk image > > ! make/modules/Makefile > ! src/share/bin/java.c > ! src/windows/bin/java_md.h > > Does this fix the build problems regarding SKIP_BOOT_CYCLE=false ? -- Jon From Mandy.Chung at Sun.COM Thu Mar 4 10:45:30 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Thu, 04 Mar 2010 10:45:30 -0800 Subject: hg: jigsaw/jigsaw/jdk: Launcher fix to run in legacy jdk image In-Reply-To: <4B8FFC35.5030302@sun.com> References: <20100304175254.946BA43DB1@hg.openjdk.java.net> <4B8FFC35.5030302@sun.com> Message-ID: <4B8FFFCA.9050007@Sun.com> On 03/04/10 10:30, Jonathan Gibbons wrote: > Mandy.Chung at Sun.COM wrote: >> Changeset: 3f11717f86c3 >> Author: mchung >> Date: 2010-03-04 09:50 -0800 >> URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/3f11717f86c3 >> >> Launcher fix to run in legacy jdk image >> >> ! make/modules/Makefile >> ! src/share/bin/java.c >> ! src/windows/bin/java_md.h >> >> > Does this fix the build problems regarding SKIP_BOOT_CYCLE=false ? Yes, this fixes the SKIP_BOOT_CYCLE=false build problem. Mandy From Jonathan.Gibbons at Sun.COM Thu Mar 4 10:48:55 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Thu, 04 Mar 2010 10:48:55 -0800 Subject: hg: jigsaw/jigsaw/jdk: Launcher fix to run in legacy jdk image In-Reply-To: <4B8FFFCA.9050007@Sun.com> References: <20100304175254.946BA43DB1@hg.openjdk.java.net> <4B8FFC35.5030302@sun.com> <4B8FFFCA.9050007@Sun.com> Message-ID: <4B900097.9000103@sun.com> Mandy Chung wrote: > On 03/04/10 10:30, Jonathan Gibbons wrote: >> Mandy.Chung at Sun.COM wrote: >>> Changeset: 3f11717f86c3 >>> Author: mchung >>> Date: 2010-03-04 09:50 -0800 >>> URL: >>> http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/3f11717f86c3 >>> >>> Launcher fix to run in legacy jdk image >>> >>> ! make/modules/Makefile >>> ! src/share/bin/java.c >>> ! src/windows/bin/java_md.h >>> >>> >> Does this fix the build problems regarding SKIP_BOOT_CYCLE=false ? > > Yes, this fixes the SKIP_BOOT_CYCLE=false build problem. > > Mandy > > Thanks. -- Jon From forax at univ-mlv.fr Thu Mar 4 13:38:23 2010 From: forax at univ-mlv.fr (=?UTF-8?B?UsOpbWkgRm9yYXg=?=) Date: Thu, 04 Mar 2010 22:38:23 +0100 Subject: Is readonly file mapping a purpose of this project? In-Reply-To: References: Message-ID: <4B90284F.2050502@univ-mlv.fr> Le 04/03/2010 16:34, Ram?n Garc?a a ?crit : > It would save memory if the Java virtual machine were able to load a > module file by mmap-ing it readonly and finding the Java class objects > in it. In this way, several virtual machine instances using the same > libraries would share memory. Is this among the features planned for > this project? > This is a feature of SUN jdk5 :) http://java.sun.com/j2se/1.5.0/docs/guide/vm/class-data-sharing.html R?mi From ramon.garcia.f+java at gmail.com Fri Mar 5 03:18:34 2010 From: ramon.garcia.f+java at gmail.com (=?UTF-8?B?UmFtw7NuIEdhcmPDrWE=?=) Date: Fri, 5 Mar 2010 12:18:34 +0100 Subject: Is readonly file mapping a purpose of this project? In-Reply-To: <4B90284F.2050502@univ-mlv.fr> References: <4B90284F.2050502@univ-mlv.fr> Message-ID: No, you did not understand my request .Class data sharing is limited to system classes. I am asking a module to be a file that can be distributed, and efficiently memory mapped and therefore shared. Android has a similar feature http://www.youtube.com/watch?v=ptjedOZEXPM On Thu, Mar 4, 2010 at 10:38 PM, R?mi Forax wrote: > Le 04/03/2010 16:34, Ram?n Garc?a a ?crit : >> >> It would save memory if the Java virtual machine were able to load a >> module file by mmap-ing it readonly and finding the Java class objects >> in it. In this way, several virtual machine instances using the same >> libraries would share memory. Is this among the features planned for >> this project? >> > > This is a feature of SUN jdk5 :) > http://java.sun.com/j2se/1.5.0/docs/guide/vm/class-data-sharing.html > > R?mi > > From mandy.chung at sun.com Fri Mar 5 10:04:44 2010 From: mandy.chung at sun.com (mandy.chung at sun.com) Date: Fri, 05 Mar 2010 18:04:44 +0000 Subject: hg: jigsaw/jigsaw/hotspot: 27 new changesets Message-ID: <20100305180608.1D1DB43F15@hg.openjdk.java.net> Changeset: 745c853ee57f Author: johnc Date: 2010-01-29 14:51 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/745c853ee57f 6885297: java -XX:RefDiscoveryPolicy=2 or -XX:TLABWasteTargetPercent=0 cause VM crash Summary: Interval checking is now being performed on the values passed in for these two flags. The current acceptable range for RefDiscoveryPolicy is [0..1], and for TLABWasteTargetPercent it is [1..100]. Reviewed-by: apetrusenko, ysr ! src/share/vm/includeDB_core ! src/share/vm/memory/referenceProcessor.hpp ! src/share/vm/runtime/arguments.cpp ! src/share/vm/runtime/arguments.hpp Changeset: 6484c4ee11cb Author: ysr Date: 2010-02-01 17:29 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/6484c4ee11cb 6904516: More object array barrier fixes, following up on 6906727 Summary: Fixed missing pre-barrier calls for G1, modified C1 to call pre- and correct post-barrier interfaces, deleted obsolete interface, (temporarily) disabled redundant deferred barrier in BacktraceBuilder. Reviewed-by: coleenp, jmasa, kvn, never ! src/share/vm/c1/c1_Runtime1.cpp ! src/share/vm/classfile/javaClasses.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/memory/barrierSet.hpp ! src/share/vm/memory/barrierSet.inline.hpp ! src/share/vm/runtime/stubRoutines.cpp Changeset: deada8912c54 Author: johnc Date: 2010-02-02 18:39 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/deada8912c54 6914402: G1: assert(!is_young_card(cached_ptr),"shouldn't get a card in young region") Summary: Invalid assert. Filter cards evicted from the card count cache instead. Reviewed-by: apetrusenko, tonyp ! src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp Changeset: 230fac611b50 Author: johnc Date: 2010-02-08 09:58 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/230fac611b50 Merge ! src/share/vm/c1/c1_Runtime1.cpp ! src/share/vm/includeDB_core Changeset: 455df1b81409 Author: kamg Date: 2010-02-08 13:49 -0500 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/455df1b81409 6587322: dtrace probe object__alloc doesn't fire in some situations on amd64 Summary: Fix misplaced probe point Reviewed-by: rasbold, phh Contributed-by: neojia at gmail.com ! src/cpu/x86/vm/templateTable_x86_64.cpp Changeset: 95d21201c29a Author: apangin Date: 2010-02-11 10:48 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/95d21201c29a Merge Changeset: 3f5b7efb9642 Author: never Date: 2010-02-05 11:07 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/3f5b7efb9642 6920293: OptimizeStringConcat causing core dumps Reviewed-by: kvn, twisti ! src/os_cpu/solaris_x86/vm/os_solaris_x86.cpp ! src/share/vm/code/nmethod.cpp ! src/share/vm/opto/stringopts.cpp ! src/share/vm/runtime/sharedRuntime.cpp Changeset: 576e77447e3c Author: kvn Date: 2010-02-07 12:15 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/576e77447e3c 6923002: assert(false,"this call site should not be polymorphic") Summary: Clear the total count when a receiver information is cleared. Reviewed-by: never, jrose ! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp ! src/cpu/sparc/vm/interp_masm_sparc.cpp ! src/cpu/sparc/vm/sharedRuntime_sparc.cpp ! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp ! src/cpu/x86/vm/interp_masm_x86_32.cpp ! src/cpu/x86/vm/interp_masm_x86_64.cpp ! src/share/vm/ci/ciMethod.cpp ! src/share/vm/oops/methodDataOop.hpp ! src/share/vm/opto/doCall.cpp ! src/share/vm/opto/runtime.cpp ! src/share/vm/runtime/arguments.cpp Changeset: f516d5d7a019 Author: kvn Date: 2010-02-08 12:20 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/f516d5d7a019 6910605: C2: NullPointerException/ClassCaseException is thrown when C2 with DeoptimizeALot is used Summary: Set the reexecute bit for runtime calls _new_array_Java when they used for _multianewarray bytecode. Reviewed-by: never ! src/share/vm/code/pcDesc.cpp ! src/share/vm/opto/graphKit.cpp ! src/share/vm/opto/parse3.cpp + test/compiler/6910605/Test.java Changeset: f70b0d9ab095 Author: kvn Date: 2010-02-09 01:31 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/f70b0d9ab095 6910618: C2: Error: assert(d->is_oop(),"JVM_ArrayCopy: dst not an oop") Summary: Mark in PcDesc call sites which return oop and save the result oop across objects reallocation during deoptimization. Reviewed-by: never ! src/share/vm/c1/c1_IR.hpp ! src/share/vm/code/debugInfoRec.cpp ! src/share/vm/code/debugInfoRec.hpp ! src/share/vm/code/nmethod.cpp ! src/share/vm/code/pcDesc.hpp ! src/share/vm/code/scopeDesc.cpp ! src/share/vm/code/scopeDesc.hpp ! src/share/vm/includeDB_core ! src/share/vm/opto/output.cpp ! src/share/vm/prims/jvmtiCodeBlobEvents.cpp ! src/share/vm/runtime/deoptimization.cpp + test/compiler/6910618/Test.java Changeset: 4ee1c645110e Author: kvn Date: 2010-02-09 10:21 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/4ee1c645110e 6924097: assert((_type == Type::MEMORY) == (_adr_type != 0),"adr_type for memory phis only") Summary: Use PhiNode::make_blank(r, n) method to construct the phi. Reviewed-by: never ! src/share/vm/opto/loopopts.cpp Changeset: e3a4305c6bc3 Author: kvn Date: 2010-02-12 08:54 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/e3a4305c6bc3 6925249: assert(last_sp < (intptr_t*) interpreter_frame_monitor_begin(),"bad tos") Summary: Fix assert since top deoptimized frame has last_sp == interpreter_frame_monitor_begin if there are no expressions. Reviewed-by: twisti ! src/cpu/x86/vm/frame_x86.inline.hpp ! src/share/vm/runtime/deoptimization.cpp ! src/share/vm/runtime/frame.cpp ! src/share/vm/runtime/vframeArray.cpp Changeset: c09ee209b65c Author: kvn Date: 2010-02-12 10:34 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/c09ee209b65c 6926048: Improve Zero performance Summary: Make Zero figure out result types in a similar way to C++ interpreter implementation. Reviewed-by: kvn Contributed-by: gbenson at redhat.com ! src/cpu/zero/vm/cppInterpreter_zero.cpp ! src/cpu/zero/vm/cppInterpreter_zero.hpp Changeset: 7b4415a18c8a Author: kvn Date: 2010-02-12 15:27 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/7b4415a18c8a Merge ! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp ! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp ! src/share/vm/includeDB_core ! src/share/vm/opto/graphKit.cpp ! src/share/vm/opto/runtime.cpp ! src/share/vm/runtime/arguments.cpp ! src/share/vm/runtime/sharedRuntime.cpp Changeset: 38836cf1d8d2 Author: tonyp Date: 2010-02-05 11:05 -0500 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/38836cf1d8d2 6920977: G1: guarantee(k == probe->klass(),"klass should be in dictionary") fails Summary: the guarantee is too strict and the test will fail (incorrectly) if the class is not in the system dictionary but in the placeholders. Reviewed-by: acorn, phh ! src/share/vm/classfile/loaderConstraints.cpp ! src/share/vm/classfile/loaderConstraints.hpp ! src/share/vm/classfile/systemDictionary.cpp ! src/share/vm/includeDB_core Changeset: 9eee977dd1a9 Author: tonyp Date: 2010-02-08 14:23 -0500 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/9eee977dd1a9 6802453: G1: hr()->is_in_reserved(from),"Precondition." Summary: The operations of re-using a RSet component and expanding the same RSet component were not mutually exlusive, and this could lead to RSets getting corrupted and entries being dropped. Reviewed-by: iveresov, johnc ! src/share/vm/gc_implementation/g1/heapRegionRemSet.cpp Changeset: 8859772195c6 Author: johnc Date: 2010-02-09 13:56 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/8859772195c6 6782663: Data produced by PrintGCApplicationConcurrentTime and PrintGCApplicationStoppedTime is not accurate. Summary: Update and display the timers associated with these flags for all safepoints. Reviewed-by: ysr, jcoomes ! src/share/vm/runtime/vmThread.cpp ! src/share/vm/services/runtimeService.cpp Changeset: 0414c1049f15 Author: iveresov Date: 2010-02-11 15:52 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/0414c1049f15 6923991: G1: improve scalability of RSet scanning Summary: Implemented block-based work stealing. Moved copying during the rset scanning phase to the main copying phase. Made the size of rset table depend on the region size. Reviewed-by: apetrusenko, tonyp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp ! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp ! src/share/vm/gc_implementation/g1/g1OopClosures.hpp ! src/share/vm/gc_implementation/g1/g1OopClosures.inline.hpp ! src/share/vm/gc_implementation/g1/g1RemSet.cpp ! src/share/vm/gc_implementation/g1/g1_globals.hpp ! src/share/vm/gc_implementation/g1/g1_specialized_oop_closures.hpp ! src/share/vm/gc_implementation/g1/heapRegionRemSet.cpp ! src/share/vm/gc_implementation/g1/heapRegionRemSet.hpp ! src/share/vm/gc_implementation/g1/sparsePRT.cpp ! src/share/vm/gc_implementation/g1/sparsePRT.hpp ! src/share/vm/memory/cardTableModRefBS.hpp ! src/share/vm/utilities/globalDefinitions.hpp Changeset: 58add740c4ee Author: johnc Date: 2010-02-16 14:11 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/58add740c4ee Merge ! src/share/vm/includeDB_core Changeset: e7b1cc79bd25 Author: kvn Date: 2010-02-16 16:17 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/e7b1cc79bd25 6926697: "optimized" VM build failed: The type "AdapterHandlerTableIterator" is incomplete Summary: Define AdapterHandlerTableIterator class as non product instead of debug. Reviewed-by: never ! src/share/vm/runtime/sharedRuntime.cpp Changeset: 106f41e88c85 Author: never Date: 2010-02-16 20:07 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/106f41e88c85 6877221: Endless deoptimizations in OSR nmethod Reviewed-by: kvn ! src/share/vm/opto/parse1.cpp Changeset: b4b440360f1e Author: twisti Date: 2010-02-18 11:35 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/b4b440360f1e 6926782: CodeBuffer size too small after 6921352 Summary: After 6921352 the CodeBuffer size was too small. Reviewed-by: kvn, never ! src/share/vm/opto/callGenerator.cpp ! src/share/vm/opto/compile.cpp ! src/share/vm/opto/compile.hpp ! src/share/vm/opto/output.cpp Changeset: 3b687c53c266 Author: twisti Date: 2010-02-18 06:54 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/3b687c53c266 6927165: Zero S/390 fixes Summary: Fixes two failures on 31-bit S/390. Reviewed-by: twisti Contributed-by: Gary Benson ! src/cpu/zero/vm/globals_zero.hpp ! src/os_cpu/linux_zero/vm/os_linux_zero.hpp Changeset: 72f1840531a4 Author: twisti Date: 2010-02-18 10:44 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/72f1840531a4 Merge Changeset: 1f341bb67b5b Author: trims Date: 2010-02-18 22:15 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/1f341bb67b5b Merge Changeset: 6c9796468b91 Author: trims Date: 2010-02-18 22:16 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/6c9796468b91 6927886: Bump the HS17 build number to 10 Summary: Update the HS17 build number to 10 Reviewed-by: jcoomes ! make/hotspot_version Changeset: c334578c3bb7 Author: mchung Date: 2010-03-05 10:00 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/c334578c3bb7 Merge ! src/share/vm/classfile/systemDictionary.cpp ! src/share/vm/runtime/arguments.cpp ! src/share/vm/runtime/arguments.hpp From mandy.chung at sun.com Fri Mar 5 10:06:18 2010 From: mandy.chung at sun.com (mandy.chung at sun.com) Date: Fri, 05 Mar 2010 18:06:18 +0000 Subject: hg: jigsaw/jigsaw/jdk: 13 new changesets Message-ID: <20100305181022.2D93A43F17@hg.openjdk.java.net> Changeset: 2ba381560071 Author: dcherepanov Date: 2010-02-12 19:58 +0300 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/2ba381560071 6705345: Enable multiple file selection in AWT FileDialog Reviewed-by: art, anthony, alexp ! src/share/classes/java/awt/FileDialog.java ! src/share/classes/sun/awt/AWTAccessor.java ! src/solaris/classes/sun/awt/X11/XFileDialogPeer.java ! src/windows/classes/sun/awt/windows/WFileDialogPeer.java ! src/windows/native/sun/windows/awt_FileDialog.cpp ! src/windows/native/sun/windows/awt_FileDialog.h + test/java/awt/FileDialog/MultipleMode/MultipleMode.html + test/java/awt/FileDialog/MultipleMode/MultipleMode.java Changeset: d6d2de6ee2d1 Author: lana Date: 2010-02-19 15:13 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/d6d2de6ee2d1 Merge Changeset: 83c34a6b1458 Author: mchung Date: 2010-02-08 23:02 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/83c34a6b1458 6924497: HotSpotDiagnosticsMXBean.getDiagnosticOptions throws NPE Summary: Check if the element in the flags array is non-null to filter unsupported flags Reviewed-by: dcubed ! src/share/classes/sun/management/Flag.java ! src/share/native/sun/management/Flag.c Changeset: ec438f2b6886 Author: chegar Date: 2010-02-10 13:23 +0000 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/ec438f2b6886 6693244: Java Web Start app fails on 6u10 beta w/ AssertionError in AuthenticationInfo.requestCompleted Reviewed-by: michaelm ! src/share/classes/sun/net/www/protocol/http/AuthenticationInfo.java ! test/ProblemList.txt ! test/java/net/Authenticator/B4769350.java Changeset: 784e52734b8d Author: mchung Date: 2010-02-10 17:51 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/784e52734b8d 6915413: Module build: building of specified jdk components instead of all Summary: Define new SUBDIRS_* variables for specifying components for one group Reviewed-by: ohair ! make/Makefile ! make/com/Makefile ! make/com/sun/Makefile ! make/com/sun/demo/Makefile ! make/com/sun/demo/jvmti/Makefile ! make/com/sun/inputmethods/Makefile ! make/com/sun/java/Makefile ! make/com/sun/java/browser/Makefile ! make/com/sun/jmx/Makefile ! make/com/sun/jndi/Makefile ! make/com/sun/jndi/rmi/Makefile ! make/com/sun/nio/Makefile ! make/com/sun/org/Makefile ! make/com/sun/org/apache/Makefile ! make/com/sun/security/Makefile ! make/com/sun/tools/Makefile ! make/com/sun/tracing/Makefile ! make/common/Defs.gmk ! make/common/Sanity.gmk + make/common/Subdirs.gmk ! make/common/shared/Sanity.gmk ! make/java/Makefile ! make/java/hpi/Makefile ! make/java/java/Makefile ! make/java/java/genlocales.gmk ! make/java/main/Makefile ! make/java/nio/FILES_java.gmk ! make/java/nio/Makefile + make/java/nio/mxbean/Makefile ! make/java/redist/Makefile - make/java/text/FILES_java.gmk ! make/java/text/Makefile + make/java/text/base/FILES_java.gmk + make/java/text/base/Makefile + make/java/text/bidi/Makefile ! make/javax/Makefile ! make/javax/rmi/Makefile ! make/javax/sound/Makefile ! make/javax/swing/Makefile ! make/jpda/Makefile ! make/jpda/transport/Makefile ! make/mkdemo/Makefile ! make/mkdemo/applets/Makefile ! make/mkdemo/jfc/Makefile ! make/mkdemo/jni/Makefile ! make/mkdemo/jvmti/Makefile ! make/mkdemo/management/Makefile ! make/mkdemo/scripting/Makefile ! make/mksample/Makefile ! make/mksample/jmx/Makefile ! make/mksample/nio/Makefile ! make/mksample/scripting/Makefile ! make/mksample/webservices/Makefile ! make/org/Makefile ! make/org/ietf/Makefile ! make/sun/Makefile ! make/sun/cmm/Makefile ! make/sun/image/Makefile ! make/sun/management/Makefile ! make/sun/net/Makefile ! make/sun/net/spi/Makefile ! make/sun/net/spi/nameservice/Makefile ! make/sun/nio/Makefile ! make/sun/org/Makefile ! make/sun/org/mozilla/Makefile ! make/sun/rmi/Makefile ! make/sun/security/Makefile ! make/sun/tracing/Makefile ! make/tools/Makefile Changeset: d7d8807fca86 Author: weijun Date: 2010-02-12 10:24 +0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/d7d8807fca86 6925639: keytool -genkeypair -help missing dname option Reviewed-by: mullan ! src/share/classes/sun/security/tools/KeyTool.java Changeset: 74f493fae483 Author: mchung Date: 2010-02-12 11:33 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/74f493fae483 6925868: Eliminate pack200's dependency on logging Summary: Replace j.u.l.Logger with sun.util.logging.PlatformLogger Reviewed-by: alanb, forax ! src/share/classes/com/sun/java/util/jar/pack/PackageReader.java ! src/share/classes/com/sun/java/util/jar/pack/PackageWriter.java ! src/share/classes/com/sun/java/util/jar/pack/Utils.java Changeset: 328c5d3974fe Author: mchung Date: 2010-02-15 10:18 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/328c5d3974fe Merge Changeset: 84792500750c Author: lana Date: 2010-02-17 10:24 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/84792500750c Merge Changeset: e83d9c0d5e95 Author: chegar Date: 2010-02-22 15:27 +0000 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/e83d9c0d5e95 6912868: "java.net.useSystemProxies" behavior fails to check "use_same_proxy" in GNOME Reviewed-by: alanb, chegar Contributed-by: damjan.jov at gmail.com ! src/solaris/native/sun/net/spi/DefaultProxySelector.c + test/java/net/ProxySelector/SystemProxies.java Changeset: c96d6cb31723 Author: chegar Date: 2010-02-23 17:08 +0000 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/c96d6cb31723 6365587: Proxy-Connection header sent through tunnel Reviewed-by: michaelm ! src/share/classes/sun/net/www/protocol/http/HttpURLConnection.java Changeset: b396584a3e64 Author: lana Date: 2010-02-23 10:17 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/b396584a3e64 Merge - make/java/text/FILES_java.gmk Changeset: 15b9074be17f Author: mchung Date: 2010-03-05 10:00 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/15b9074be17f Merge ! make/Makefile ! make/common/Defs.gmk ! make/java/java/Makefile - make/java/text/FILES_java.gmk ! make/modules/BuildPackages.gmk ! make/modules/Makefile ! make/org/Makefile From Mandy.Chung at Sun.COM Fri Mar 5 10:15:59 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Fri, 05 Mar 2010 10:15:59 -0800 Subject: Add modules target in the forest's topdir Makefile Message-ID: <4B914A5F.2000905@sun.com> Kelly, I'm adding a "modules" target in the topdir's Makefile that will pass the modules target to the jdk build. This provides a convenience way to build the jdk module images. In addition, the jprt build will archive the jdk-module-image and use it for testing. Can you please review this change? http://cr.openjdk.java.net/~mchung/jigsaw/modules-target/ Thanks Mandy From forax at univ-mlv.fr Fri Mar 5 10:39:32 2010 From: forax at univ-mlv.fr (=?UTF-8?B?UsOpbWkgRm9yYXg=?=) Date: Fri, 05 Mar 2010 19:39:32 +0100 Subject: Is readonly file mapping a purpose of this project? In-Reply-To: References: <4B90284F.2050502@univ-mlv.fr> Message-ID: <4B914FE4.50502@univ-mlv.fr> Le 05/03/2010 12:18, Ram?n Garc?a a ?crit : > No, you did not understand my request .Class data sharing is limited > to system classes. No, CDS is controlled by a file named classlist in jre/lib. You can change it and regenerate a CDS image with -Xshare:dump > I am asking a module to be a file that can be > distributed, and efficiently memory mapped and therefore shared. > The module architecture enables that kind of optimisations ; another one is Ahead Of Time compilation ; but you need time to implement them. > Android has a similar feature > http://www.youtube.com/watch?v=ptjedOZEXPM > R?mi > On Thu, Mar 4, 2010 at 10:38 PM, R?mi Forax wrote: > >> Le 04/03/2010 16:34, Ram?n Garc?a a ?crit : >> >>> It would save memory if the Java virtual machine were able to load a >>> module file by mmap-ing it readonly and finding the Java class objects >>> in it. In this way, several virtual machine instances using the same >>> libraries would share memory. Is this among the features planned for >>> this project? >>> >>> >> This is a feature of SUN jdk5 :) >> http://java.sun.com/j2se/1.5.0/docs/guide/vm/class-data-sharing.html >> >> R?mi >> >> >> From Jonathan.Gibbons at Sun.COM Fri Mar 5 10:34:47 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 05 Mar 2010 10:34:47 -0800 Subject: upcoming merge Message-ID: <4B914EC7.3020804@sun.com> Jigsaw folk, What are the expected mechanics for the upcoming merge of the Jigsaw code into JDK? First, I guess, what is the expected integration area for the merge? tl, I presume? Second, I presume we cannot simply push from the jigsaw forest into the tl forest, because of the wild and sundry nature of the individual changesets in the jigsaw forest -- jcheck will prevent that. So, how is it expected that we will review and push the bits? I'm not seeing much evidence of any formal or auditable review process for jigsaw at this point. For my part, in langtools, I would prefer to stage the work into the integration area via a (small) number of formally reviewed changesets. Is it practical to make that happen? Does the overall merge have to happen somewhat atomically, or can we stage work into the integration area (e.g. tl) providing that at each stage we pass a basic set of quality tests (e.g. full build cycle, pass standard regression tests, etc) -- Jon From Mandy.Chung at Sun.COM Fri Mar 5 10:45:58 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Fri, 05 Mar 2010 10:45:58 -0800 Subject: upcoming merge In-Reply-To: <4B914EC7.3020804@sun.com> References: <4B914EC7.3020804@sun.com> Message-ID: <4B915166.6030302@sun.com> Adding one more to the complexity, the jigsaw jdk depends on a new hotspot entry point. To avoid the flag day, the jigsaw hotspot change has to be integrated to jdk7 one build before the jigsaw jdk change to go in. Mandy Jonathan Gibbons wrote: > Jigsaw folk, > > What are the expected mechanics for the upcoming merge of the Jigsaw > code into JDK? > > First, I guess, what is the expected integration area for the merge? > tl, I presume? > > Second, I presume we cannot simply push from the jigsaw forest into > the tl forest, because of the wild and sundry nature of the individual > changesets in the jigsaw forest -- jcheck will prevent that. > So, how is it expected that we will review and push the bits? I'm not > seeing much evidence of any formal or auditable review process for > jigsaw at this point. > > > For my part, in langtools, I would prefer to stage the work into the > integration area via a (small) number of formally reviewed changesets. > Is it practical to make that happen? Does the overall merge have to > happen somewhat atomically, or can we stage work into the integration > area (e.g. tl) providing that at each stage we pass a basic set of > quality tests (e.g. full build cycle, pass standard regression tests, > etc) > > -- Jon From Kelly.Ohair at Sun.COM Fri Mar 5 11:46:38 2010 From: Kelly.Ohair at Sun.COM (Kelly O'Hair) Date: Fri, 05 Mar 2010 11:46:38 -0800 Subject: Add modules target in the forest's topdir Makefile In-Reply-To: <4B914A5F.2000905@sun.com> References: <4B914A5F.2000905@sun.com> Message-ID: <4B915F9E.70306@sun.com> I'm not sure you need the ifdef HAVE_JPRT_SAVE_MODULES I suspect you can just use ifdef BUILD_MODULES, assuming if modules were built, you would always want them bundled up and returned back, but it's up to you. Otherwise, looks fine. -kto On 3/5/10 10:15 AM, Mandy Chung wrote: > Kelly, > > I'm adding a "modules" target in the topdir's Makefile that will pass > the modules target to the jdk build. This provides a convenience way to > build the jdk module images. In addition, the jprt build will archive > the jdk-module-image and use it for testing. > > Can you please review this change? > http://cr.openjdk.java.net/~mchung/jigsaw/modules-target/ > > Thanks > Mandy From Mandy.Chung at Sun.COM Fri Mar 5 12:04:59 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Fri, 05 Mar 2010 12:04:59 -0800 Subject: Add modules target in the forest's topdir Makefile In-Reply-To: <4B915F9E.70306@sun.com> References: <4B914A5F.2000905@sun.com> <4B915F9E.70306@sun.com> Message-ID: <4B9163EB.7090306@sun.com> Kelly O'Hair wrote: > > I'm not sure you need the > ifdef HAVE_JPRT_SAVE_MODULES > Yes. I would always want to bundle the modules up. I'll replace it with ifdef BUILD_MODULES. Some time in the future, I would want some way to specify what modules to build and install on top of the base-image and perform the appropriate testing. jdk-module-image won't need to be bundled at that time. Thanks for the review. Mandy > I suspect you can just use ifdef BUILD_MODULES, assuming if > modules were built, you would always want them bundled up and > returned back, but it's up to you. > > Otherwise, looks fine. > > -kto > > On 3/5/10 10:15 AM, Mandy Chung wrote: >> Kelly, >> >> I'm adding a "modules" target in the topdir's Makefile that will pass >> the modules target to the jdk build. This provides a convenience way to >> build the jdk module images. In addition, the jprt build will archive >> the jdk-module-image and use it for testing. >> >> Can you please review this change? >> http://cr.openjdk.java.net/~mchung/jigsaw/modules-target/ >> >> Thanks >> Mandy From mandy.chung at sun.com Fri Mar 5 12:17:57 2010 From: mandy.chung at sun.com (mandy.chung at sun.com) Date: Fri, 05 Mar 2010 20:17:57 +0000 Subject: hg: jigsaw/jigsaw: Support "modules" target and BUILD_MODULES buildenv in the forest build Message-ID: <20100305201757.D53A743F35@hg.openjdk.java.net> Changeset: 00c40d39181b Author: mchung Date: 2010-03-05 12:16 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/rev/00c40d39181b Support "modules" target and BUILD_MODULES buildenv in the forest build Summary: gnumake modules or gnumake BUILD_MODULES=all all to build modules Reviewed-by: ohair ! Makefile ! make/jdk-rules.gmk ! make/jprt.gmk From Mandy.Chung at Sun.COM Fri Mar 5 13:58:11 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Fri, 05 Mar 2010 13:58:11 -0800 Subject: Review request for the fix to generate jmod files for jdk modules Message-ID: <4B917E73.7030507@sun.com> Hi Dalibor, Can you review the change to generate jmod files for jdk modules? Webrev: http://cr.openjdk.java.net/~mchung/jigsaw/gen-jmod-files/ The modules build will generate jmod files under $outputdir/jigsaw-pkgs/jmod directory. The debian packages are in $outputdir/jigsaw-pkgs/deb. It calls jpkg to create the jmod files for jdk modules. I temporarily disable ensureShortNativePath check in ModuleFileFormat.java until we make the jdk changes to conform the module file format spec (i.e. the path names of native-code files must not include more than one element and no execute permission set). include header files are in the jdk.boot module and only copied to jdk-base-image and jdk-module-image. In addition, I clean up Modules.gmk as there is no reason to have the jre subdirectory in the jdk-module-image and jdk-base-image. They should be the same as jre-base-image with a set of jdk modules installed. Thanks Mandy From Dalibor.Topic at Sun.COM Fri Mar 5 14:40:19 2010 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Fri, 05 Mar 2010 23:40:19 +0100 Subject: Review request for the fix to generate jmod files for jdk modules In-Reply-To: <4B917E73.7030507@sun.com> References: <4B917E73.7030507@sun.com> Message-ID: <4B918853.1040105@sun.com> Mandy Chung wrote: > Hi Dalibor, > > Can you review the change to generate jmod files for > jdk modules? Looks fine to me, thanks Mandy! > I temporarily disable ensureShortNativePath check in > ModuleFileFormat.java until we make the jdk changes > to conform the module file format spec (i.e. the path > names of native-code files must not include more than > one element and no execute permission set). OK. > In addition, I clean up Modules.gmk as there is no reason > to have the jre subdirectory in the jdk-module-image > and jdk-base-image. They should be the same as jre-base-image with a > set of jdk modules installed. Sounds good, too. cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der, Wolfgang Engels Vorsitzender des Aufsichtsrates: Martin H?ring From mandy.chung at sun.com Fri Mar 5 15:29:00 2010 From: mandy.chung at sun.com (mandy.chung at sun.com) Date: Fri, 05 Mar 2010 23:29:00 +0000 Subject: hg: jigsaw/jigsaw/jdk: Generate jmod files for jdk modules Message-ID: <20100305232924.CCEF543F63@hg.openjdk.java.net> Changeset: d2eb7c15c45c Author: mchung Date: 2010-03-05 15:28 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/d2eb7c15c45c Generate jmod files for jdk modules Summary: Generate jmod files for jdk modules Reviewed-by: robilad ! make/common/BuildPackages.gmk < make/modules/BuildPackages.gmk ! make/common/Defs.gmk ! make/common/Modules.gmk ! make/java/jvm/Makefile ! make/jpda/transport/Makefile ! make/modules/Makefile ! make/sun/jawt/Makefile ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java From Alan.Bateman at Sun.COM Tue Mar 9 02:25:48 2010 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 09 Mar 2010 10:25:48 +0000 Subject: modules.config updates Message-ID: <4B96222C.7040306@sun.com> I'd like to fix up a couple of the modules definitions - should I push them to the jigsaw/jdk repo or TL? I don't want to make the merge too hard so I'm okay to push these changes to the jigsaw repo. -Alan. From Mandy.Chung at Sun.COM Tue Mar 9 08:20:18 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Tue, 09 Mar 2010 08:20:18 -0800 Subject: modules.config updates In-Reply-To: <4B96222C.7040306@sun.com> References: <4B96222C.7040306@sun.com> Message-ID: <4B967542.9090508@sun.com> The modules.config in the jigsaw repo is the most up-to-date version. I suggest to push your changes to jigsaw repo and you would be able to catch any issue to the modules build with your change such as modules installation. Thanks Mandy Alan Bateman wrote: > I'd like to fix up a couple of the modules definitions - should I push > them to the jigsaw/jdk repo or TL? I don't want to make the merge too > hard so I'm okay to push these changes to the jigsaw repo. > > -Alan. From Alan.Bateman at Sun.COM Tue Mar 9 10:02:52 2010 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 09 Mar 2010 18:02:52 +0000 Subject: modules.config updates In-Reply-To: <4B967542.9090508@sun.com> References: <4B96222C.7040306@sun.com> <4B967542.9090508@sun.com> Message-ID: <4B968D4C.5020003@sun.com> Mandy Chung wrote: > The modules.config in the jigsaw repo is the most up-to-date version. > I suggest to push your changes to jigsaw repo and you would be able to > catch any issue to the modules build with your change such as modules > installation. Will do - thanks! From mr at sun.com Tue Mar 9 10:42:31 2010 From: mr at sun.com (Mark Reinhold) Date: Tue, 09 Mar 2010 10:42:31 -0800 Subject: upcoming merge In-Reply-To: jonathan.gibbons@sun.com; Fri, 05 Mar 2010 10:34:47 PST; <4B914EC7.3020804@sun.com> Message-ID: <20100309184231.C111F574@eggemoggin.niobe.net> > Date: Fri, 05 Mar 2010 10:34:47 -0800 > From: jonathan.gibbons at sun.com > What are the expected mechanics for the upcoming merge of the Jigsaw code into > JDK? > > First, I guess, what is the expected integration area for the merge? tl, I > presume? Yes. > Second, I presume we cannot simply push from the jigsaw forest into the tl > forest, because of the wild and sundry nature of the individual changesets in > the jigsaw forest -- jcheck will prevent that. Right. > So, how is it expected that we will review and push the bits? I'm not seeing > much evidence of any formal or auditable review process for jigsaw at this > point. That's been intentional during early development. The fun will end soon. > For my part, in langtools, I would prefer to stage the work into the > integration area via a (small) number of formally reviewed changesets. Is it > practical to make that happen? Does the overall merge have to happen somewhat > atomically, or can we stage work into the integration area (e.g. tl) providing > that at each stage we pass a basic set of quality tests (e.g. full build cycle, > pass standard regression tests, etc) I think the initial merge needs to be done in two steps: First the VM changes, as Mandy pointed out, and then the jdk and langtools changes. If you want to do javac in phases then we could integrate the current jigsaw langtools and jdk repos in the same step, and then you could push the fully-module-aware javac later on. Given all the interdependences I'm not sure it'd be worth slicing the merge any more finely than that, but I'm open to suggestions. Once we finish the current work in the jdk repo -- hopefully in the next week or so -- my plan is to generate a big webrev against the current TL forest which we can parcel out piecewise to appropriate reviewers. In parallel with that we'll draft some basic documentation and submit the CCC requests. - Mark From Jonathan.Gibbons at Sun.COM Tue Mar 9 12:16:38 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Tue, 09 Mar 2010 12:16:38 -0800 Subject: can't build jigsaw with new javac Message-ID: <4B96ACA6.1070203@sun.com> Mandy, What are you trying to do here? You can't have -Xbootclasspath and -modulepath. > /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/bin/javac -source 7 > -target 7 -encoding ascii > "-Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/classes" > -d /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ > > -Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/classes\ > -modulepath > /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ > -sourcepath > /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src \ > > /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src/*/module-info.java > javac: cannot specify both -bootclasspath and -modulepath > Usage: javac > use -help for a list of possible options See this response from Mark back in December, available here: http://mail.openjdk.java.net/pipermail/jigsaw-dev/2009-December/000421.html > >/ -- do we allow or forbid any use of bootclasspath with any modular code? > / > Forbid. > -- Jon From Mandy.Chung at Sun.COM Tue Mar 9 13:38:37 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Tue, 09 Mar 2010 13:38:37 -0800 Subject: can't build jigsaw with new javac In-Reply-To: <4B96ACA6.1070203@sun.com> References: <4B96ACA6.1070203@sun.com> Message-ID: <4B96BFDD.4020901@Sun.com> On 03/09/10 12:16, Jonathan Gibbons wrote: > Mandy, > > What are you trying to do here? You can't have -Xbootclasspath and > -modulepath. That was a hack for javac to find classes that are not in the default bootclasspath (e.g. lib/sa-jdi.jar). For example, $outputdir/tmp/modules/src/jsadebugd/module-info.java has a main class of sun.jvm.hotspot.jdi.SADebugServer that is not in $outputdir/classes Is there a better way to do this? Mandy > >> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/bin/javac -source 7 >> -target 7 -encoding ascii >> "-Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/classes" >> -d /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >> >> -Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/classes\ >> -modulepath >> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >> -sourcepath >> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src \ >> >> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src/*/module-info.java >> javac: cannot specify both -bootclasspath and -modulepath >> Usage: javac >> use -help for a list of possible options > > See this response from Mark back in December, available here: > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2009-December/000421.html > >> >/ -- do we allow or forbid any use of bootclasspath with any modular code? >> / >> Forbid. >> > > > -- Jon From Jonathan.Gibbons at Sun.COM Tue Mar 9 14:16:54 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Tue, 09 Mar 2010 14:16:54 -0800 Subject: can't build jigsaw with new javac In-Reply-To: <4B96BFDD.4020901@Sun.com> References: <4B96ACA6.1070203@sun.com> <4B96BFDD.4020901@Sun.com> Message-ID: <4B96C8D6.9020202@sun.com> Uuugh. Well, you've got a mixed up world with the classes in a classes/ hierarchy and the module-info files in a modules hierarchy under tmp/modules/src. On top of which you've got an empty directory on modulepath and no -d. No wonder javac is confused ;-) It would seem that we either have to resolve the conflict between the classes/ and module-info hierarchies, or we have to revisit the decision to forbid use of bootclasspath with module compilation for javac. -- Jon Mandy Chung wrote: > On 03/09/10 12:16, Jonathan Gibbons wrote: >> Mandy, >> >> What are you trying to do here? You can't have -Xbootclasspath and >> -modulepath. > > That was a hack for javac to find classes that are not in the default > bootclasspath (e.g. lib/sa-jdi.jar). > > For example, $outputdir/tmp/modules/src/jsadebugd/module-info.java > has a main class of sun.jvm.hotspot.jdi.SADebugServer that is not > in $outputdir/classes > > Is there a better way to do this? > > Mandy > >> >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/bin/javac -source 7 >>> -target 7 -encoding ascii >>> "-Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/classes" >>> -d /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>> >>> -Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/classes\ >>> >>> -modulepath >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>> -sourcepath >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src \ >>> >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src/*/module-info.java >>> >>> javac: cannot specify both -bootclasspath and -modulepath >>> Usage: javac >>> use -help for a list of possible options >> >> See this response from Mark back in December, available here: >> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2009-December/000421.html >> >> >>> >/ -- do we allow or forbid any use of bootclasspath with any >>> modular code? >>> / >>> Forbid. >>> >> >> >> -- Jon From Jonathan.Gibbons at Sun.COM Tue Mar 9 17:55:40 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Tue, 09 Mar 2010 17:55:40 -0800 Subject: can't build jigsaw with new javac In-Reply-To: <4B96BFDD.4020901@Sun.com> References: <4B96ACA6.1070203@sun.com> <4B96BFDD.4020901@Sun.com> Message-ID: <4B96FC1C.4010004@sun.com> Mandy, It seems anomalous that there are two different values for -Xbootclasspath in the modularize target. > > /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/bin/javac -source 7 > -target 7 -encoding ascii > "-Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/classes" > -d /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ > > -Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/classes\ > > -modulepath > /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ > -sourcepath > /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src \ > > /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src/*/module-info.java > > javac: cannot specify both -bootclasspath and -modulepath > Usage: javac > use -help for a list of possible options One value comes in via $(JAVACFLAGS), the other is explicit. -- Jon Mandy Chung wrote: > On 03/09/10 12:16, Jonathan Gibbons wrote: >> Mandy, >> >> What are you trying to do here? You can't have -Xbootclasspath and >> -modulepath. > > That was a hack for javac to find classes that are not in the default > bootclasspath (e.g. lib/sa-jdi.jar). > > For example, $outputdir/tmp/modules/src/jsadebugd/module-info.java > has a main class of sun.jvm.hotspot.jdi.SADebugServer that is not > in $outputdir/classes > > Is there a better way to do this? > > Mandy > >> >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/bin/javac -source 7 >>> -target 7 -encoding ascii >>> "-Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/classes" >>> -d /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>> >>> -Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/classes\ >>> >>> -modulepath >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>> -sourcepath >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src \ >>> >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src/*/module-info.java >>> >>> javac: cannot specify both -bootclasspath and -modulepath >>> Usage: javac >>> use -help for a list of possible options >> >> See this response from Mark back in December, available here: >> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2009-December/000421.html >> >> >>> >/ -- do we allow or forbid any use of bootclasspath with any >>> modular code? >>> / >>> Forbid. >>> >> >> >> -- Jon From Mandy.Chung at Sun.COM Tue Mar 9 18:42:09 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Tue, 09 Mar 2010 18:42:09 -0800 Subject: can't build jigsaw with new javac In-Reply-To: <4B96FC1C.4010004@sun.com> References: <4B96ACA6.1070203@sun.com> <4B96BFDD.4020901@Sun.com> <4B96FC1C.4010004@sun.com> Message-ID: <4B970701.1030406@sun.com> Jonathan Gibbons wrote: > Mandy, > > It seems anomalous that there are two different values for > -Xbootclasspath in the modularize target. > >> >> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/bin/javac -source 7 >> -target 7 -encoding ascii >> "-Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/classes" > This is the value coming from $(JAVACFLAGS) that doesn't have some classes from other jar files (sa-jdi.jar). If sa-jdi.jar is the only missing jar file (I have to double check), I could use -Xbootclasspath/a instead >> -d /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >> >> -Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/classes\ >> That's the value I want. I unjar all jdk jar files in $outputdir/tmp/modules/classes as part of the modularization process. Since $outputdir/tmp/modules/classes has all the jdk classes, I took the easiest approach to replace the entire bootclasspath with that. What about if I use -J-Xbootclasspath:$outputdir/tmp/modules/classes? Would javac be able to look up all jdk modules and compile all module-info.java? Would that be a better temporary workaround for now? Mandy >> -modulepath >> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >> -sourcepath >> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src \ >> >> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src/*/module-info.java >> >> javac: cannot specify both -bootclasspath and -modulepath >> Usage: javac >> use -help for a list of possible options > > One value comes in via $(JAVACFLAGS), the other is explicit. > > -- Jon > > > Mandy Chung wrote: >> On 03/09/10 12:16, Jonathan Gibbons wrote: >>> Mandy, >>> >>> What are you trying to do here? You can't have -Xbootclasspath and >>> -modulepath. >> >> That was a hack for javac to find classes that are not in the default >> bootclasspath (e.g. lib/sa-jdi.jar). >> >> For example, $outputdir/tmp/modules/src/jsadebugd/module-info.java >> has a main class of sun.jvm.hotspot.jdi.SADebugServer that is not >> in $outputdir/classes >> >> Is there a better way to do this? >> >> Mandy >> >>> >>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/bin/javac -source >>>> 7 -target 7 -encoding ascii >>>> "-Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/classes" >>>> -d /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>>> >>>> -Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/classes\ >>>> >>>> -modulepath >>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>>> -sourcepath >>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src \ >>>> >>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src/*/module-info.java >>>> >>>> javac: cannot specify both -bootclasspath and -modulepath >>>> Usage: javac >>>> use -help for a list of possible options >>> >>> See this response from Mark back in December, available here: >>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2009-December/000421.html >>> >>> >>>> >/ -- do we allow or forbid any use of bootclasspath with any >>>> modular code? >>>> / >>>> Forbid. >>>> >>> >>> >>> -- Jon > From Jonathan.Gibbons at Sun.COM Tue Mar 9 18:49:11 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Tue, 09 Mar 2010 18:49:11 -0800 Subject: can't build jigsaw with new javac In-Reply-To: <4B970701.1030406@sun.com> References: <4B96ACA6.1070203@sun.com> <4B96BFDD.4020901@Sun.com> <4B96FC1C.4010004@sun.com> <4B970701.1030406@sun.com> Message-ID: <4B9708A7.5080605@sun.com> Mandy Chung wrote: > Jonathan Gibbons wrote: >> Mandy, >> >> It seems anomalous that there are two different values for >> -Xbootclasspath in the modularize target. >> >>> >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/bin/javac -source 7 >>> -target 7 -encoding ascii >>> "-Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/classes" >> >> > > This is the value coming from $(JAVACFLAGS) that doesn't have some > classes from other jar files (sa-jdi.jar). If sa-jdi.jar is the only > missing jar file (I have to double check), I could use > -Xbootclasspath/a instead >>> -d /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>> >>> -Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/classes\ >>> > > That's the value I want. I unjar all jdk jar files in > $outputdir/tmp/modules/classes as part of the modularization process. > > Since $outputdir/tmp/modules/classes has all the jdk classes, I took > the easiest approach to replace the entire bootclasspath with that. > > What about if I use -J-Xbootclasspath:$outputdir/tmp/modules/classes? > Would javac be able to look up all jdk modules and compile all > module-info.java? Would that be a better temporary workaround for now? -J-Xbootclasspath means something completely different and affects the path used to find javac's own classes. Almost certainly, you don't mean that. Tomorrow I will look at removing -Xbootclasspath from the command line used for the modularize target, to see whether that is sufficient to fix the current problem. -- Jon > > Mandy > >>> -modulepath >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>> -sourcepath >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src \ >>> >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src/*/module-info.java >>> >>> javac: cannot specify both -bootclasspath and -modulepath >>> Usage: javac >>> use -help for a list of possible options >> >> One value comes in via $(JAVACFLAGS), the other is explicit. >> >> -- Jon >> >> >> Mandy Chung wrote: >>> On 03/09/10 12:16, Jonathan Gibbons wrote: >>>> Mandy, >>>> >>>> What are you trying to do here? You can't have -Xbootclasspath and >>>> -modulepath. >>> >>> That was a hack for javac to find classes that are not in the default >>> bootclasspath (e.g. lib/sa-jdi.jar). >>> >>> For example, $outputdir/tmp/modules/src/jsadebugd/module-info.java >>> has a main class of sun.jvm.hotspot.jdi.SADebugServer that is not >>> in $outputdir/classes >>> >>> Is there a better way to do this? >>> >>> Mandy >>> >>>> >>>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/bin/javac -source >>>>> 7 -target 7 -encoding ascii >>>>> "-Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/classes" >>>>> -d /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>>>> >>>>> -Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/classes\ >>>>> >>>>> -modulepath >>>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>>>> -sourcepath >>>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src \ >>>>> >>>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src/*/module-info.java >>>>> >>>>> javac: cannot specify both -bootclasspath and -modulepath >>>>> Usage: javac >>>>> use -help for a list of possible options >>>> >>>> See this response from Mark back in December, available here: >>>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2009-December/000421.html >>>> >>>> >>>>> >/ -- do we allow or forbid any use of bootclasspath with any >>>>> modular code? >>>>> / >>>>> Forbid. >>>>> >>>> >>>> >>>> -- Jon >> > From Jonathan.Gibbons at Sun.COM Wed Mar 10 09:58:05 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Wed, 10 Mar 2010 09:58:05 -0800 Subject: can't build jigsaw with new javac In-Reply-To: <4B9708A7.5080605@sun.com> References: <4B96ACA6.1070203@sun.com> <4B96BFDD.4020901@Sun.com> <4B96FC1C.4010004@sun.com> <4B970701.1030406@sun.com> <4B9708A7.5080605@sun.com> Message-ID: <4B97DDAD.1080701@sun.com> Mandy, For the immediate term, I'm still hoping we can get away with removing -Xbootclasspath from that command. Long term, I don't think we can avoid allowing and supporting -Xbootclasspath again, because we'll potentially need access to those class files as soon as someone puts annotations in the module-info files. -- Jon Jonathan Gibbons wrote: > Mandy Chung wrote: >> Jonathan Gibbons wrote: >>> Mandy, >>> >>> It seems anomalous that there are two different values for >>> -Xbootclasspath in the modularize target. >>> >>>> >>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/bin/javac -source >>>> 7 -target 7 -encoding ascii >>>> "-Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/classes" >>> >>> >>> >> >> This is the value coming from $(JAVACFLAGS) that doesn't have some >> classes from other jar files (sa-jdi.jar). If sa-jdi.jar is the only >> missing jar file (I have to double check), I could use >> -Xbootclasspath/a instead >>>> -d /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>>> >>>> -Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/classes\ >>>> >> >> That's the value I want. I unjar all jdk jar files in >> $outputdir/tmp/modules/classes as part of the modularization process. >> >> Since $outputdir/tmp/modules/classes has all the jdk classes, I took >> the easiest approach to replace the entire bootclasspath with that. >> >> What about if I use >> -J-Xbootclasspath:$outputdir/tmp/modules/classes? Would javac be >> able to look up all jdk modules and compile all module-info.java? >> Would that be a better temporary workaround for now? > > -J-Xbootclasspath means something completely different and affects the > path used to find javac's own classes. Almost certainly, you don't > mean that. Tomorrow I will look at removing -Xbootclasspath from the > command line used for the modularize target, to see whether that is > sufficient to fix the current problem. > > -- Jon > >> >> Mandy >> >>>> -modulepath >>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>>> -sourcepath >>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src \ >>>> >>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src/*/module-info.java >>>> >>>> javac: cannot specify both -bootclasspath and -modulepath >>>> Usage: javac >>>> use -help for a list of possible options >>> >>> One value comes in via $(JAVACFLAGS), the other is explicit. >>> >>> -- Jon >>> >>> >>> Mandy Chung wrote: >>>> On 03/09/10 12:16, Jonathan Gibbons wrote: >>>>> Mandy, >>>>> >>>>> What are you trying to do here? You can't have -Xbootclasspath >>>>> and -modulepath. >>>> >>>> That was a hack for javac to find classes that are not in the default >>>> bootclasspath (e.g. lib/sa-jdi.jar). >>>> >>>> For example, $outputdir/tmp/modules/src/jsadebugd/module-info.java >>>> has a main class of sun.jvm.hotspot.jdi.SADebugServer that is not >>>> in $outputdir/classes >>>> >>>> Is there a better way to do this? >>>> >>>> Mandy >>>> >>>>> >>>>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/bin/javac >>>>>> -source 7 -target 7 -encoding ascii >>>>>> "-Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/classes" >>>>>> -d /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>>>>> >>>>>> -Xbootclasspath:/mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/classes\ >>>>>> >>>>>> -modulepath >>>>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/modules \ >>>>>> -sourcepath >>>>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src \ >>>>>> >>>>>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/tmp/modules/src/*/module-info.java >>>>>> >>>>>> javac: cannot specify both -bootclasspath and -modulepath >>>>>> Usage: javac >>>>>> use -help for a list of possible options >>>>> >>>>> See this response from Mark back in December, available here: >>>>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2009-December/000421.html >>>>> >>>>> >>>>>> >/ -- do we allow or forbid any use of bootclasspath with any >>>>>> modular code? >>>>>> / >>>>>> Forbid. >>>>>> >>>>> >>>>> >>>>> -- Jon >>> >> > From dalibor.topic at sun.com Thu Mar 11 15:54:57 2010 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Thu, 11 Mar 2010 23:54:57 +0000 Subject: hg: jigsaw/jigsaw/jdk: Merged in fix to close all opened classes from mr Message-ID: <20100311235548.C264B447A5@hg.openjdk.java.net> Changeset: 3cf5ddc582ac Author: robilad Date: 2010-03-12 00:29 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/3cf5ddc582ac Merged in fix to close all opened classes from mr ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java From Jonathan.Gibbons at Sun.COM Thu Mar 11 16:11:52 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Thu, 11 Mar 2010 16:11:52 -0800 Subject: platform requirements Message-ID: <4B9986C8.8000608@sun.com> Mark, In a collection of resolved modules, are there any invariants or guarantees about those modules which are identified as platform modules by Platform.isPlatformModuleName ? For example: -- will there be only one and only one such module in any collection of resolved modules -- will such modules always have no other dependencies? I'm having to support -Xbootclasspath* options in javac to cope with the current jigsaw build, and I'm trying to figure out how tolerant I need to be. -- Jon From Jonathan.Gibbons at Sun.COM Thu Mar 11 17:59:15 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Thu, 11 Mar 2010 17:59:15 -0800 Subject: jmod problem Message-ID: <4B999FF3.1020008@sun.com> I have a reasonably up to date copy of the jigsaw forest, and I have built it successfully on Ubuntu. If I try and run jmod to create a simple library, it tells me it cannot find the parent library, and indeed, the path it specifies does not exist. > % ./build/linux-amd64/j2sdk-image/bin/jmod create -L /tmp/xxx > /tmp/xxx: Cannot open parent library > /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/j2sdk-image/jre/lib/modules Am I out of date with how things are supposed to work? Is jmod in j2sdk-image supposed to work? This is breaking one of the langtools regression tests. -- Jon From Mandy.Chung at Sun.COM Thu Mar 11 19:05:24 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Thu, 11 Mar 2010 19:05:24 -0800 Subject: jmod problem In-Reply-To: <4B999FF3.1020008@sun.com> References: <4B999FF3.1020008@sun.com> Message-ID: <4B99AF74.60704@sun.com> Jonathan Gibbons wrote: > I have a reasonably up to date copy of the jigsaw forest, and I have > built it successfully on Ubuntu. > > If I try and run jmod to create a simple library, it tells me it > cannot find the parent library, and indeed, the path it specifies does > not exist. > >> % ./build/linux-amd64/j2sdk-image/bin/jmod create -L /tmp/xxx >> /tmp/xxx: Cannot open parent library >> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/j2sdk-image/jre/lib/modules >> > > Am I out of date with how things are supposed to work? Is jmod in > j2sdk-image supposed to work? > I leave the legacy jdk image as it is today and didn't create a system module library. If it's for jtreg tests, you can do jmod create -N -L /tmp/xxx. jdk-module-image is the modular jdk that has the system module library created. Mandy > This is breaking one of the langtools regression tests. > > -- Jon From Jonathan.Gibbons at Sun.COM Thu Mar 11 19:27:54 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Thu, 11 Mar 2010 19:27:54 -0800 Subject: jmod problem In-Reply-To: <4B99AF74.60704@sun.com> References: <4B999FF3.1020008@sun.com> <4B99AF74.60704@sun.com> Message-ID: <4B99B4BA.6030109@sun.com> Mandy Chung wrote: > Jonathan Gibbons wrote: >> I have a reasonably up to date copy of the jigsaw forest, and I have >> built it successfully on Ubuntu. >> >> If I try and run jmod to create a simple library, it tells me it >> cannot find the parent library, and indeed, the path it specifies >> does not exist. >> >>> % ./build/linux-amd64/j2sdk-image/bin/jmod create -L /tmp/xxx >>> /tmp/xxx: Cannot open parent library >>> /mnt/w/jjg/work/jigsaw/jigsaw/build/linux-amd64/j2sdk-image/jre/lib/modules >>> >> >> Am I out of date with how things are supposed to work? Is jmod in >> j2sdk-image supposed to work? >> > > I leave the legacy jdk image as it is today and didn't create a system > module library. If it's for jtreg tests, you can do jmod create -N -L > /tmp/xxx. > > jdk-module-image is the modular jdk that has the system module library > created. > > Mandy >> This is breaking one of the langtools regression tests. >> >> -- Jon > Mandy, Thanks for the help. I guess I would suggest that either jmod should not be included in the legacy image, or it should have a more friendly error if there is no system module image available. There should also be a way for code to query the system to determine if this is a legacy image or a module image, so that tests (for example) can adapt their behavior accordingly. -- Jon From mr at sun.com Fri Mar 12 08:25:35 2010 From: mr at sun.com (Mark Reinhold) Date: Fri, 12 Mar 2010 08:25:35 -0800 Subject: platform requirements In-Reply-To: jonathan.gibbons@sun.com; Thu, 11 Mar 2010 16:11:52 PST; <4B9986C8.8000608@sun.com> Message-ID: <20100312162535.D25C53A9@eggemoggin.niobe.net> > Date: Thu, 11 Mar 2010 16:11:52 -0800 > From: jonathan.gibbons at sun.com > In a collection of resolved modules, are there any invariants or guarantees > about those modules which are identified as platform modules by > Platform.isPlatformModuleName ? > > For example: > -- will there be only one and only one such module in any collection of > resolved modules No, there could be many. > -- will such modules always have no other dependencies? No, they'll have dependences, but only upon other platform modules. - Mark From mr at sun.com Fri Mar 12 08:40:36 2010 From: mr at sun.com (Mark Reinhold) Date: Fri, 12 Mar 2010 08:40:36 -0800 Subject: jmod problem In-Reply-To: jonathan.gibbons@sun.com; Thu, 11 Mar 2010 19:27:54 PST; <4B99B4BA.6030109@sun.com> Message-ID: <20100312164036.4309D409@eggemoggin.niobe.net> > Date: Thu, 11 Mar 2010 19:27:54 -0800 > From: jonathan.gibbons at sun.com > Mandy Chung wrote: >> I leave the legacy jdk image as it is today and didn't create a system module >> library. If it's for jtreg tests, you can do jmod create -N -L >> /tmp/xxx. >> >> jdk-module-image is the modular jdk that has the system module library >> created. > > I guess I would suggest that either jmod should not be included in the legacy > image, or it should have a more friendly error if there is no system module > image available. There should also be a way for code to query the system to > determine if this is a legacy image or a module image, so that tests (for > example) can adapt their behavior accordingly. Are you trying to run tests using $OUTPUTDIR/bin/java, i.e., the development build, rather than one of the $OUTPUTDIR/*-image images? Prior to Mandy's most recent changes the development build included the lib/modules directory, with all modules pre-installed, and that's the default parent library, so $OUTPUTDIR/bin/jmod just worked. This is valuable for development, so I think we should reinstate it. The "legacy" j2{re,sdk} images will ultimately be replaced by module images. We could omit the Jigsaw tools from those images for now if that makes life simpler. - Mark From Jonathan.Gibbons at Sun.COM Fri Mar 12 09:00:09 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 12 Mar 2010 09:00:09 -0800 Subject: platform requirements In-Reply-To: <20100312162535.D25C53A9@eggemoggin.niobe.net> References: <20100312162535.D25C53A9@eggemoggin.niobe.net> Message-ID: <4B9A7319.5090900@sun.com> Mark Reinhold wrote: >> Date: Thu, 11 Mar 2010 16:11:52 -0800 >> From: jonathan.gibbons at sun.com >> > > >> In a collection of resolved modules, are there any invariants or guarantees >> about those modules which are identified as platform modules by >> Platform.isPlatformModuleName ? >> >> For example: >> -- will there be only one and only one such module in any collection of >> resolved modules >> > > No, there could be many. > > >> -- will such modules always have no other dependencies? >> > > No, they'll have dependences, but only upon other platform modules. > > - Mark > Thanks. I'll take that into account. -- Jon From Jonathan.Gibbons at Sun.COM Fri Mar 12 09:04:18 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 12 Mar 2010 09:04:18 -0800 Subject: jmod problem In-Reply-To: <20100312164036.4309D409@eggemoggin.niobe.net> References: <20100312164036.4309D409@eggemoggin.niobe.net> Message-ID: <4B9A7412.2000600@sun.com> Mark Reinhold wrote: >> Date: Thu, 11 Mar 2010 19:27:54 -0800 >> From: jonathan.gibbons at sun.com >> > > >> Mandy Chung wrote: >> >>> I leave the legacy jdk image as it is today and didn't create a system module >>> library. If it's for jtreg tests, you can do jmod create -N -L >>> /tmp/xxx. >>> >>> jdk-module-image is the modular jdk that has the system module library >>> created. >>> >> I guess I would suggest that either jmod should not be included in the legacy >> image, or it should have a more friendly error if there is no system module >> image available. There should also be a way for code to query the system to >> determine if this is a legacy image or a module image, so that tests (for >> example) can adapt their behavior accordingly. >> > > Are you trying to run tests using $OUTPUTDIR/bin/java, i.e., the > development build, rather than one of the $OUTPUTDIR/*-image images? > > Prior to Mandy's most recent changes the development build included the > lib/modules directory, with all modules pre-installed, and that's the > default parent library, so $OUTPUTDIR/bin/jmod just worked. This is > valuable for development, so I think we should reinstate it. > > The "legacy" j2{re,sdk} images will ultimately be replaced by module > images. We could omit the Jigsaw tools from those images for now if > that makes life simpler. > > - Mark > I was using j2sdk-image, which includes jmod, but not does have a module library, so the basic jmod command to create a new library fails with a messy internal message about not finding (the internal path for the module library). The comments were mostly about QoS and what should happen when people rightly or wrongly choose to use the legacy images. -- Jon From Dalibor.Topic at Sun.COM Fri Mar 12 10:30:21 2010 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Fri, 12 Mar 2010 19:30:21 +0100 Subject: jmod problem In-Reply-To: <4B9A7412.2000600@sun.com> References: <20100312164036.4309D409@eggemoggin.niobe.net> <4B9A7412.2000600@sun.com> Message-ID: <4B9A883D.1050907@sun.com> Jonathan Gibbons wrote: > I was using j2sdk-image, which includes jmod, but not does have a module > library, so the basic jmod command to create a new library fails with a > messy internal message about not finding (the internal path for the > module library). You need to make modules first to create the jdk image with installed modules, then running the tests with that jdk works fine for me. > The comments were mostly about QoS and what should happen when people > rightly or wrongly choose to use the legacy images. Yeah, I'll take a look at fixing the issue. cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der, Wolfgang Engels Vorsitzender des Aufsichtsrates: Martin H?ring From Mandy.Chung at Sun.COM Fri Mar 12 10:58:09 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Fri, 12 Mar 2010 10:58:09 -0800 Subject: jmod problem In-Reply-To: <20100312164036.4309D409@eggemoggin.niobe.net> References: <20100312164036.4309D409@eggemoggin.niobe.net> Message-ID: <4B9A8EC1.2000904@Sun.com> On 03/12/10 08:40, Mark Reinhold wrote: >> Date: Thu, 11 Mar 2010 19:27:54 -0800 >> From: jonathan.gibbons at sun.com > >> Mandy Chung wrote: >>> I leave the legacy jdk image as it is today and didn't create a system module >>> library. If it's for jtreg tests, you can do jmod create -N -L >>> /tmp/xxx. >>> >>> jdk-module-image is the modular jdk that has the system module library >>> created. >> I guess I would suggest that either jmod should not be included in the legacy >> image, or it should have a more friendly error if there is no system module >> image available. There should also be a way for code to query the system to >> determine if this is a legacy image or a module image, so that tests (for >> example) can adapt their behavior accordingly. > > Are you trying to run tests using $OUTPUTDIR/bin/java, i.e., the > development build, rather than one of the $OUTPUTDIR/*-image images? > > Prior to Mandy's most recent changes the development build included the > lib/modules directory, with all modules pre-installed, and that's the > default parent library, so $OUTPUTDIR/bin/jmod just worked. This is > valuable for development, so I think we should reinstate it. The reason why I changed it not to preinstall the jdk modules in $OUTPUTDIR is to fix the images build issue as all $OUTPUTDIR/lib/* are copied j2{re,sdk}-image. Instead the modules are preinstalled in $OUTPUTDIR/tmp/jigsaw-lib/modules to catch if any module resolution issue. I somehow neglected our use of $OUTPUTDIR for modules testing/development but thought that jdk-module-image is adequent for our development purpose. I will reinstate the preinstallation of jdk modules in $OUTPUTDIR. Mandy > The "legacy" j2{re,sdk} images will ultimately be replaced by module > images. We could omit the Jigsaw tools from those images for now if > that makes life simpler. > - Mark From Jonathan.Gibbons at Sun.COM Fri Mar 12 11:03:49 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 12 Mar 2010 11:03:49 -0800 Subject: jmod problem In-Reply-To: <4B9A8EC1.2000904@Sun.com> References: <20100312164036.4309D409@eggemoggin.niobe.net> <4B9A8EC1.2000904@Sun.com> Message-ID: <4B9A9015.1000803@sun.com> Mandy Chung wrote: > On 03/12/10 08:40, Mark Reinhold wrote: >>> Date: Thu, 11 Mar 2010 19:27:54 -0800 >>> From: jonathan.gibbons at sun.com >> >>> Mandy Chung wrote: >>>> I leave the legacy jdk image as it is today and didn't create a >>>> system module >>>> library. If it's for jtreg tests, you can do jmod create -N -L >>>> /tmp/xxx. >>>> >>>> jdk-module-image is the modular jdk that has the system module library >>>> created. >>> I guess I would suggest that either jmod should not be included in >>> the legacy >>> image, or it should have a more friendly error if there is no system >>> module >>> image available. There should also be a way for code to query the >>> system to >>> determine if this is a legacy image or a module image, so that tests >>> (for >>> example) can adapt their behavior accordingly. >> >> Are you trying to run tests using $OUTPUTDIR/bin/java, i.e., the >> development build, rather than one of the $OUTPUTDIR/*-image images? >> >> Prior to Mandy's most recent changes the development build included the >> lib/modules directory, with all modules pre-installed, and that's the >> default parent library, so $OUTPUTDIR/bin/jmod just worked. This is >> valuable for development, so I think we should reinstate it. > > The reason why I changed it not to preinstall the jdk modules in > $OUTPUTDIR > is to fix the images build issue as all $OUTPUTDIR/lib/* are copied > j2{re,sdk}-image. > Instead the modules are preinstalled in $OUTPUTDIR/tmp/jigsaw-lib/modules > to catch if any module resolution issue. > > I somehow neglected our use of $OUTPUTDIR for modules testing/development > but thought that jdk-module-image is adequent for our development > purpose. > > I will reinstate the preinstallation of jdk modules in $OUTPUTDIR. > > Mandy > >> The "legacy" j2{re,sdk} images will ultimately be replaced by module >> images. We could omit the Jigsaw tools from those images for now if >> that makes life simpler. >> - Mark Mandy, I'm not sure if there is some confusion here. Mark asked if I was using $OUTPUTDIR/bin/java to which I said I was using the j2sdk-image. I was not using $OUTPUTDIR directly, nor was I suggesting that we should install modules directly in $OUTPUTDIR. I was commenting on the behavior of jmod in j2sdk-image, with respect to missing module images. If the confusion is mine, I apologize. :-) -- Jon From Mandy.Chung at Sun.COM Fri Mar 12 11:47:29 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Fri, 12 Mar 2010 11:47:29 -0800 Subject: jmod problem In-Reply-To: <4B9A9015.1000803@sun.com> References: <20100312164036.4309D409@eggemoggin.niobe.net> <4B9A8EC1.2000904@Sun.com> <4B9A9015.1000803@sun.com> Message-ID: <4B9A9A51.9050603@Sun.com> On 03/12/10 11:03, Jonathan Gibbons wrote: > Mandy, > > I'm not sure if there is some confusion here. Mark asked if I was using > $OUTPUTDIR/bin/java to which I said I was using the j2sdk-image. I was > not using $OUTPUTDIR directly, nor was I suggesting that we should > install modules directly in $OUTPUTDIR. I was commenting on the > behavior of jmod in j2sdk-image, with respect to missing module images. > > If the confusion is mine, I apologize. :-) Sorry to confuse you. I should have made it clear. There are two different issues: 1. j2sdk-image doesn't have a system module library. To address your comment, we can either remove jmod from j2sdk-image or output a helpful error message. The simple programmatic way to determine if it's a legacy image or module image is by checking if $java.home/lib/rt.jar exists. 2. $OUTPUTDIR doesn't have the system module library, with the modules installed. For now, $OUTPUTDIR/bin/jmod create -L lib doesn't work. I agree that for our development use, it'll be valuable if $OUTPUTDIR/bin/jmod works and we can also run jtreg modules tests with $OUTPUTDIR as it is today. Until we have the zero mod library implemented (the idea you and Mark discussed) that can read modules from "modulepath" that no installation is required, we need to have the system module library in $OUTPUTDIR. Does this make sense? Mandy From Jonathan.Gibbons at Sun.COM Fri Mar 12 14:59:53 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 12 Mar 2010 14:59:53 -0800 Subject: control build for Jigsaw? Message-ID: <4B9AC769.1030403@sun.com> It seems I cannot do a full control build and make modules images, is that right? There is no mention of modules in the control build Makefiles, and the default target in jdk, "all", does not live up to its name. -- Jon From Mandy.Chung at Sun.COM Fri Mar 12 15:10:24 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Fri, 12 Mar 2010 15:10:24 -0800 Subject: control build for Jigsaw? In-Reply-To: <4B9AC769.1030403@sun.com> References: <4B9AC769.1030403@sun.com> Message-ID: <4B9AC9E0.1040400@sun.com> Jonathan Gibbons wrote: > It seems I cannot do a full control build and make modules images, is > that right? There is no mention of modules in the control build > Makefiles, and the default target in jdk, "all", does not live up to > its name. Is your jigsaw repo up-to-date? I added the modules target last week. woody:/export/mchung/jigsaw/ws/jigsaw-fixes > gnumake help Makefile for the JDK builds (all the JDK). --- Common Targets --- all -- build the core JDK (default target) modules -- build the JDK module images help -- Print out help information check -- Check make variable values for correctness sanity -- Perform detailed sanity checks on system and settings fastdebug_build -- build the core JDK in 'fastdebug' mode (-g -O) debug_build -- build the core JDK in 'debug' mode (-g) clean -- remove all built and imported files clobber -- same as clean Mandy From Jonathan.Gibbons at Sun.COM Fri Mar 12 15:17:50 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 12 Mar 2010 15:17:50 -0800 Subject: control build for Jigsaw? In-Reply-To: <4B9AC9E0.1040400@sun.com> References: <4B9AC769.1030403@sun.com> <4B9AC9E0.1040400@sun.com> Message-ID: <4B9ACB9E.3060508@sun.com> Mandy Chung wrote: > Jonathan Gibbons wrote: >> It seems I cannot do a full control build and make modules images, is >> that right? There is no mention of modules in the control build >> Makefiles, and the default target in jdk, "all", does not live up to >> its name. > Is your jigsaw repo up-to-date? I added the modules > target last week. > > woody:/export/mchung/jigsaw/ws/jigsaw-fixes > gnumake help > Makefile for the JDK builds (all the JDK). > > --- Common Targets --- all -- build the core JDK > (default target) > modules -- build the JDK module images > help -- Print out help information > check -- Check make variable values for correctness > sanity -- Perform detailed sanity checks on system and > settings > fastdebug_build -- build the core JDK in 'fastdebug' mode (-g -O) > debug_build -- build the core JDK in 'debug' mode (-g) > clean -- remove all built and imported files > clobber -- same as clean > Mandy > > Mea culpa. I thought I checked that; obviously not. Sorry. -- Jon From Jonathan.Gibbons at Sun.COM Fri Mar 12 16:01:57 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 12 Mar 2010 16:01:57 -0800 Subject: images Message-ID: <4B9AD5F5.6060104@sun.com> Mandy, Can you give a brief summary of all the (new) images made by "make modules"? -- Jon > $ ls -d1 build/*/*image > build/linux-amd64/j2re-image > build/linux-amd64/j2sdk-image > build/linux-amd64/jdk-base-image > build/linux-amd64/jdk-module-image > build/linux-amd64/jre-base-image > build/linux-amd64/jre-module-image From mr at sun.com Fri Mar 12 16:16:44 2010 From: mr at sun.com (Mark Reinhold) Date: Fri, 12 Mar 2010 16:16:44 -0800 Subject: jmod problem In-Reply-To: mandy.chung@sun.com; Fri, 12 Mar 2010 11:47:29 PST; <4B9A9A51.9050603@Sun.com> Message-ID: <20100313001644.C691145F@eggemoggin.niobe.net> > Date: Fri, 12 Mar 2010 11:47:29 -0800 > From: mandy.chung at sun.com > On 03/12/10 11:03, Jonathan Gibbons wrote: >> I'm not sure if there is some confusion here. Mark asked if I was using >> $OUTPUTDIR/bin/java to which I said I was using the j2sdk-image. I was not >> using $OUTPUTDIR directly, nor was I suggesting that we should install >> modules directly in $OUTPUTDIR. I was commenting on the behavior of jmod >> in j2sdk-image, with respect to missing module images. >> >> If the confusion is mine, I apologize. :-) > > Sorry to confuse you. I should have made it clear. > > There are two different issues: > 1. j2sdk-image doesn't have a system module library. > > To address your comment, we can either remove jmod from j2sdk-image or output a > helpful error message. It seems simpler just to remove the Jigsaw tools from the legacy images. > ... > > 2. $OUTPUTDIR doesn't have the system module library, with the modules > installed. > > For now, $OUTPUTDIR/bin/jmod create -L lib doesn't work. > > I agree that for our development use, it'll be valuable if $OUTPUTDIR/bin/jmod > works and we can also run jtreg modules tests with $OUTPUTDIR as it is today. > Until we have the zero mod library implemented (the idea you and Mark > discussed) that can read modules from "modulepath" that no installation is > required, we need to have the system module library in $OUTPUTDIR. To clarify, "zero mod" in this context means something like the "zero mod" facility now in javac, but implemented in the Java launcher so that you can say something like $ java -L $OUTPUTDIR/mods MyUnitTest rather than have to install a module into a library first just in order to test it. - Mark From dalibor.topic at sun.com Fri Mar 12 18:17:12 2010 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sat, 13 Mar 2010 02:17:12 +0000 Subject: hg: jigsaw/jigsaw/jdk: Store rather then deflate temporary jar entries Message-ID: <20100313021732.1181F44920@hg.openjdk.java.net> Changeset: 1522d8600a60 Author: robilad Date: 2010-03-13 02:33 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/1522d8600a60 Store rather then deflate temporary jar entries ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java From Mandy.Chung at Sun.COM Fri Mar 12 23:27:52 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Fri, 12 Mar 2010 23:27:52 -0800 Subject: images In-Reply-To: <4B9AD5F5.6060104@sun.com> References: <4B9AD5F5.6060104@sun.com> Message-ID: <4B9B3E78.7000203@sun.com> Jonathan Gibbons wrote: > Mandy, > > Can you give a brief summary of all the (new) images made by "make > modules"? > -- Jon > >> $ ls -d1 build/*/*image >> build/linux-amd64/j2re-image >> build/linux-amd64/j2sdk-image These two are legacy JRE and JDK images. They are not built by the "modules" target. > build/linux-amd64/jre-base-image jre-base-image is the minimal Java runtime only with jdk.boot and jdk.base modules installed > build/linux-amd64/jdk-base-image jdk-base-image is jre-base-image with the language tools including javac, javah, javap, and javadoc. This is convenience for running tests for the base image that needs javac to compile. >> build/linux-amd64/jdk-module-image Full JDK (equivalent to j2sdk-image) i.e. jre-module-image + all other jdk modules installed (e.g. all tools, debugging, hprof, etc). >> >> build/linux-amd64/jre-module-image > Full Java runtime environment with all JRE modules installed (equivalent to j2re-image) The new module images output by the "modules" target are also described in jdk/make/common/Modules.gmk. The modules target also generates jdk modules in: build/linux-amd64/jigsaw-pkgs/[jmod, deb] - all jdk modules are packaged in jmod file format under the jmod directory - if building on debian systems, debian packages are created under the deb directory. Alan has hacked up a quick tool to generate the javadoc for each of the modules. Before the javadoc becomes module-aware, it would be useful to post the per-module javadoc for reference. Mandy From Jonathan.Gibbons at Sun.COM Sat Mar 13 07:34:26 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Sat, 13 Mar 2010 07:34:26 -0800 Subject: images In-Reply-To: <4B9B3E78.7000203@sun.com> References: <4B9AD5F5.6060104@sun.com> <4B9B3E78.7000203@sun.com> Message-ID: <4B9BB082.1030900@sun.com> Mandy, Thanks. Given an image, how can I tell what is in it? Are there techniques that do not directly use jigsaw APIs? For example, if jtreg is running on top of JDK 1.6, and is asked to run tests using a JDK 1.7 image, can it look at the image to determine what classes are available? -- Jon Mandy Chung wrote: > Jonathan Gibbons wrote: >> Mandy, >> >> Can you give a brief summary of all the (new) images made by "make >> modules"? >> -- Jon >> >>> $ ls -d1 build/*/*image >>> build/linux-amd64/j2re-image >>> build/linux-amd64/j2sdk-image > > These two are legacy JRE and JDK images. They are not built by the > "modules" target. > >> build/linux-amd64/jre-base-image > jre-base-image is the minimal Java runtime only with jdk.boot and > jdk.base modules installed > >> build/linux-amd64/jdk-base-image > > jdk-base-image is jre-base-image with the language tools including > javac, javah, javap, and javadoc. This is convenience for running > tests for the base image that needs javac to compile. > >>> build/linux-amd64/jdk-module-image > > Full JDK (equivalent to j2sdk-image) > i.e. jre-module-image + all other jdk modules installed (e.g. all > tools, debugging, hprof, etc). > >>> >>> build/linux-amd64/jre-module-image >> > Full Java runtime environment with all JRE modules installed > (equivalent to j2re-image) > > The new module images output by the "modules" target are also > described in jdk/make/common/Modules.gmk. > > The modules target also generates jdk modules in: > build/linux-amd64/jigsaw-pkgs/[jmod, deb] > - all jdk modules are packaged in jmod file format under the jmod > directory > - if building on debian systems, debian packages are created under > the deb directory. > > Alan has hacked up a quick tool to generate the javadoc for each of > the modules. Before the javadoc becomes module-aware, it would be > useful to post the per-module javadoc for reference. > > Mandy From Mandy.Chung at Sun.COM Sat Mar 13 10:21:13 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Sat, 13 Mar 2010 10:21:13 -0800 Subject: images In-Reply-To: <4B9BB082.1030900@sun.com> References: <4B9AD5F5.6060104@sun.com> <4B9B3E78.7000203@sun.com> <4B9BB082.1030900@sun.com> Message-ID: <4B9BD799.6080804@sun.com> Jonathan Gibbons wrote: > Mandy, > > Thanks. Given an image, how can I tell what is in it? Are there > techniques that do not directly use jigsaw APIs? For example, if > jtreg is running on top of JDK 1.6, and is asked to run tests using a > JDK 1.7 image, can it look at the image to determine what classes are > available? There isn't necessarily a way. The module content layout is specific to the module library implementation. Like the simple library that puts all classes on disk in certain layout, the implementation might change. We could not count on the module content layout. Mark might have more to say about this. Mandy From Jonathan.Gibbons at Sun.COM Mon Mar 15 07:31:25 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 15 Mar 2010 07:31:25 -0700 Subject: images In-Reply-To: <4B9BD799.6080804@sun.com> References: <4B9AD5F5.6060104@sun.com> <4B9B3E78.7000203@sun.com> <4B9BB082.1030900@sun.com> <4B9BD799.6080804@sun.com> Message-ID: <4B9E44BD.2030704@sun.com> Mandy Chung wrote: > Jonathan Gibbons wrote: >> Mandy, >> >> Thanks. Given an image, how can I tell what is in it? Are there >> techniques that do not directly use jigsaw APIs? For example, if >> jtreg is running on top of JDK 1.6, and is asked to run tests using a >> JDK 1.7 image, can it look at the image to determine what classes are >> available? > > There isn't necessarily a way. The module content layout is specific > to the module library implementation. Like the simple library that > puts all classes on disk in certain layout, the implementation might > change. We could not count on the module content layout. > > Mark might have more to say about this. > > Mandy jmod might give me what I want -- I can check the file system to see if jmod exists, and if it does, I can then exec it query for specific modules. -- Jon From Mandy.Chung at Sun.COM Mon Mar 15 11:28:08 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Mon, 15 Mar 2010 11:28:08 -0700 Subject: images In-Reply-To: <4B9E44BD.2030704@sun.com> References: <4B9AD5F5.6060104@sun.com> <4B9B3E78.7000203@sun.com> <4B9BB082.1030900@sun.com> <4B9BD799.6080804@sun.com> <4B9E44BD.2030704@sun.com> Message-ID: <4B9E7C38.1040306@Sun.com> On 03/15/10 07:31, Jonathan Gibbons wrote: > Mandy Chung wrote: >> Jonathan Gibbons wrote: >>> Mandy, >>> >>> Thanks. Given an image, how can I tell what is in it? Are there >>> techniques that do not directly use jigsaw APIs? For example, if >>> jtreg is running on top of JDK 1.6, and is asked to run tests using a >>> JDK 1.7 image, can it look at the image to determine what classes are >>> available? >> >> There isn't necessarily a way. The module content layout is specific >> to the module library implementation. Like the simple library that >> puts all classes on disk in certain layout, the implementation might >> change. We could not count on the module content layout. >> >> Mark might have more to say about this. >> >> Mandy > > jmod might give me what I want -- I can check the file system to see if > jmod exists, and if it does, I can then exec it query for specific modules. That's right. I was too focus on the image layout and didn't consider tools. jmod would be the best tool to provide the information. Currently, jmod show -v will dump the list of classes but the output may be hard to parse. It works for a root module (i.e a module with the main entry point). What exactly do you need to find out from a given image? How is the info being used? Mandy From dalibor.topic at sun.com Mon Mar 15 12:17:20 2010 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Mon, 15 Mar 2010 19:17:20 +0000 Subject: hg: jigsaw/jigsaw/jdk: Use a RandomAccessFile for module writing Message-ID: <20100315191814.063C544CB2@hg.openjdk.java.net> Changeset: b478a0f30fac Author: robilad Date: 2010-03-15 12:10 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/b478a0f30fac Use a RandomAccessFile for module writing ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From Jonathan.Gibbons at Sun.COM Mon Mar 15 11:23:52 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 15 Mar 2010 11:23:52 -0700 Subject: images In-Reply-To: <4B9E7C38.1040306@Sun.com> References: <4B9AD5F5.6060104@sun.com> <4B9B3E78.7000203@sun.com> <4B9BB082.1030900@sun.com> <4B9BD799.6080804@sun.com> <4B9E44BD.2030704@sun.com> <4B9E7C38.1040306@Sun.com> Message-ID: <4B9E7B38.1050501@sun.com> Mandy Chung wrote: > On 03/15/10 07:31, Jonathan Gibbons wrote: >> Mandy Chung wrote: >>> Jonathan Gibbons wrote: >>>> Mandy, >>>> >>>> Thanks. Given an image, how can I tell what is in it? Are there >>>> techniques that do not directly use jigsaw APIs? For example, if >>>> jtreg is running on top of JDK 1.6, and is asked to run tests using >>>> a JDK 1.7 image, can it look at the image to determine what classes >>>> are available? >>> >>> There isn't necessarily a way. The module content layout is >>> specific to the module library implementation. Like the simple >>> library that puts all classes on disk in certain layout, the >>> implementation might change. We could not count on the module >>> content layout. >>> >>> Mark might have more to say about this. >>> >>> Mandy >> >> jmod might give me what I want -- I can check the file system to see >> if jmod exists, and if it does, I can then exec it query for specific >> modules. > > That's right. I was too focus on the image layout and didn't > consider tools. jmod would be the best tool to provide the information. > > Currently, jmod show -v will dump the list of classes but the output > may be hard to parse. It works for a root module (i.e a module with > the main entry point). > > What exactly do you need to find out from a given image? How is the > info being used? > > Mandy Mostly, I'm just trying to think ahead for jtreg. When presented with an image, it needs to determine whether it can support compilation. In the past, this was done by poking around for "jre/" and "tools.jar" and stuff like that. Those may not exist in future, so I was trying to figure out what jtreg might do. Note in particular that I would like jtreg to remain somewhat version neutral, for versions >= 1.5. i.e the same copy of jtreg should work for running tests on 1.5 or later. -- Jon From mr at sun.com Mon Mar 15 12:34:23 2010 From: mr at sun.com (Mark Reinhold) Date: Mon, 15 Mar 2010 12:34:23 -0700 Subject: images In-Reply-To: jonathan.gibbons@sun.com; Mon, 15 Mar 2010 11:23:52 PDT; <4B9E7B38.1050501@sun.com> Message-ID: <20100315193423.D60384CE@eggemoggin.niobe.net> > Date: Mon, 15 Mar 2010 11:23:52 -0700 > From: jonathan.gibbons at sun.com > Mandy Chung wrote: >> On 03/15/10 07:31, Jonathan Gibbons wrote: >>> jmod might give me what I want -- I can check the file system to see if jmod >>> exists, and if it does, I can then exec it query for specific modules. Yes, that's what I'd have suggested. You could also use reflection to load the org.openjdk.jigsaw.Library class, and if that's successful then open the system default library and query it. Either way, beware of interface shimmer over the coming months. >> What exactly do you need to find out from a given image? How is the info >> being used? > > Mostly, I'm just trying to think ahead for jtreg. When presented with an > image, it needs to determine whether it can support compilation. In the past, > this was done by poking around for "jre/" and "tools.jar" and stuff like that. > Those may not exist in future, so I was trying to figure out what jtreg might > do. Note in particular that I would like jtreg to remain somewhat version > neutral, for versions >= 1.5. i.e the same copy of jtreg should work for > running tests on 1.5 or later. Completely agree -- thanks for thinking ahead about this. - Mark From Jonathan.Gibbons at Sun.COM Mon Mar 15 12:54:56 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 15 Mar 2010 12:54:56 -0700 Subject: images In-Reply-To: <20100315193423.D60384CE@eggemoggin.niobe.net> References: <20100315193423.D60384CE@eggemoggin.niobe.net> Message-ID: <4B9E9090.3050507@sun.com> Mark Reinhold wrote: >> Date: Mon, 15 Mar 2010 11:23:52 -0700 >> From: jonathan.gibbons at sun.com >> > > >> Mandy Chung wrote: >> >>> On 03/15/10 07:31, Jonathan Gibbons wrote: >>> >>>> jmod might give me what I want -- I can check the file system to see if jmod >>>> exists, and if it does, I can then exec it query for specific modules. >>>> > > Yes, that's what I'd have suggested. > > You could also use reflection to load the org.openjdk.jigsaw.Library > class, and if that's successful then open the system default library > and query it. > > Either way, beware of interface shimmer over the coming months. > > >>> What exactly do you need to find out from a given image? How is the info >>> being used? >>> >> Mostly, I'm just trying to think ahead for jtreg. When presented with an >> image, it needs to determine whether it can support compilation. In the past, >> this was done by poking around for "jre/" and "tools.jar" and stuff like that. >> Those may not exist in future, so I was trying to figure out what jtreg might >> do. Note in particular that I would like jtreg to remain somewhat version >> neutral, for versions >= 1.5. i.e the same copy of jtreg should work for >> running tests on 1.5 or later. >> > > Completely agree -- thanks for thinking ahead about this. > > - Mark > Here's another similar-but-different issue: A significant amount of our development and testing infrastructure relies on using "java -Xbootclasspath/p:" which can be used by developers to override system classes with the classes they want to test and run. Has anyone considered how we will achieve the same effect in a jigsaw-based world? -- Jon From Dalibor.Topic at Sun.COM Mon Mar 15 15:06:36 2010 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Mon, 15 Mar 2010 23:06:36 +0100 Subject: hg: jigsaw/jigsaw/jdk: Use a RandomAccessFile for module writing In-Reply-To: <20100315191814.063C544CB2@hg.openjdk.java.net> References: <20100315191814.063C544CB2@hg.openjdk.java.net> Message-ID: <4B9EAF6C.2050900@sun.com> Dalibor.Topic at Sun.COM wrote: > Changeset: b478a0f30fac > Author: robilad > Date: 2010-03-15 12:10 -0700 > URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/b478a0f30fac > > Use a RandomAccessFile for module writing > > ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java > ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java > Hm, the patch triggers an odd issue with one of the modules upon jmod extract - investigating. cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der From Jonathan.Gibbons at Sun.COM Mon Mar 15 14:40:55 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 15 Mar 2010 14:40:55 -0700 Subject: modules.config Message-ID: <4B9EA967.3050905@sun.com> > > module gui.tools { > include com.sun.tools.example.debug.bdi.**, > com.sun.tools.example.debug.gui.**, > com.sun.tools.internal.xjc.**; > include com.sun.tools.javac.Launcher; > } I'm not sure javac.Launcher should be included here. I'm not even sure the class should continue to exist. It is a hack to facilitate starting javac from NetBeans for debugging. -- Jon From Jonathan.Gibbons at Sun.COM Mon Mar 15 14:49:58 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 15 Mar 2010 14:49:58 -0700 Subject: tools in modules.config Message-ID: <4B9EAB86.2060703@sun.com> I'm mildly surprised by the layout of the tools and related modules. I'm curious if the current layout is by necessity or choice. FWIW, apt and javadoc (and javah) depend on javac, but not the other way around. By analogy with javah and javap, I would have expected the apt classes to live in the apt module, and likewise the javadoc and doclets classes to live in the doclets module. Are there any dependencies you discovered that makes it necessary to have the classes as they are? -- Jon > > > module tools { > include com.sun.tools.apt.**; > include com.sun.tools.javac.**, > com.sun.source.**; > include com.sun.tools.doclets.**; > include com.sun.tools.javadoc.**, > com.sun.javadoc.**; > exclude com.sun.tools.javac.Launcher; > } > > module javac { > class com.sun.tools.javac.Main; > } > > module apt { > class com.sun.tools.apt.Main; > } > > module javadoc { > class com.sun.tools.javadoc.Main; > } > > module javah { > include com.sun.tools.javah.**; > class com.sun.tools.javah.Main; > } > > module javap { > include com.sun.tools.javap.**, > com.sun.tools.classfile.**; > class com.sun.tools.javap.Main; > } From Mandy.Chung at Sun.COM Mon Mar 15 15:59:26 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Mon, 15 Mar 2010 15:59:26 -0700 Subject: modules.config In-Reply-To: <4B9EA967.3050905@sun.com> References: <4B9EA967.3050905@sun.com> Message-ID: <4B9EBBCE.9030503@Sun.com> On 03/15/10 14:40, Jonathan Gibbons wrote: >> >> module gui.tools { >> include com.sun.tools.example.debug.bdi.**, >> com.sun.tools.example.debug.gui.**, >> com.sun.tools.internal.xjc.**; >> include com.sun.tools.javac.Launcher; >> } > > I'm not sure javac.Launcher should be included here. I'm not even sure > the class should continue to exist. It is a hack to facilitate starting > javac from NetBeans for debugging. Thanks for the info. Would it be better to move it to the deprecated.tools module for now until it is removed from the source? Mandy From Jonathan.Gibbons at Sun.COM Mon Mar 15 15:06:47 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 15 Mar 2010 15:06:47 -0700 Subject: modules.config In-Reply-To: <4B9EBBCE.9030503@Sun.com> References: <4B9EA967.3050905@sun.com> <4B9EBBCE.9030503@Sun.com> Message-ID: <4B9EAF77.7020309@sun.com> Mandy Chung wrote: > On 03/15/10 14:40, Jonathan Gibbons wrote: >>> >>> module gui.tools { >>> include com.sun.tools.example.debug.bdi.**, >>> com.sun.tools.example.debug.gui.**, >>> com.sun.tools.internal.xjc.**; >>> include com.sun.tools.javac.Launcher; >>> } >> >> I'm not sure javac.Launcher should be included here. I'm not even >> sure the class should continue to exist. It is a hack to facilitate >> starting javac from NetBeans for debugging. > > Thanks for the info. Would it be better to move it to the > deprecated.tools module for now until it is removed from the source? > > Mandy I guess. I'll also investigate whether the class can be removed. I see deprecated.tools contains old javac -- I believe rmic still uses old javac, so you might want to consider if you want rmic to depend on all this deprecated cruft. -- Jon From Jonathan.Gibbons at Sun.COM Mon Mar 15 16:10:55 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 15 Mar 2010 16:10:55 -0700 Subject: testing jigsaw builds Message-ID: <4B9EBE7F.6040409@sun.com> Is there a standard set of tests (other than "everything") for testing a jigsaw build? -- Jon From Mandy.Chung at Sun.COM Mon Mar 15 16:14:59 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Mon, 15 Mar 2010 16:14:59 -0700 Subject: modules.config In-Reply-To: <4B9EAF77.7020309@sun.com> References: <4B9EA967.3050905@sun.com> <4B9EBBCE.9030503@Sun.com> <4B9EAF77.7020309@sun.com> Message-ID: <4B9EBF73.7040107@Sun.com> On 03/15/10 15:06, Jonathan Gibbons wrote: > > I see deprecated.tools contains old javac -- I believe rmic still uses > old javac, so you might want to consider if you want rmic to depend on > all this deprecated cruft. We haven't done much in investigating the dependencies in the tools (including rmic) area. I did wonder if rmic would not depend on the oldjava one day. Since it still uses old javac, I'll move the old javac classes to rmic module. Mandy From Jonathan.Gibbons at Sun.COM Mon Mar 15 16:19:21 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 15 Mar 2010 16:19:21 -0700 Subject: modules.config In-Reply-To: <4B9EBF73.7040107@Sun.com> References: <4B9EA967.3050905@sun.com> <4B9EBBCE.9030503@Sun.com> <4B9EAF77.7020309@sun.com> <4B9EBF73.7040107@Sun.com> Message-ID: <4B9EC079.80906@sun.com> Mandy Chung wrote: > On 03/15/10 15:06, Jonathan Gibbons wrote: >> >> I see deprecated.tools contains old javac -- I believe rmic still >> uses old javac, so you might want to consider if you want rmic to >> depend on all this deprecated cruft. > > We haven't done much in investigating the dependencies in the tools > (including rmic) area. I did wonder if rmic would not depend on the > oldjava one day. Since it still uses old javac, I'll move the old > javac classes to rmic module. > > Mandy Ideally, rmic would be rewritten as an extension to standard javac, using the JSR 199 and JSR 269 API. However, I don't see anyone in line able and willing to do it :-( -- Jon From Mandy.Chung at Sun.COM Mon Mar 15 16:25:58 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Mon, 15 Mar 2010 16:25:58 -0700 Subject: testing jigsaw builds In-Reply-To: <4B9EBE7F.6040409@sun.com> References: <4B9EBE7F.6040409@sun.com> Message-ID: <4B9EC206.1000707@Sun.com> I have been running the following tests for testing jigsaw builds on jdk-module-image. ${TEST}/java/lang ${TEST}/java/nio ${TEST}/java/util ${TEST}/java/text ${TEST}/java/security ${TEST}/java/beans ${TEST}/java/math ${TEST}/org/openjdk/jigsaw This is not a standard set of tests for testing the jigsaw build but these tests test classes in multiple modules and it should provide good coverage. Mandy On 03/15/10 16:10, Jonathan Gibbons wrote: > Is there a standard set of tests (other than "everything") for testing a > jigsaw build? > > -- Jon From Jonathan.Gibbons at Sun.COM Mon Mar 15 16:29:50 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 15 Mar 2010 16:29:50 -0700 Subject: testing jigsaw builds In-Reply-To: <4B9EC206.1000707@Sun.com> References: <4B9EBE7F.6040409@sun.com> <4B9EC206.1000707@Sun.com> Message-ID: <4B9EC2EE.4000407@sun.com> Thanks. Dare I ask whether all these can be run in samevm mode? -- Jon Mandy Chung wrote: > I have been running the following tests for testing jigsaw builds on > jdk-module-image. > > ${TEST}/java/lang > ${TEST}/java/nio > ${TEST}/java/util > ${TEST}/java/text > ${TEST}/java/security > ${TEST}/java/beans > ${TEST}/java/math > ${TEST}/org/openjdk/jigsaw > > This is not a standard set of tests for testing the jigsaw build but > these tests test classes in multiple modules and it should provide > good coverage. > > Mandy > > On 03/15/10 16:10, Jonathan Gibbons wrote: >> Is there a standard set of tests (other than "everything") for >> testing a jigsaw build? >> >> -- Jon From Mandy.Chung at Sun.COM Mon Mar 15 16:43:43 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Mon, 15 Mar 2010 16:43:43 -0700 Subject: testing jigsaw builds In-Reply-To: <4B9EC2EE.4000407@sun.com> References: <4B9EBE7F.6040409@sun.com> <4B9EC206.1000707@Sun.com> <4B9EC2EE.4000407@sun.com> Message-ID: <4B9EC62F.3080508@Sun.com> On 03/15/10 16:29, Jonathan Gibbons wrote: > Thanks. Dare I ask whether all these can be run in samevm mode? No, unfortunately since tools.jar no longer exists in any module image. javax.tools.ToolProvider.getSystemJavaCompiler() looks for the javac classes from tools.jar. As we discussed some time ago, fixing it will be tricky since the code needs to be able to compiled and run on JDK 6. (i.e. bootstrap mode.) Mandy > -- Jon > > > Mandy Chung wrote: >> I have been running the following tests for testing jigsaw builds on >> jdk-module-image. >> >> ${TEST}/java/lang >> ${TEST}/java/nio >> ${TEST}/java/util >> ${TEST}/java/text >> ${TEST}/java/security >> ${TEST}/java/beans >> ${TEST}/java/math >> ${TEST}/org/openjdk/jigsaw >> >> This is not a standard set of tests for testing the jigsaw build but >> these tests test classes in multiple modules and it should provide >> good coverage. >> >> Mandy >> >> On 03/15/10 16:10, Jonathan Gibbons wrote: >>> Is there a standard set of tests (other than "everything") for >>> testing a jigsaw build? >>> >>> -- Jon > From Jonathan.Gibbons at Sun.COM Mon Mar 15 16:51:05 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 15 Mar 2010 16:51:05 -0700 Subject: testing jigsaw builds In-Reply-To: <4B9EC62F.3080508@Sun.com> References: <4B9EBE7F.6040409@sun.com> <4B9EC206.1000707@Sun.com> <4B9EC2EE.4000407@sun.com> <4B9EC62F.3080508@Sun.com> Message-ID: <4B9EC7E9.20307@sun.com> Hmm, it sounds like jtreg changes maybe more urgent than I thought. Also, changes to javax.tools.ToolProvider.getSystemJavaCompiler() -- Jon Mandy Chung wrote: > On 03/15/10 16:29, Jonathan Gibbons wrote: >> Thanks. Dare I ask whether all these can be run in samevm mode? > > No, unfortunately since tools.jar no longer exists in any module > image. javax.tools.ToolProvider.getSystemJavaCompiler() looks for > the javac classes from tools.jar. As we discussed some time ago, > fixing it will be tricky since the code needs to be able to compiled > and run on JDK 6. (i.e. bootstrap mode.) > > Mandy > >> -- Jon >> >> >> Mandy Chung wrote: >>> I have been running the following tests for testing jigsaw builds on >>> jdk-module-image. >>> >>> ${TEST}/java/lang >>> ${TEST}/java/nio >>> ${TEST}/java/util >>> ${TEST}/java/text >>> ${TEST}/java/security >>> ${TEST}/java/beans >>> ${TEST}/java/math >>> ${TEST}/org/openjdk/jigsaw >>> >>> This is not a standard set of tests for testing the jigsaw build but >>> these tests test classes in multiple modules and it should provide >>> good coverage. >>> >>> Mandy >>> >>> On 03/15/10 16:10, Jonathan Gibbons wrote: >>>> Is there a standard set of tests (other than "everything") for >>>> testing a jigsaw build? >>>> >>>> -- Jon >> From Mandy.Chung at Sun.COM Mon Mar 15 16:54:17 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Mon, 15 Mar 2010 16:54:17 -0700 Subject: tools in modules.config In-Reply-To: <4B9EAB86.2060703@sun.com> References: <4B9EAB86.2060703@sun.com> Message-ID: <4B9EC8A9.509@Sun.com> On 03/15/10 14:49, Jonathan Gibbons wrote: > I'm mildly surprised by the layout of the tools and related modules. I'm > curious if the current layout is by necessity or choice. There will be one module per tool since each tool has its own entry point. In my first attempt, I grouped the langtools classes like what you expected (e.g. apt classes in apt module). I recalled that there is some sort of circular dependencies that I took the quick-and-dirty solution to group the apt, javadoc, doclets and javac in the tools module and plan to revisit later. Since the decision whether the jdk modules would be coarse-grained modules or fine-grained is yet to be made, I didn't pursue further modularize the tools' classes (at one point in time - it was inclined to have coarse-grained modules). In any case, it would be good to clean up the dependencies. I will do some experiment, find out the dependencies and let you know. Mandy > FWIW, apt and javadoc (and javah) depend on javac, but not the other way > around. > > By analogy with javah and javap, I would have expected the apt classes > to live in the apt module, and likewise the javadoc and doclets classes > to live in the doclets module. > > Are there any dependencies you discovered that makes it necessary to > have the classes as they are? > > -- Jon > >> >> >> module tools { >> include com.sun.tools.apt.**; >> include com.sun.tools.javac.**, >> com.sun.source.**; >> include com.sun.tools.doclets.**; >> include com.sun.tools.javadoc.**, >> com.sun.javadoc.**; >> exclude com.sun.tools.javac.Launcher; >> } >> >> module javac { >> class com.sun.tools.javac.Main; >> } >> >> module apt { >> class com.sun.tools.apt.Main; >> } >> >> module javadoc { >> class com.sun.tools.javadoc.Main; >> } >> >> module javah { >> include com.sun.tools.javah.**; >> class com.sun.tools.javah.Main; >> } >> >> module javap { >> include com.sun.tools.javap.**, >> com.sun.tools.classfile.**; >> class com.sun.tools.javap.Main; >> } > From Jonathan.Gibbons at Sun.COM Mon Mar 15 16:55:14 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 15 Mar 2010 16:55:14 -0700 Subject: testing jigsaw builds In-Reply-To: <4B9EC7E9.20307@sun.com> References: <4B9EBE7F.6040409@sun.com> <4B9EC206.1000707@Sun.com> <4B9EC2EE.4000407@sun.com> <4B9EC62F.3080508@Sun.com> <4B9EC7E9.20307@sun.com> Message-ID: <4B9EC8E2.2080001@sun.com> getSystemJavaCompiler() will only look in tools.jar if the classes are not already available in the existing classloader. If you're running a jdk module image, does that not have the javac classes available? -- Jon Jonathan Gibbons wrote: > Hmm, it sounds like jtreg changes maybe more urgent than I thought. > Also, changes to javax.tools.ToolProvider.getSystemJavaCompiler() > > -- Jon > > > Mandy Chung wrote: >> On 03/15/10 16:29, Jonathan Gibbons wrote: >>> Thanks. Dare I ask whether all these can be run in samevm mode? >> >> No, unfortunately since tools.jar no longer exists in any module >> image. javax.tools.ToolProvider.getSystemJavaCompiler() looks for >> the javac classes from tools.jar. As we discussed some time ago, >> fixing it will be tricky since the code needs to be able to compiled >> and run on JDK 6. (i.e. bootstrap mode.) >> >> Mandy >> >>> -- Jon >>> >>> >>> Mandy Chung wrote: >>>> I have been running the following tests for testing jigsaw builds >>>> on jdk-module-image. >>>> >>>> ${TEST}/java/lang >>>> ${TEST}/java/nio >>>> ${TEST}/java/util >>>> ${TEST}/java/text >>>> ${TEST}/java/security >>>> ${TEST}/java/beans >>>> ${TEST}/java/math >>>> ${TEST}/org/openjdk/jigsaw >>>> >>>> This is not a standard set of tests for testing the jigsaw build >>>> but these tests test classes in multiple modules and it should >>>> provide good coverage. >>>> >>>> Mandy >>>> >>>> On 03/15/10 16:10, Jonathan Gibbons wrote: >>>>> Is there a standard set of tests (other than "everything") for >>>>> testing a jigsaw build? >>>>> >>>>> -- Jon >>> > > From Jonathan.Gibbons at Sun.COM Mon Mar 15 16:56:48 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 15 Mar 2010 16:56:48 -0700 Subject: tools in modules.config In-Reply-To: <4B9EC8A9.509@Sun.com> References: <4B9EAB86.2060703@sun.com> <4B9EC8A9.509@Sun.com> Message-ID: <4B9EC940.3000404@sun.com> I would certainly be interested to hear of anything that causes circular dependencies between javac, apt and javadoc. -- Jon Mandy Chung wrote: > On 03/15/10 14:49, Jonathan Gibbons wrote: >> I'm mildly surprised by the layout of the tools and related modules. >> I'm curious if the current layout is by necessity or choice. > > There will be one module per tool since each tool has its own entry > point. In my first attempt, I grouped the langtools classes like what > you expected (e.g. apt classes in apt module). I recalled that there > is some sort of circular dependencies that I took the quick-and-dirty > solution to group the apt, javadoc, doclets and javac in the tools > module and plan to revisit later. > > Since the decision whether the jdk modules would be coarse-grained > modules or fine-grained is yet to be made, I didn't pursue further > modularize the tools' classes (at one point in time - it was inclined > to have coarse-grained modules). > > In any case, it would be good to clean up the dependencies. I will do > some experiment, find out the dependencies and let you know. > > Mandy > >> FWIW, apt and javadoc (and javah) depend on javac, but not the other >> way around. >> >> By analogy with javah and javap, I would have expected the apt >> classes to live in the apt module, and likewise the javadoc and >> doclets classes to live in the doclets module. >> >> Are there any dependencies you discovered that makes it necessary to >> have the classes as they are? >> >> -- Jon >> >>> >>> >>> module tools { >>> include com.sun.tools.apt.**; >>> include com.sun.tools.javac.**, >>> com.sun.source.**; >>> include com.sun.tools.doclets.**; >>> include com.sun.tools.javadoc.**, >>> com.sun.javadoc.**; >>> exclude com.sun.tools.javac.Launcher; >>> } >>> >>> module javac { >>> class com.sun.tools.javac.Main; >>> } >>> >>> module apt { >>> class com.sun.tools.apt.Main; >>> } >>> >>> module javadoc { >>> class com.sun.tools.javadoc.Main; >>> } >>> >>> module javah { >>> include com.sun.tools.javah.**; >>> class com.sun.tools.javah.Main; >>> } >>> >>> module javap { >>> include com.sun.tools.javap.**, >>> com.sun.tools.classfile.**; >>> class com.sun.tools.javap.Main; >>> } >> From Kelly.Ohair at Sun.COM Mon Mar 15 20:32:35 2010 From: Kelly.Ohair at Sun.COM (Kelly O'Hair) Date: Mon, 15 Mar 2010 20:32:35 -0700 Subject: testing jigsaw builds In-Reply-To: <4B9EC2EE.4000407@sun.com> References: <4B9EBE7F.6040409@sun.com> <4B9EC206.1000707@Sun.com> <4B9EC2EE.4000407@sun.com> Message-ID: <684825E9-0E77-435B-9AEC-FC95B37E9FE1@sun.com> I think most of the java/* tests are pretty samevm safe, except for java/beans maybe... -kto On Mar 15, 2010, at 4:29 PM, Jonathan Gibbons wrote: > Thanks. Dare I ask whether all these can be run in samevm mode? > > -- Jon > > > Mandy Chung wrote: >> I have been running the following tests for testing jigsaw builds >> on jdk-module-image. >> >> ${TEST}/java/lang >> ${TEST}/java/nio >> ${TEST}/java/util >> ${TEST}/java/text >> ${TEST}/java/security >> ${TEST}/java/beans >> ${TEST}/java/math >> ${TEST}/org/openjdk/jigsaw >> >> This is not a standard set of tests for testing the jigsaw build >> but these tests test classes in multiple modules and it should >> provide good coverage. >> >> Mandy >> >> On 03/15/10 16:10, Jonathan Gibbons wrote: >>> Is there a standard set of tests (other than "everything") for >>> testing a jigsaw build? >>> >>> -- Jon > From Sean.Mullan at Sun.COM Tue Mar 16 06:31:27 2010 From: Sean.Mullan at Sun.COM (Sean Mullan) Date: Tue, 16 Mar 2010 09:31:27 -0400 Subject: Signed module-file format Message-ID: <4B9F882F.9030201@sun.com> Hi, Here's an initial proposal to extend the module-file format to support digital signatures. Comments welcome. Thanks, Sean Signed Module File Format ========================= This proposal discusses a format for a signed module. It is based on the current jigsaw module-file format: http://cr.openjdk.java.net/~mr/jigsaw/notes/module-file-format/ Layout ------ A new SIGNATURE section is defined to contain the signature. It is optional. If specified, it must be the first optional section (i.e. directly after the MODULE_INFO section). Grammar: ModuleFileHeader SectionHeader(MODULE_INFO) [module-info data] (SectionHeader(SIGNATURE)[signature data])? (SectionHeader(CLASSES) [class data])? (SectionHeader(RESOURCES) (SubSectionFileHeader [file data])+)? (SectionHeader(NATIVE_LIBS) (SubSectionFileHeader [file data])+)? (SectionHeader(NATIVE_CMDS) (SubSectionFileHeader [file data])+)? (SectionHeader(CONFIG) (SubSectionFileHeader+ [file data])+)? The ModuleFileHeader whole file hash must also exclude the SIGNATURE section, if specified: ModuleFileHeader { ... u2 hashType; // One of FileConstants.ModuleFile.HashType // (applies to all hashes in this file) u2 hashLength; // Length of following hash b* hash; // Hash of entire file (except this hash // and SIGNATURE section, if specified) } The SIGNATURE data is defined as follows: Signature { u2 signatureType; // One of FileConstants.ModuleFile.SignatureType u2 signatureLength; // Length of following signature b* signature; // Signature } The signatureType describes the format of the signature (ex: PKCS #7, XML Signature, PGP). All JDK 7 implementations MUST support signatures of type PKCS7. Section/Header Hashes --------------------- It should be possible to verify the hash of each section as they are read. However, one issue with the current module file format is that it is not possible to verify the integrity of the ModuleFileHeader or the SectionHeaders without downloading the whole file and checking the whole file hash. Two changes will address this: 1. Add an additional hash that covers just the ModuleFileHeader. 2. Change the section hashes to cover both the header (excluding the hash itself) and the content. PKCS7 Signature Type -------------------- The PKCS7 Signature Type is a PKCS #7 signature (the Signed-data content type). It is encoded according to the PKCS #7 format defined here: http://www.rsa.com/rsalabs/node.asp?id=2129 SignedData ::= SEQUENCE { version Version, digestAlgorithms DigestAlgorithmIdentifiers, contentInfo ContentInfo, certificates [0] IMPLICIT ExtendedCertificatesAndCertificates OPTIONAL, crls [1] IMPLICIT CertificateRevocationLists OPTIONAL, signerInfos SignerInfos } The digestAlgorithms field contains one element - the digest algorithm (as an AlgorithmIdentifier) specified in the module header. The contentInfo (what is being signed) is of type data. The content itself is omitted. This avoids replicating the data in the module file. The certificates field contains the certificates of the signer's certificate chain. The crls field is currently unused. All JDK 7 implementations MUST support the SHA256withRSA signature algorithm. What is signed -------------- The content that is signed is the following: Data ::= OCTET STRING where the octet string contains the following bytes: ToBeSignedContent { u2 moduleHeaderHashLength; b* moduleHeaderHash; u2 moduleInfoHashLength; b* moduleInfoHash; u2 sectionHashLength; b* sectionHash; ... // other section hashes (must be in same order as module file) ... u2 moduleFileHashLength; b* moduleFileHash; } The hashes (and their lengths) are the corresponding values in the module file. Tool Support ------------ jpkg will need to be enhanced to support generation of signed modules. New command line options will be needed to specify keystore filenames, aliases, passwords, etc. jmod will need to be enhanced to automatically verify signed modules. (A -noverify option can be specified to disable verification). It would be useful to have a public API for signing and verifying modules. Open Issues ----------- * Multiple signer support: it should be possible to add support for this without changing the file format, since the underlying PKCS7 signature format supports multiple signers. * It may be useful to have a mechanism to allow you to re-check the integrity of what is stored on the filesystem (the uncompressed data). Since the hashes may be over compressed data, the signature cannot be used to verify that same data, after it is uncompressed. * Should we require support for more than one cryptographic signature algorithm? This is probably a good idea and SHA256withECDSA would be a logical choice. * Support for timestamps: signed JARs already support timestamps, and this is useful to verify that a signature was created when the certificate was valid and not revoked. We should allow revocation information (CRLs or OCSP responses) to be added to the signed module when it is timestamped. This allows the verifier to verify that the certificate was not revoked when it was signed. These features should not require changes to the signed module-file format (since the underlying PKCS7 format can carry timestamps and CRLs). * Certificate duplication: certificates may be duplicated across modules created by the same signer. It should be possible to avoid this redundancy. Possible solutions include: 1) using the id-ad-caIssuers field of the X.509 AuthorityInformationAccess extension to get the intermediate CA certificates in a signer's chain, and 2) providing a mechanism to download the signer's certificate. From mr at sun.com Tue Mar 16 08:12:48 2010 From: mr at sun.com (Mark Reinhold) Date: Tue, 16 Mar 2010 08:12:48 -0700 Subject: testing jigsaw builds In-Reply-To: jonathan.gibbons@sun.com; Mon, 15 Mar 2010 16:55:14 PDT; <4B9EC8E2.2080001@sun.com> Message-ID: <20100316151248.A79E23A6@eggemoggin.niobe.net> > Date: Mon, 15 Mar 2010 16:55:14 -0700 > From: jonathan.gibbons at sun.com > getSystemJavaCompiler() will only look in tools.jar if the classes are not > already available in the existing classloader. If you're running a jdk module > image, does that not have the javac classes available? Yes, if the the module containing javax.tools.ToolProvider (i.e., sun.langtools) is declared to have an optional dependence upon the javac module, and the javac module is installed. - Mark From Mandy.Chung at Sun.COM Tue Mar 16 08:42:44 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Tue, 16 Mar 2010 08:42:44 -0700 Subject: testing jigsaw builds In-Reply-To: <20100316151248.A79E23A6@eggemoggin.niobe.net> References: <20100316151248.A79E23A6@eggemoggin.niobe.net> Message-ID: <4B9FA6F4.7000404@sun.com> Mark Reinhold wrote: >> Date: Mon, 15 Mar 2010 16:55:14 -0700 >> From: jonathan.gibbons at sun.com >> > > >> getSystemJavaCompiler() will only look in tools.jar if the classes are not >> already available in the existing classloader. If you're running a jdk module >> image, does that not have the javac classes available? >> > > Yes, if the the module containing javax.tools.ToolProvider (i.e., > sun.langtools) is declared to have an optional dependence upon the > javac module, and the javac module is installed. > > Jon, If you want to experiment this, you can edit the generated module-info.java file for now ($outputdir/tmp/modules/src). I'll update the classanalyzer to support the optional dependence declaration. Mandy From Dalibor.Topic at Sun.COM Tue Mar 16 12:10:05 2010 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Tue, 16 Mar 2010 20:10:05 +0100 Subject: hg: jigsaw/jigsaw/jdk: Use a RandomAccessFile for module writing In-Reply-To: <4B9EAF6C.2050900@sun.com> References: <20100315191814.063C544CB2@hg.openjdk.java.net> <4B9EAF6C.2050900@sun.com> Message-ID: <4B9FD78D.5030801@sun.com> Dalibor Topic wrote: > Dalibor.Topic at Sun.COM wrote: >> Changeset: b478a0f30fac >> Author: robilad >> Date: 2010-03-15 12:10 -0700 >> URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/b478a0f30fac >> >> Use a RandomAccessFile for module writing >> >> ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java >> ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java >> > > Hm, the patch triggers an odd issue with one of the modules upon > jmod extract - investigating. Turned out to be a kernel oops on the remote machine I was testing the patch on. Works fine after a powercycle again. ;) cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der From Mandy.Chung at Sun.COM Tue Mar 16 12:44:45 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Tue, 16 Mar 2010 12:44:45 -0700 Subject: Detect if bootclasspath/classpath is set in module mode Message-ID: <4B9FDFAD.6090400@Sun.com> Karen, Alan, We don't support bootclasspath and classpath in module mode. I added the check in hotspot and java launcher. Can you review this simple fix? Webrev at: http://cr.openjdk.java.net/~mchung/jigsaw/modular-mode-checks/ Thanks Mandy From Mandy.Chung at Sun.COM Tue Mar 16 13:38:12 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Tue, 16 Mar 2010 13:38:12 -0700 Subject: Build change to support cross architecture modules build Message-ID: <4B9FEC34.4020201@Sun.com> Hi Kelly, This is the build change we discussed to support building modules when the jdk is built for a target architecture different than that of the host system. Thanks for the suggestion of the make variable names. Summary: jmod needs to run during the modules build to install modules that depends on the jigsaw module system. jigsaw has native code and hence we can't use the bootclasspath trick for jmod as used by JAVAC_CMD. This fix defines a new JDK_HOST_PATH that set to OUTPUTDIR by default and several HOST_tool_CMD variables. ALT_JDK_HOST_PATH can be set if needed for cross compilation and it has to be the same version or newer as the built JDK. Webrev at: http://cr.openjdk.java.net/~mchung/jigsaw/alt-jdk-host-path/ In make/modules/Makefile, thanks for pointing out that at one point in time, GNU make would exec sh for every line, even if it was a shell comment line. So I moved the comments above the target. Thanks Mandy From Kelly.Ohair at Sun.COM Tue Mar 16 13:48:59 2010 From: Kelly.Ohair at Sun.COM (Kelly O'Hair) Date: Tue, 16 Mar 2010 13:48:59 -0700 Subject: Build change to support cross architecture modules build In-Reply-To: <4B9FEC34.4020201@Sun.com> References: <4B9FEC34.4020201@Sun.com> Message-ID: <7B0ABC9B-4BE8-4D0A-9E45-2E6B50A3BB8D@Sun.COM> This looks good to me. -kto On Mar 16, 2010, at 1:38 PM, Mandy Chung wrote: > Hi Kelly, > > This is the build change we discussed to support building modules > when the jdk is built for a target architecture different than that > of the host system. Thanks for the suggestion of the make variable > names. > > Summary: > jmod needs to run during the modules build to install modules that > depends on the jigsaw module system. jigsaw has native code and > hence we can't use the bootclasspath trick for jmod as used by > JAVAC_CMD. This fix defines a new JDK_HOST_PATH that set to > OUTPUTDIR by default and several HOST_tool_CMD variables. > ALT_JDK_HOST_PATH can be set if needed for cross compilation and it > has to be the same version or newer as the built JDK. > > Webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/alt-jdk-host-path/ > > In make/modules/Makefile, thanks for pointing out that at one point > in time, GNU make would exec sh for every line, even if it was a > shell comment line. So I moved the comments above the target. > > Thanks > Mandy From Alan.Bateman at Sun.COM Tue Mar 16 13:49:57 2010 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 16 Mar 2010 20:49:57 +0000 Subject: Build change to support cross architecture modules build In-Reply-To: <4B9FEC34.4020201@Sun.com> References: <4B9FEC34.4020201@Sun.com> Message-ID: <4B9FEEF5.3010609@sun.com> Mandy Chung wrote: > Hi Kelly, > > This is the build change we discussed to support building modules when > the jdk is built for a target architecture different than that of the > host system. Thanks for the suggestion of the make variable names. > > Summary: > jmod needs to run during the modules build to install modules that > depends on the jigsaw module system. jigsaw has native code and hence > we can't use the bootclasspath trick for jmod as used by JAVAC_CMD. > This fix defines a new JDK_HOST_PATH that set to OUTPUTDIR by default > and several HOST_tool_CMD variables. ALT_JDK_HOST_PATH can be set if > needed for cross compilation and it has to be the same version or > newer as the built JDK. > > Webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/alt-jdk-host-path/ > > In make/modules/Makefile, thanks for pointing out that at one point > in time, GNU make would exec sh for every line, even if it was a shell > comment line. So I moved the comments above the target. > > Thanks > Mandy I've gone through it and it looks good to me. -Alan. From Mandy.Chung at Sun.COM Tue Mar 16 13:53:36 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Tue, 16 Mar 2010 13:53:36 -0700 Subject: Build change to support cross architecture modules build In-Reply-To: <4B9FEEF5.3010609@sun.com> References: <4B9FEC34.4020201@Sun.com> <4B9FEEF5.3010609@sun.com> Message-ID: <4B9FEFD0.9040306@Sun.com> Thanks, Kelly and Alan, for the review. Mandy On 03/16/10 13:49, Alan Bateman wrote: > Mandy Chung wrote: >> Hi Kelly, >> >> This is the build change we discussed to support building modules when >> the jdk is built for a target architecture different than that of the >> host system. Thanks for the suggestion of the make variable names. >> >> Summary: >> jmod needs to run during the modules build to install modules that >> depends on the jigsaw module system. jigsaw has native code and hence >> we can't use the bootclasspath trick for jmod as used by JAVAC_CMD. >> This fix defines a new JDK_HOST_PATH that set to OUTPUTDIR by default >> and several HOST_tool_CMD variables. ALT_JDK_HOST_PATH can be set if >> needed for cross compilation and it has to be the same version or >> newer as the built JDK. >> >> Webrev at: >> http://cr.openjdk.java.net/~mchung/jigsaw/alt-jdk-host-path/ >> >> In make/modules/Makefile, thanks for pointing out that at one point >> in time, GNU make would exec sh for every line, even if it was a shell >> comment line. So I moved the comments above the target. >> >> Thanks >> Mandy > I've gone through it and it looks good to me. > > -Alan. From Alan.Bateman at Sun.COM Tue Mar 16 14:07:47 2010 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 16 Mar 2010 21:07:47 +0000 Subject: Detect if bootclasspath/classpath is set in module mode In-Reply-To: <4B9FDFAD.6090400@Sun.com> References: <4B9FDFAD.6090400@Sun.com> Message-ID: <4B9FF323.20505@sun.com> Mandy Chung wrote: > Karen, Alan, > > We don't support bootclasspath and classpath in module mode. I added > the check in hotspot and java launcher. Can you review this simple fix? > > Webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/modular-mode-checks/ > > Thanks > Mandy Looks okay to me. A minor comment is that "is_modules_mode" might be a bit better than "is_modular_mode". -Alan. From Karen.Kinnear at Sun.COM Tue Mar 16 14:18:55 2010 From: Karen.Kinnear at Sun.COM (Karen Kinnear) Date: Tue, 16 Mar 2010 16:18:55 -0500 Subject: Detect if bootclasspath/classpath is set in module mode In-Reply-To: <4B9FDFAD.6090400@Sun.com> References: <4B9FDFAD.6090400@Sun.com> Message-ID: <4B9FF5BF.4070604@sun.com> Mandy, Thanks for doing this so quickly. The code all looks good. So I see you checking for -cp and -classpath on the command line. I was under the impression you could also set CLASSPATH in your environment. Do we ignore that - that works fine for me I just wanted to know if that was intentional. thanks, Karen Mandy Chung wrote: > Karen, Alan, > > We don't support bootclasspath and classpath in module mode. I added > the check in hotspot and java launcher. Can you review this simple fix? > > Webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/modular-mode-checks/ > > Thanks > Mandy From Mandy.Chung at Sun.COM Tue Mar 16 15:05:03 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Tue, 16 Mar 2010 15:05:03 -0700 Subject: Detect if bootclasspath/classpath is set in module mode In-Reply-To: <4B9FF5BF.4070604@sun.com> References: <4B9FDFAD.6090400@Sun.com> <4B9FF5BF.4070604@sun.com> Message-ID: <4BA0008F.6070007@Sun.com> On 03/16/10 14:18, Karen Kinnear wrote: > Mandy, > > Thanks for doing this so quickly. > > The code all looks good. > So I see you checking for -cp and -classpath on the command line. > I was under the impression you could also set CLASSPATH in your > environment. Do we ignore that - that works fine for me I just wanted > to know if that was intentional. The check is in java.c line 306-311. I forgot to mention it in the email. Alan suggests to rename the is_modular_mode function name in hotspot: > Looks okay to me. A minor comment is that "is_modules_mode" might be a bit better than "is_modular_mode". Do you have any opinion? I'm thinking is_module_mode (singular) might be better. Mandy > thanks, > Karen > > Mandy Chung wrote: >> Karen, Alan, >> >> We don't support bootclasspath and classpath in module mode. I added >> the check in hotspot and java launcher. Can you review this simple fix? >> >> Webrev at: >> http://cr.openjdk.java.net/~mchung/jigsaw/modular-mode-checks/ >> >> Thanks >> Mandy > From reinier at zwitserloot.com Tue Mar 16 15:58:19 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Tue, 16 Mar 2010 23:58:19 +0100 Subject: hash type at beginning, hash itself at end. Message-ID: <560fb5ed1003161558i536e0ea8sf182ed8efbd3a233@mail.gmail.com> In Mr. Mullen's proposal for signatures, it looks like the hash (the one that's been in the proposal for a while now, for data integrity, not for verification of author) is still at the top of the file. This is convenient for readers, but not for writers. You don't know the hash yet at the beginning! The easy workaround is to rewrite the file after you're done, but this is only possible when the target is on the file system. You can't rewrite parts of the stream when the target is a db, a pipe, or the network. If you can't rewrite the stream, then the process is hopeless: - store the entire file in memory first, hash it, then write it. - store the entire file with a dummy hash in a temporary file, then stream that file, filling in the right hash. A true solution to this problem is trivial: Send the *type* of the hash algorithm at the top, and send the actual hash content at the bottom. This is easy for both readers and writers. --Reinier Zwitserloot From dalibor.topic at sun.com Tue Mar 16 18:22:21 2010 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Wed, 17 Mar 2010 01:22:21 +0000 Subject: hg: jigsaw/jigsaw/jdk: Check for left over bytes in module files Message-ID: <20100317012241.930E744E65@hg.openjdk.java.net> Changeset: 338848ea1686 Author: robilad Date: 2010-03-17 02:09 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/338848ea1686 Check for left over bytes in module files ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java From reinier at zwitserloot.com Tue Mar 16 22:24:15 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Wed, 17 Mar 2010 06:24:15 +0100 Subject: hash type at beginning, hash itself at end. In-Reply-To: <4BA027D5.10004@sun.com> References: <560fb5ed1003161558i536e0ea8sf182ed8efbd3a233@mail.gmail.com> <4BA027D5.10004@sun.com> Message-ID: <560fb5ed1003162224g6ad59c01n9f20014f083f2e5d@mail.gmail.com> Ah, well, with the added speediness on verifying authentications there's more of an argument to keep the hash where it is, at the top. Though, the time taken by parsing the stream is effectively nil; skipping a section is very simple by simply reading and discarding (size) bytes. Creating jmods on the fly doesn't look like its going to be a common event, but nevertheless *if* there was no other reason to put the hash at the top, accomodating it is better than not accomodating it. However, if hash-at-top has extra benefits, as it looks like it does, then, consider my suggestion dropped. --Reinier Zwitserloot On Wed, Mar 17, 2010 at 1:52 AM, Roger Riggs wrote: > Hi Reinier, > > The current placement allows authentication checks of the hashes to be > performed > without reading the entire package including parsing of all of the > sections. > Using the same hash for integrity and authenticity avoids duplication in > the package > and potential for inconsistency. > > As proposed, the header, signature, and module-info all can be verified > against > the signing information without reading the entire file to get the final > hash. > For larger modules early placement of this information allows verification > earlier and quicker identification of dependencies which can help decrease > processing and download times in cases where additional modules need to > be identified and fetched. > > It is likely that modules will be created on the systems where the > compilation > and build processes occur and random access files are available and the > need > to rewrite the hashes is not a problem. It is much more likely that the > device > creating the package has more resources than the device receiving the > package. > > Are you envisaging that creation of modules is frequently done on demand > and in contexts where streaming is critical? Can you give an example? > > Roger Riggs > > > > Reinier Zwitserloot wrote: > > In Mr. Mullen's proposal for signatures, it looks like the hash (the one > that's been in the proposal for a while now, for data integrity, not for > verification of author) is still at the top of the file. This is convenient > for readers, but not for writers. You don't know the hash yet at the > beginning! > > The easy workaround is to rewrite the file after you're done, but this is > only possible when the target is on the file system. You can't rewrite parts > of the stream when the target is a db, a pipe, or the network. > > If you can't rewrite the stream, then the process is hopeless: > > - store the entire file in memory first, hash it, then write it. > - store the entire file with a dummy hash in a temporary file, then stream > that file, filling in the right hash. > > > A true solution to this problem is trivial: Send the *type* of the hash > algorithm at the top, and send the actual hash content at the bottom. This > is easy for both readers and writers. > > --Reinier Zwitserloot > > > From mandy.chung at sun.com Wed Mar 17 00:00:36 2010 From: mandy.chung at sun.com (mandy.chung at sun.com) Date: Wed, 17 Mar 2010 07:00:36 +0000 Subject: hg: jigsaw/jigsaw/jdk: 2 new changesets Message-ID: <20100317070114.87F5D44EB7@hg.openjdk.java.net> Changeset: e4e534434688 Author: mchung Date: 2010-03-16 23:58 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/e4e534434688 Support cross architecture modules build Summary: Define a JDK_HOST_PATH variable for a JDK that can run on the host Reviewed-by: ohair, alanb ! make/common/BuildPackages.gmk ! make/common/Modules.gmk ! make/common/Sanity.gmk ! make/common/shared/Defs-java.gmk ! make/common/shared/Defs.gmk ! make/common/shared/Sanity-Settings.gmk ! make/common/shared/Sanity.gmk ! make/modules/Makefile ! make/modules/tools/Makefile Changeset: d1f78dfbc217 Author: mchung Date: 2010-03-17 00:00 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/d1f78dfbc217 Merge From Sean.Mullan at Sun.COM Wed Mar 17 09:49:22 2010 From: Sean.Mullan at Sun.COM (Sean Mullan) Date: Wed, 17 Mar 2010 12:49:22 -0400 Subject: Whole file hash in module header Message-ID: <4BA10812.6000303@sun.com> Hi Dalibor, I took a quick scan of the Writer code and it looks like the whole file hash doesn't cover the module file header itself, which I think it should. Is my assumption correct? Is that a known problem? Thanks, Sean From Sean.Mullan at Sun.COM Wed Mar 17 09:58:24 2010 From: Sean.Mullan at Sun.COM (Sean Mullan) Date: Wed, 17 Mar 2010 12:58:24 -0400 Subject: hash type at beginning, hash itself at end. In-Reply-To: <560fb5ed1003162224g6ad59c01n9f20014f083f2e5d@mail.gmail.com> References: <560fb5ed1003161558i536e0ea8sf182ed8efbd3a233@mail.gmail.com> <4BA027D5.10004@sun.com> <560fb5ed1003162224g6ad59c01n9f20014f083f2e5d@mail.gmail.com> Message-ID: <4BA10A30.6050408@sun.com> My proposal for signed modules did not consider streamed writing as a requirement since this is not listed as a goal in the module file format [1]. As Roger points out, since the signature is over the hashes, the hashes must be calculated before writing the signature. Thanks, Sean [1]: http://cr.openjdk.java.net/~mr/jigsaw/notes/module-file-format/ Reinier Zwitserloot wrote: > Ah, well, with the added speediness on verifying authentications there's > more of an argument to keep the hash where it is, at the top. Though, the > time taken by parsing the stream is effectively nil; skipping a section is > very simple by simply reading and discarding (size) bytes. > > Creating jmods on the fly doesn't look like its going to be a common event, > but nevertheless *if* there was no other reason to put the hash at the top, > accomodating it is better than not accomodating it. However, if hash-at-top > has extra benefits, as it looks like it does, then, consider my suggestion > dropped. > > --Reinier Zwitserloot > > > > On Wed, Mar 17, 2010 at 1:52 AM, Roger Riggs wrote: > >> Hi Reinier, >> >> The current placement allows authentication checks of the hashes to be >> performed >> without reading the entire package including parsing of all of the >> sections. >> Using the same hash for integrity and authenticity avoids duplication in >> the package >> and potential for inconsistency. >> >> As proposed, the header, signature, and module-info all can be verified >> against >> the signing information without reading the entire file to get the final >> hash. >> For larger modules early placement of this information allows verification >> earlier and quicker identification of dependencies which can help decrease >> processing and download times in cases where additional modules need to >> be identified and fetched. >> >> It is likely that modules will be created on the systems where the >> compilation >> and build processes occur and random access files are available and the >> need >> to rewrite the hashes is not a problem. It is much more likely that the >> device >> creating the package has more resources than the device receiving the >> package. >> >> Are you envisaging that creation of modules is frequently done on demand >> and in contexts where streaming is critical? Can you give an example? >> >> Roger Riggs >> >> >> >> Reinier Zwitserloot wrote: >> >> In Mr. Mullen's proposal for signatures, it looks like the hash (the one >> that's been in the proposal for a while now, for data integrity, not for >> verification of author) is still at the top of the file. This is convenient >> for readers, but not for writers. You don't know the hash yet at the >> beginning! >> >> The easy workaround is to rewrite the file after you're done, but this is >> only possible when the target is on the file system. You can't rewrite parts >> of the stream when the target is a db, a pipe, or the network. >> >> If you can't rewrite the stream, then the process is hopeless: >> >> - store the entire file in memory first, hash it, then write it. >> - store the entire file with a dummy hash in a temporary file, then stream >> that file, filling in the right hash. >> >> >> A true solution to this problem is trivial: Send the *type* of the hash >> algorithm at the top, and send the actual hash content at the bottom. This >> is easy for both readers and writers. >> >> --Reinier Zwitserloot >> >> >> From Dalibor.Topic at Sun.COM Wed Mar 17 11:02:06 2010 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Wed, 17 Mar 2010 19:02:06 +0100 Subject: making jmod run method public Message-ID: <4BA1191E.3060305@sun.com> Hi Mark, I'd like to make the run method of jmod public, and move the System.exit calls up into the main method. This would make writing jtreg tests for exceptional conditions raised by jmod easier. OK? cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der From mr at sun.com Wed Mar 17 11:06:46 2010 From: mr at sun.com (Mark Reinhold) Date: Wed, 17 Mar 2010 11:06:46 -0700 Subject: making jmod run method public In-Reply-To: dalibor.topic@sun.com; Wed, 17 Mar 2010 19:02:06 BST; <4BA1191E.3060305@sun.com> Message-ID: <20100317180646.CA01F384@eggemoggin.niobe.net> > Date: Wed, 17 Mar 2010 19:02:06 +0100 > From: dalibor.topic at sun.com > I'd like to make the run method of jmod public, and move the System.exit > calls up into the main method. This would make writing jtreg tests for > exceptional conditions raised by jmod easier. OK? Sure. - Mark From dalibor.topic at sun.com Wed Mar 17 11:19:43 2010 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Wed, 17 Mar 2010 18:19:43 +0000 Subject: hg: jigsaw/jigsaw/jdk: Truncate a module file being written to if it exists Message-ID: <20100317182003.85CD944F91@hg.openjdk.java.net> Changeset: 47e9e7172d3d Author: robilad Date: 2010-03-17 11:13 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/47e9e7172d3d Truncate a module file being written to if it exists ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java From Dalibor.Topic at Sun.COM Wed Mar 17 13:34:57 2010 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Wed, 17 Mar 2010 21:34:57 +0100 Subject: Whole file hash in module header In-Reply-To: <4BA10812.6000303@sun.com> References: <4BA10812.6000303@sun.com> Message-ID: <4BA13CF1.1030409@sun.com> Sean Mullan wrote: > Hi Dalibor, > > I took a quick scan of the Writer code and it looks like the whole file > hash > doesn't cover the module file header itself, which I think it should. Is my > assumption correct? Is that a known problem? Correct - it covers everything after the header, and follows what every section hash does (only hash the payload, don't hash the metadata). Upon a second reading of the spec, I think you're right, though. So I'll update the code to hash the ModuleFileHeader metadata without the final byte array. cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?er From Mandy.Chung at Sun.COM Wed Mar 17 15:55:51 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Wed, 17 Mar 2010 15:55:51 -0700 Subject: Modules build minor cleanup Message-ID: <4BA15DF7.4090002@Sun.com> Hi Dalibor, Can you take a look at the fix in the modules build? 1. It preinstalls the jdk modules in $outputdir 2. Not to include .map and .pdb files in the modules. We should package the .map and .pdb files in some debug modules so that we can install them for our debugging purpose. Webrev at: http://cr.openjdk.java.net/~mchung/jigsaw/modules-build-cleanup/ Thanks Mandy From Mandy.Chung at Sun.COM Wed Mar 17 16:00:26 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Wed, 17 Mar 2010 16:00:26 -0700 Subject: Modules build minor cleanup In-Reply-To: <4BA15DF7.4090002@Sun.com> References: <4BA15DF7.4090002@Sun.com> Message-ID: <4BA15F0A.1040902@Sun.com> On 03/17/10 15:55, Mandy Chung wrote: > Hi Dalibor, > > Can you take a look at the fix in the modules build? > > 1. It preinstalls the jdk modules in $outputdir > 2. Not to include .map and .pdb files in the modules. We should package > the .map and .pdb files in some debug modules so that we can install > them for our debugging purpose. Forgot to mention this: 3. Not to include jmod and jpkg in the legacy image. jmod and jpkg are only included in the module images. Mandy > Webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/modules-build-cleanup/ > > Thanks > Mandy > From Dalibor.Topic at Sun.COM Wed Mar 17 16:17:39 2010 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Thu, 18 Mar 2010 00:17:39 +0100 Subject: Modules build minor cleanup In-Reply-To: <4BA15F0A.1040902@Sun.com> References: <4BA15DF7.4090002@Sun.com> <4BA15F0A.1040902@Sun.com> Message-ID: <4BA16313.6060604@sun.com> Mandy Chung wrote: > On 03/17/10 15:55, Mandy Chung wrote: >> Hi Dalibor, >> >> Can you take a look at the fix in the modules build? >> >> 1. It preinstalls the jdk modules in $outputdir I have one question here: in modularize: 154 for d in bin lib etc include ; do \ 155 if [ -d $(TMP)/$$s/$$d ] ; then \ 156 $(CP) -rf $(TMP)/$$s/$$d $$mlib; \ 157 fi ; \ 158 done \ 159 fi ; \ the loop doesn't seem to copy resource files. Is that intentional? >> 2. Not to include .map and .pdb files in the modules. We should >> package the .map and .pdb files in some debug modules so that we can >> install them for our debugging purpose. Looks good. > Forgot to mention this: > > 3. Not to include jmod and jpkg in the legacy image. Looks good. I assume they are still available in the jdk-module-image? cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?er From Mandy.Chung at Sun.COM Wed Mar 17 16:44:40 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Wed, 17 Mar 2010 16:44:40 -0700 Subject: Modules build minor cleanup In-Reply-To: <4BA16313.6060604@sun.com> References: <4BA15DF7.4090002@Sun.com> <4BA15F0A.1040902@Sun.com> <4BA16313.6060604@sun.com> Message-ID: <4BA16968.6000606@sun.com> Dalibor Topic wrote: > Mandy Chung wrote: >> On 03/17/10 15:55, Mandy Chung wrote: >>> Hi Dalibor, >>> >>> Can you take a look at the fix in the modules build? >>> >>> 1. It preinstalls the jdk modules in $outputdir > > I have one question here: > > in modularize: > > 154 for d in bin lib etc include ; do \ > 155 if [ -d $(TMP)/$$s/$$d ] ; then \ > 156 $(CP) -rf $(TMP)/$$s/$$d $$mlib; \ > 157 fi ; \ > 158 done \ > 159 fi ; \ > > the loop doesn't seem to copy resource files. Is that intentional? > classes and resources are handled in line 163-170. This loop is to copy non-class and non-resource files from $outputdir/tmp/modules to the top-level module. One or more modules defined in modules.config could be mapped in one top-level module. >>> 2. Not to include .map and .pdb files in the modules. We should >>> package the .map and .pdb files in some debug modules so that we can >>> install them for our debugging purpose. > > Looks good. > >> Forgot to mention this: >> >> 3. Not to include jmod and jpkg in the legacy image. > > Looks good. I assume they are still available in the jdk-module-image? > Right. jmod is in {jdk,jre}-base-module and {jdk,jre}-module-image and jpkg is in jdk-{base,module}-image. Mandy > cheers, > dalibor topic From Dalibor.Topic at Sun.COM Wed Mar 17 16:57:24 2010 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Thu, 18 Mar 2010 00:57:24 +0100 Subject: Modules build minor cleanup In-Reply-To: <4BA16968.6000606@sun.com> References: <4BA15DF7.4090002@Sun.com> <4BA15F0A.1040902@Sun.com> <4BA16313.6060604@sun.com> <4BA16968.6000606@sun.com> Message-ID: <4BA16C64.2020704@sun.com> Mandy Chung wrote: > classes and resources are handled in line 163-170. This loop is to copy > non-class and non-resource files from $outputdir/tmp/modules to the > top-level module. One or more modules defined in modules.config could > be mapped in one top-level module. OK, that makes sense, thanks Mandy. The patch looks fine to me. cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?er From dalibor.topic at sun.com Wed Mar 17 17:19:30 2010 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Thu, 18 Mar 2010 00:19:30 +0000 Subject: hg: jigsaw/jigsaw/jdk: Made run methods of jpkg and jmod public for easier testing Message-ID: <20100318002015.29B4244FE7@hg.openjdk.java.net> Changeset: fbecea164206 Author: robilad Date: 2010-03-18 00:53 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/fbecea164206 Made run methods of jpkg and jmod public for easier testing ! src/share/classes/org/openjdk/jigsaw/cli/Librarian.java ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From mandy.chung at sun.com Wed Mar 17 17:37:04 2010 From: mandy.chung at sun.com (mandy.chung at sun.com) Date: Thu, 18 Mar 2010 00:37:04 +0000 Subject: hg: jigsaw/jigsaw/jdk: Remove jmod, jpkg from legacy image and preinstall modules in outputdir Message-ID: <20100318003723.4E9AC44FED@hg.openjdk.java.net> Changeset: 391d0e9e9b77 Author: mchung Date: 2010-03-17 17:35 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/391d0e9e9b77 Remove jmod,jpkg from legacy image and preinstall modules in outputdir Reviewed-by: robilad ! make/common/Release.gmk ! make/modules/Makefile From dalibor.topic at sun.com Thu Mar 18 09:26:08 2010 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Thu, 18 Mar 2010 16:26:08 +0000 Subject: hg: jigsaw/jigsaw/jdk: Removed unnecessary seek Message-ID: <20100318162707.2F327440FB@hg.openjdk.java.net> Changeset: 5d5f800e3a0e Author: robilad Date: 2010-03-18 16:53 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/5d5f800e3a0e Removed unnecessary seek ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java From dalibor.topic at sun.com Thu Mar 18 12:46:16 2010 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Thu, 18 Mar 2010 19:46:16 +0000 Subject: hg: jigsaw/jigsaw/jdk: Added regression test for leftover bytes in module file Message-ID: <20100318194704.0E4D14412C@hg.openjdk.java.net> Changeset: 0e0ab125161f Author: robilad Date: 2010-03-18 20:29 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/0e0ab125161f Added regression test for leftover bytes in module file ! src/share/classes/org/openjdk/jigsaw/cli/Librarian.java + test/org/openjdk/jigsaw/cli/ModuleFormatTestLeftOverBytes.java From dalibor.topic at sun.com Fri Mar 19 09:46:33 2010 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Fri, 19 Mar 2010 16:46:33 +0000 Subject: hg: jigsaw/jigsaw/jdk: File hash applies to ModuleFileHeader without the hash bytes Message-ID: <20100319164727.0842A44289@hg.openjdk.java.net> Changeset: 99ad5eff6fb1 Author: robilad Date: 2010-03-19 17:44 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/99ad5eff6fb1 File hash applies to ModuleFileHeader without the hash bytes ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java + test/org/openjdk/jigsaw/cli/ModuleFormatHeaderHashTest.java From Dalibor.Topic at Sun.COM Fri Mar 19 09:53:31 2010 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Fri, 19 Mar 2010 17:53:31 +0100 Subject: Whole file hash in module header In-Reply-To: <4BA13CF1.1030409@sun.com> References: <4BA10812.6000303@sun.com> <4BA13CF1.1030409@sun.com> Message-ID: <4BA3AC0B.8010003@sun.com> Dalibor Topic wrote: > Sean Mullan wrote: >> Hi Dalibor, >> >> I took a quick scan of the Writer code and it looks like the whole >> file hash >> doesn't cover the module file header itself, which I think it should. >> Is my >> assumption correct? Is that a known problem? > > Correct - it covers everything after the header, and follows what every > section hash does (only hash the payload, don't hash the metadata). > > Upon a second reading of the spec, I think you're right, though. So I'll > update the code to hash the ModuleFileHeader metadata without the final > byte array. Fixed with changeset http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/99ad5eff6fb1 cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der, Wolfgang Engels Vorsitzender des Aufsichtsrates: Martin H?ring From Mandy.Chung at Sun.COM Fri Mar 19 10:35:15 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Fri, 19 Mar 2010 10:35:15 -0700 Subject: Detect if bootclasspath/classpath is set in module mode In-Reply-To: <4B9FDFAD.6090400@Sun.com> References: <4B9FDFAD.6090400@Sun.com> Message-ID: <4BA3B5D3.2010107@sun.com> Alan, I updated the webrev that renames is_modular_mode to is_module_mode in hotspot and also fixes an embarrassing bug of mine in src/share/bin/java.c line 793 that writes '\0' in a bad location. Webrev at: http://cr.openjdk.java.net/~mchung/jigsaw/modular-mode-checks/jdk.01/ http://cr.openjdk.java.net/~mchung/jigsaw/modular-mode-checks/hotspot.01/ Thanks Mandy Mandy Chung wrote: > Karen, Alan, > > We don't support bootclasspath and classpath in module mode. I added > the check in hotspot and java launcher. Can you review this simple fix? > > Webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/modular-mode-checks/ > > Thanks > Mandy From Karen.Kinnear at Sun.COM Fri Mar 19 10:43:50 2010 From: Karen.Kinnear at Sun.COM (Karen Kinnear) Date: Fri, 19 Mar 2010 13:43:50 -0400 Subject: Detect if bootclasspath/classpath is set in module mode In-Reply-To: <4BA3B5D3.2010107@sun.com> References: <4B9FDFAD.6090400@Sun.com> <4BA3B5D3.2010107@sun.com> Message-ID: <59EEEF3D-AC5F-4E1D-B6D7-6D0627121C4B@Sun.COM> Mandy, They both look good. Glad you caught the early null termination and thanks for adding comments about temporary paths :-) thanks, Karen On Mar 19, 2010, at 1:35 PM, Mandy Chung wrote: > Alan, > > I updated the webrev that renames is_modular_mode to is_module_mode > in hotspot and also fixes an embarrassing bug of mine in src/share/ > bin/java.c line 793 that writes '\0' in a bad location. > > Webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/modular-mode-checks/jdk.01/ > http://cr.openjdk.java.net/~mchung/jigsaw/modular-mode-checks/hotspot.01/ > > Thanks > Mandy > > Mandy Chung wrote: >> Karen, Alan, >> >> We don't support bootclasspath and classpath in module mode. I >> added the check in hotspot and java launcher. Can you review this >> simple fix? >> >> Webrev at: >> http://cr.openjdk.java.net/~mchung/jigsaw/modular-mode-checks/ >> >> Thanks >> Mandy > From dalibor.topic at sun.com Fri Mar 19 16:10:15 2010 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Fri, 19 Mar 2010 23:10:15 +0000 Subject: hg: jigsaw/jigsaw/jdk: Use random access file directly when writing sections Message-ID: <20100319231111.4453D442F3@hg.openjdk.java.net> Changeset: 9629cb2652c9 Author: robilad Date: 2010-03-20 00:08 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/9629cb2652c9 Use random access file directly when writing sections ! src/share/classes/org/openjdk/jigsaw/ModuleFileFormat.java From mandy.chung at sun.com Sat Mar 20 16:58:37 2010 From: mandy.chung at sun.com (mandy.chung at sun.com) Date: Sat, 20 Mar 2010 23:58:37 +0000 Subject: hg: jigsaw/jigsaw/hotspot: Output error if bootclasspath or classpath is set in module mode Message-ID: <20100320235851.70E8544456@hg.openjdk.java.net> Changeset: 7541747f1818 Author: mchung Date: 2010-03-20 16:48 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/7541747f1818 Output error if bootclasspath or classpath is set in module mode Reviewed-by: alanb, acorn ! src/share/vm/runtime/arguments.cpp ! src/share/vm/runtime/arguments.hpp From mandy.chung at sun.com Sat Mar 20 16:58:38 2010 From: mandy.chung at sun.com (mandy.chung at sun.com) Date: Sat, 20 Mar 2010 23:58:38 +0000 Subject: hg: jigsaw/jigsaw/jdk: Output error if bootclasspath or classpath is set in module mode Message-ID: <20100320235921.90AC444457@hg.openjdk.java.net> Changeset: 0331906295d6 Author: mchung Date: 2010-03-20 16:47 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/0331906295d6 Output error if bootclasspath or classpath is set in module mode Reviewed-by: alanb, acorn ! src/share/bin/emessages.h ! src/share/bin/java.c ! src/share/classes/sun/launcher/resources/launcher.properties From Sean.Mullan at Sun.COM Mon Mar 22 12:04:46 2010 From: Sean.Mullan at Sun.COM (Sean Mullan) Date: Mon, 22 Mar 2010 15:04:46 -0400 Subject: Minor revision of proposed signed module-file format Message-ID: <4BA7BF4E.3070304@sun.com> Hi all, I have posted a minor revision of the proposed signed module-file format here: http://cr.openjdk.java.net/~mullan/signed-module-file-format A couple of small changes: 1) Previously in the PKCS7 Signature Type section, the contentInfo (what is being signed) was omitted in order to avoid duplicate hashes in both the module file and the signature. However, omitting this information requires the entire file to be processed before the ToBeSignedContent could be reconstructed and the signature verified. This doesn't work for use cases where you only want to read part of the module-file. So this has been changed to require the ToBeSignedContent to be included in the signature. 2) I added an open issue as to whether the module-file hashes are really necessary when not signing: * Are module hashes necessary when not signing? Without a signature, the hashes provide little, if any security benefit since the module data can be modified and the hash can be replaced without detection. The hashes could be removed from the module file (since they are already contained in the ToBeSignedContent). A field could be added to the header to indicate if the module was signed or not, the main purpose of which would be to enable the reader to start digesting the contents before it reads the signature. Thanks, Sean From Sean.Mullan at Sun.COM Wed Mar 24 12:25:29 2010 From: Sean.Mullan at Sun.COM (Sean Mullan) Date: Wed, 24 Mar 2010 15:25:29 -0400 Subject: Signed Module and Security Functional Specification (DRAFT) Message-ID: <4BAA6729.6080208@sun.com> Hi, Please see http://cr.openjdk.java.net/~mullan/signed-module-functional-spec for further details on generating and validating signed modules and an initial proposal for permission requests and other enhancements that allow policy to grant signed modules only the permissions it needs, up to a maximum. Some details are TBD. Comments are welcome and appreciated. Thanks, Sean From Jonathan.Gibbons at Sun.COM Fri Mar 26 10:47:55 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 26 Mar 2010 10:47:55 -0700 Subject: build nit Message-ID: <4BACF34B.8090901@sun.com> Mandy, For jigsaw, "make modules" creates an empty "j2sdk-image" directory. -- Jon From Jonathan.Gibbons at Sun.COM Fri Mar 26 13:33:05 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 26 Mar 2010 13:33:05 -0700 Subject: -Xbootclasspath/p: (or similar) please Message-ID: <4BAD1A01.8090502@sun.com> What are the limitations of the bootclasspath options in the jigsaw module images? "java -X" says they cannot be used when executing a module. Does that also apply when the module is implicit, as in tools invoked by the launcher, like javac, javadoc, etc. If so, can I put it a BIG request that -Xbootclasspath/p: or an equivalent should be supported. I think it is CRITICALLY important for JDK developers to have a way of substituting new classes for existing ones when working on bug fixes and similar work. In the jdk/ repo I know that folk tend to do a partial build and execute out of the "build/classes/" directory. This will not work in the modules world. In the langtools/ repo, we routinely compile just the langtools classes and execute them with a existing JDK using -Xbootclasspath/p. The limitation on -Xbootclasspath means that this too will not work in the modules world. Development will be severely impacted if we have to build new modules images every time we want to try out a fix. From a testing point of view, I know that most of the langtools tests exercise functionality that is independent of whether JDK is running in legacy mode or modules mode. But, currently 31 tests fail in modules mode. These tests do not fail in legacy mode. This means the fixes for these tests will need to be tested on a full modules build. Starting from scratch, a full modules build takes me about an hour, whereas building langtools and using -Xbootclasspath typically takes under half a minute. (That's a factor of 100 or more faster.) In javac, it proved necessary to implement -Xbootclasspath options even in module mode (Note, I mean the javac options here, not the underlying JVM options). In practice, it was reasonably easy to augment the search for platform classes with the extra locations being provided by -Xbootclasspath, and -Xbootclasspath/p: in particular. Surely it is possible to do something similar in the JVM. It might be idealistic to say we should not to subvert the classes provided by the module system, but pragmatically, there are times when it will be necessary. -- Jon From forax at univ-mlv.fr Fri Mar 26 15:49:52 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 26 Mar 2010 23:49:52 +0100 Subject: -Xbootclasspath/p: (or similar) please In-Reply-To: <4BAD1A01.8090502@sun.com> References: <4BAD1A01.8090502@sun.com> Message-ID: <4BAD3A10.9040709@univ-mlv.fr> Le 26/03/2010 21:33, Jonathan Gibbons a ?crit : > What are the limitations of the bootclasspath options in the jigsaw > module images? > > "java -X" says they cannot be used when executing a module. Does that > also apply when the module is implicit, as in tools invoked by the > launcher, like javac, javadoc, etc. > > If so, can I put it a BIG request that -Xbootclasspath/p: or an > equivalent should be supported. I think it is CRITICALLY important > for JDK developers to have a way of substituting new classes for > existing ones when working on bug fixes and similar work. In the jdk/ > repo I know that folk tend to do a partial build and execute out of > the "build/classes/" directory. This will not work in the modules > world. In the langtools/ repo, we routinely compile just the langtools > classes and execute them with a existing JDK using > -Xbootclasspath/p. The limitation on -Xbootclasspath means that this > too will not work in the modules world. > > Development will be severely impacted if we have to build new modules > images every time we want to try out a fix. > > From a testing point of view, I know that most of the langtools tests > exercise functionality that is independent of whether JDK is running > in legacy mode or modules mode. But, currently 31 tests fail in > modules mode. These tests do not fail in legacy mode. This means the > fixes for these tests will need to be tested on a full modules build. > Starting from scratch, a full modules build takes me about an hour, > whereas building langtools and using -Xbootclasspath typically takes > under half a minute. (That's a factor of 100 or more faster.) > > In javac, it proved necessary to implement -Xbootclasspath options > even in module mode (Note, I mean the javac options here, not the > underlying JVM options). In practice, it was reasonably easy to > augment the search for platform classes with the extra locations being > provided by -Xbootclasspath, and -Xbootclasspath/p: in particular. > Surely it is possible to do something similar in the JVM. It might be > idealistic to say we should not to subvert the classes provided by the > module system, but pragmatically, there are times when it will be > necessary. > > -- Jon +1 Patching anything in the JDK without -Xbootclasspath takes a loooong time. Also some languages that run on top of the VM like JRuby or Jython put their own runtime in the boot classpath. R?mi From Jonathan.Gibbons at Sun.COM Fri Mar 26 15:49:09 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 26 Mar 2010 15:49:09 -0700 Subject: -Xbootclasspath/p: (or similar) please In-Reply-To: <4BAD3A10.9040709@univ-mlv.fr> References: <4BAD1A01.8090502@sun.com> <4BAD3A10.9040709@univ-mlv.fr> Message-ID: <4BAD39E5.5050606@sun.com> R?mi Forax wrote: > Le 26/03/2010 21:33, Jonathan Gibbons a ?crit : >> What are the limitations of the bootclasspath options in the jigsaw >> module images? >> >> "java -X" says they cannot be used when executing a module. Does >> that also apply when the module is implicit, as in tools invoked by >> the launcher, like javac, javadoc, etc. >> >> If so, can I put it a BIG request that -Xbootclasspath/p: or an >> equivalent should be supported. I think it is CRITICALLY important >> for JDK developers to have a way of substituting new classes for >> existing ones when working on bug fixes and similar work. In the >> jdk/ repo I know that folk tend to do a partial build and execute out >> of the "build/classes/" directory. This will not work in the modules >> world. In the langtools/ repo, we routinely compile just the >> langtools classes and execute them with a existing JDK using >> -Xbootclasspath/p. The limitation on -Xbootclasspath means that >> this too will not work in the modules world. >> >> Development will be severely impacted if we have to build new modules >> images every time we want to try out a fix. >> >> From a testing point of view, I know that most of the langtools tests >> exercise functionality that is independent of whether JDK is running >> in legacy mode or modules mode. But, currently 31 tests fail in >> modules mode. These tests do not fail in legacy mode. This means the >> fixes for these tests will need to be tested on a full modules >> build. Starting from scratch, a full modules build takes me about an >> hour, whereas building langtools and using -Xbootclasspath typically >> takes under half a minute. (That's a factor of 100 or more faster.) >> >> In javac, it proved necessary to implement -Xbootclasspath options >> even in module mode (Note, I mean the javac options here, not the >> underlying JVM options). In practice, it was reasonably easy to >> augment the search for platform classes with the extra locations >> being provided by -Xbootclasspath, and -Xbootclasspath/p: in >> particular. Surely it is possible to do something similar in the >> JVM. It might be idealistic to say we should not to subvert the >> classes provided by the module system, but pragmatically, there are >> times when it will be necessary. >> >> -- Jon > > +1 > Patching anything in the JDK without -Xbootclasspath takes a loooong > time. > > Also some languages that run on top of the VM like JRuby or Jython > put their own runtime in the boot classpath. > > R?mi I would expect that any software packages that get delivered to the end user would include a module that can be installed, so that it is not necessary to put classes on the bootclasspath. After all, we *are* trying to eliminate "jar hell". The issue is arguably less clear for software that is of a transitory nature, such as a patch being tested. In this case, a little bit of carefully controlled "jar hell" may be good for the sole time the patch is going to be run. -- Jon From Jonathan.Gibbons at Sun.COM Fri Mar 26 15:59:53 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 26 Mar 2010 15:59:53 -0700 Subject: langtools failures on modules build Message-ID: <4BAD3C69.403@sun.com> Here's the analysis of the test failures from the langtools tests on a full modules build (jdk-module-image, using my latest langtools) The good news is that many of the test failures are the result of assumptions that are no longer true, such as the existence of rt.jar, tools.jar and ct.sym. It is a fairly common pattern in langtools to examine rt.jar and similar files directly, to infer the correct results of using public API, especially JSR199 API. These tests all have to be adapted to the brave new world of modules, preferably making the tests work in both legacy and modular scenarios. javah and apt both have hard failures that need to be fixed. The javah failure will probably prevent using jdk-module-image as a BOOT_DIR to build JDK. The new javac itself was good enough to build JDK in legacy mode. -- Jon -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: jtreg.langtools.txt Url: http://mail.openjdk.java.net/pipermail/jigsaw-dev/attachments/20100326/52c530b2/jtreg.langtools.txt From forax at univ-mlv.fr Fri Mar 26 16:39:07 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sat, 27 Mar 2010 00:39:07 +0100 Subject: -Xbootclasspath/p: (or similar) please In-Reply-To: <4BAD39E5.5050606@sun.com> References: <4BAD1A01.8090502@sun.com> <4BAD3A10.9040709@univ-mlv.fr> <4BAD39E5.5050606@sun.com> Message-ID: <4BAD459B.7020502@univ-mlv.fr> Le 26/03/2010 23:49, Jonathan Gibbons a ?crit : > R?mi Forax wrote: >> Le 26/03/2010 21:33, Jonathan Gibbons a ?crit : >>> What are the limitations of the bootclasspath options in the jigsaw >>> module images? >>> >>> "java -X" says they cannot be used when executing a module. Does >>> that also apply when the module is implicit, as in tools invoked by >>> the launcher, like javac, javadoc, etc. >>> >>> If so, can I put it a BIG request that -Xbootclasspath/p: or an >>> equivalent should be supported. I think it is CRITICALLY important >>> for JDK developers to have a way of substituting new classes for >>> existing ones when working on bug fixes and similar work. In the >>> jdk/ repo I know that folk tend to do a partial build and execute >>> out of the "build/classes/" directory. This will not work in the >>> modules world. In the langtools/ repo, we routinely compile just the >>> langtools classes and execute them with a existing JDK using >>> -Xbootclasspath/p. The limitation on -Xbootclasspath means that >>> this too will not work in the modules world. >>> >>> Development will be severely impacted if we have to build new >>> modules images every time we want to try out a fix. >>> >>> From a testing point of view, I know that most of the langtools >>> tests exercise functionality that is independent of whether JDK is >>> running in legacy mode or modules mode. But, currently 31 tests fail >>> in modules mode. These tests do not fail in legacy mode. This means >>> the fixes for these tests will need to be tested on a full modules >>> build. Starting from scratch, a full modules build takes me about >>> an hour, whereas building langtools and using -Xbootclasspath >>> typically takes under half a minute. (That's a factor of 100 or more >>> faster.) >>> >>> In javac, it proved necessary to implement -Xbootclasspath options >>> even in module mode (Note, I mean the javac options here, not the >>> underlying JVM options). In practice, it was reasonably easy to >>> augment the search for platform classes with the extra locations >>> being provided by -Xbootclasspath, and -Xbootclasspath/p: in >>> particular. Surely it is possible to do something similar in the >>> JVM. It might be idealistic to say we should not to subvert the >>> classes provided by the module system, but pragmatically, there are >>> times when it will be necessary. >>> >>> -- Jon >> >> +1 >> Patching anything in the JDK without -Xbootclasspath takes a loooong >> time. >> >> Also some languages that run on top of the VM like JRuby or Jython >> put their own runtime in the boot classpath. >> >> R?mi > > I would expect that any software packages that get delivered to the > end user would include a module that can be installed, so that it is > not necessary to put classes on the bootclasspath. After all, we *are* > trying to eliminate "jar hell". The issue is arguably less clear for > software that is of a transitory nature, such as a patch being > tested. In this case, a little bit of carefully controlled "jar hell" > may be good for the sole time the patch is going to be run. JVM language runtimes are often more than sofware packages and use nasty tricks like accessing to jdk internals (sun.misc.unsafe) or relying on bytecode injection, etc. > > -- Jon R?mi From Jonathan.Gibbons at Sun.COM Fri Mar 26 17:08:59 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 26 Mar 2010 17:08:59 -0700 Subject: -Xbootclasspath/p: (or similar) please In-Reply-To: <4BAD459B.7020502@univ-mlv.fr> References: <4BAD1A01.8090502@sun.com> <4BAD3A10.9040709@univ-mlv.fr> <4BAD39E5.5050606@sun.com> <4BAD459B.7020502@univ-mlv.fr> Message-ID: <4BAD4C9B.3030800@sun.com> R?mi Forax wrote: > Le 26/03/2010 23:49, Jonathan Gibbons a ?crit : >> R?mi Forax wrote: >>> Le 26/03/2010 21:33, Jonathan Gibbons a ?crit : >>>> What are the limitations of the bootclasspath options in the jigsaw >>>> module images? >>>> >>>> "java -X" says they cannot be used when executing a module. Does >>>> that also apply when the module is implicit, as in tools invoked by >>>> the launcher, like javac, javadoc, etc. >>>> >>>> If so, can I put it a BIG request that -Xbootclasspath/p: or an >>>> equivalent should be supported. I think it is CRITICALLY important >>>> for JDK developers to have a way of substituting new classes for >>>> existing ones when working on bug fixes and similar work. In the >>>> jdk/ repo I know that folk tend to do a partial build and execute >>>> out of the "build/classes/" directory. This will not work in the >>>> modules world. In the langtools/ repo, we routinely compile just >>>> the langtools classes and execute them with a existing JDK using >>>> -Xbootclasspath/p. The limitation on -Xbootclasspath means that >>>> this too will not work in the modules world. >>>> >>>> Development will be severely impacted if we have to build new >>>> modules images every time we want to try out a fix. >>>> >>>> From a testing point of view, I know that most of the langtools >>>> tests exercise functionality that is independent of whether JDK is >>>> running in legacy mode or modules mode. But, currently 31 tests >>>> fail in modules mode. These tests do not fail in legacy mode. This >>>> means the fixes for these tests will need to be tested on a full >>>> modules build. Starting from scratch, a full modules build takes >>>> me about an hour, whereas building langtools and using >>>> -Xbootclasspath typically takes under half a minute. (That's a >>>> factor of 100 or more faster.) >>>> >>>> In javac, it proved necessary to implement -Xbootclasspath options >>>> even in module mode (Note, I mean the javac options here, not the >>>> underlying JVM options). In practice, it was reasonably easy to >>>> augment the search for platform classes with the extra locations >>>> being provided by -Xbootclasspath, and -Xbootclasspath/p: in >>>> particular. Surely it is possible to do something similar in the >>>> JVM. It might be idealistic to say we should not to subvert the >>>> classes provided by the module system, but pragmatically, there are >>>> times when it will be necessary. >>>> >>>> -- Jon >>> >>> +1 >>> Patching anything in the JDK without -Xbootclasspath takes a loooong >>> time. >>> >>> Also some languages that run on top of the VM like JRuby or Jython >>> put their own runtime in the boot classpath. >>> >>> R?mi >> >> I would expect that any software packages that get delivered to the >> end user would include a module that can be installed, so that it is >> not necessary to put classes on the bootclasspath. After all, we >> *are* trying to eliminate "jar hell". The issue is arguably less >> clear for software that is of a transitory nature, such as a patch >> being tested. In this case, a little bit of carefully controlled >> "jar hell" may be good for the sole time the patch is going to be run. > > JVM language runtimes are often more than sofware packages and use > nasty tricks like > accessing to jdk internals (sun.misc.unsafe) or relying on bytecode > injection, etc. > >> >> -- Jon > > R?mi Oh boy, what fun we have ahead ;-) -- Jon From Mandy.Chung at Sun.COM Fri Mar 26 23:36:14 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Fri, 26 Mar 2010 23:36:14 -0700 Subject: -Xbootclasspath/p: (or similar) please In-Reply-To: <4BAD1A01.8090502@sun.com> References: <4BAD1A01.8090502@sun.com> Message-ID: <4BADA75E.4070403@sun.com> Jonathan Gibbons wrote: > What are the limitations of the bootclasspath options in the jigsaw > module images? > bootclasspath continues to work as it is today in legacy mode. bootclasspath is not supported in module mode. > "java -X" says they cannot be used when executing a module. Does that > also apply when the module is implicit, as in tools invoked by the > launcher, like javac, javadoc, etc. I converted most of the jdk tools to run in module mode and -J-Xbootclasspath can no longer be passed to those tools running in module mode. If a tool remains in legacy mode, it will continue to accept -J-Xbootclasspath option. > > If so, can I put it a BIG request that -Xbootclasspath/p: or an > equivalent should be supported. I think it is CRITICALLY important > for JDK developers to have a way of substituting new classes for > existing ones when working on bug fixes and similar work. Completely agree. > In the jdk/ repo I know that folk tend to do a partial build and > execute out of the "build/classes/" directory. This will not work in > the modules world. We talked about implementing the "zero mod" facility (currently in javac) in the Java launcher so that we can say something like: $ java -L $OUTPUTDIR/modules MyUnitTest > In the langtools/ repo, we routinely compile just the langtools > classes and execute them with a existing JDK using > -Xbootclasspath/p. The limitation on -Xbootclasspath means that this > too will not work in the modules world. > I would imagine that we could do something like this for langtools: $ java -L langtools/dist/modules JavacTest I haven't put much thought about this though and there are possibly issues to be resolved. Hmm.... javac and other jdk tools take -J and pass the argument to the launcher invoking the tool. I don't think we can prepend "-L " with -J. > Development will be severely impacted if we have to build new modules > images every time we want to try out a fix. > > From a testing point of view, I know that most of the langtools tests > exercise functionality that is independent of whether JDK is running > in legacy mode or modules mode. But, currently 31 tests fail in > modules mode. These tests do not fail in legacy mode. This means the > fixes for these tests will need to be tested on a full modules build. > Starting from scratch, a full modules build takes me about an hour, > whereas building langtools and using -Xbootclasspath typically takes > under half a minute. (That's a factor of 100 or more faster.) > My bad. I ought to speed up the modules build to help our development. It's on my list to work on. Mark also has a patch to replace cpio with rsync to improve the subsequent modules build time. > In javac, it proved necessary to implement -Xbootclasspath options > even in module mode (Note, I mean the javac options here, not the > underlying JVM options). In practice, it was reasonably easy to > augment the search for platform classes with the extra locations being > provided by -Xbootclasspath, and -Xbootclasspath/p: in particular. > Surely it is possible to do something similar in the JVM. It might be > idealistic to say we should not to subvert the classes provided by the > module system, but pragmatically, there are times when it will be > necessary. > I would hope that there is a way to address this requirement in the modules world while not needed to keep bootclasspath :) Mandy From Jonathan.Gibbons at Sun.COM Sat Mar 27 11:05:42 2010 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Sat, 27 Mar 2010 11:05:42 -0700 Subject: -Xbootclasspath/p: (or similar) please In-Reply-To: <4BADA75E.4070403@sun.com> References: <4BAD1A01.8090502@sun.com> <4BADA75E.4070403@sun.com> Message-ID: <4BAE48F6.5000901@sun.com> Mandy Chung wrote: > Jonathan Gibbons wrote: >> What are the limitations of the bootclasspath options in the jigsaw >> module images? >> > > bootclasspath continues to work as it is today in legacy mode. > bootclasspath is not supported in module mode. >> "java -X" says they cannot be used when executing a module. Does >> that also apply when the module is implicit, as in tools invoked by >> the launcher, like javac, javadoc, etc. > > I converted most of the jdk tools to run in module mode and > -J-Xbootclasspath can no longer be passed to those tools running in > module mode. > > If a tool remains in legacy mode, it will continue to accept > -J-Xbootclasspath option. > >> >> If so, can I put it a BIG request that -Xbootclasspath/p: or an >> equivalent should be supported. I think it is CRITICALLY important >> for JDK developers to have a way of substituting new classes for >> existing ones when working on bug fixes and similar work. > > Completely agree. How about if I suggest that all Jigsaw developers have to fix one or more bugs in jdk-module-image to share the pain ;-) >> In the jdk/ repo I know that folk tend to do a partial build and >> execute out of the "build/classes/" directory. This will not work in >> the modules world. > > We talked about implementing the "zero mod" facility (currently in > javac) in the Java launcher so that we can say something like: > > $ java -L $OUTPUTDIR/modules MyUnitTest We need another name here. "ZeroMod" in javac refers to something else. It is a private minimal alternate module system provided by javac for bootstrapping and for testing javac functionality independently of any external module system, such as Jigsaw. In time, ZeroMod should never be used outside the langtools/ repository. Separate from ZeroMod, javac can handle -L, -Xbootclasspath etc, and can handle "module directories", meaning directory hierarchies of classes that have not been converted into "proper modules" with jmod. This all works in conjunction with the Jigsaw module system, and does not require the ZeroMod module system. As a historical note, I had intended/expected ZeroMod to be used to bootstrap compile parts of the jdk/ repo up to the point where the Jigsaw module system was available. In practice, you have worked around the bootstrapping problem a different way, by compiling most of the jdk classes the same old way as ever (without any module info) and then compiling the **/module-info files separately, as a much later step. This allows us to use the Jigsaw code to handle these **/module-info files, at a cost of providing and using javac support for -Xbootclasspath. > >> In the langtools/ repo, we routinely compile just the langtools >> classes and execute them with a existing JDK using >> -Xbootclasspath/p. The limitation on -Xbootclasspath means that >> this too will not work in the modules world. >> > I would imagine that we could do something like this for langtools: > > $ java -L langtools/dist/modules JavacTest > > I haven't put much thought about this though and there are possibly > issues to be resolved. Hmm.... javac and other jdk tools take > -J and pass the argument to the launcher invoking the tool. > I don't think we can prepend "-L " with -J. While -L might be the basis of a direction to take, note that we would still need some effective way of incrementally updating a module library. Currently all the make rules for building modules and for knowing what goes where are all embodied in jdk/make/modules/ which is all-or-nothing and not incremental-friendly at all. Perhaps it would help if one of the by-products of a build was data in a form that could be given to a tool like jmod to say, "build me a new module Foo with these classes...". > >> Development will be severely impacted if we have to build new modules >> images every time we want to try out a fix. >> >> From a testing point of view, I know that most of the langtools tests >> exercise functionality that is independent of whether JDK is running >> in legacy mode or modules mode. But, currently 31 tests fail in >> modules mode. These tests do not fail in legacy mode. This means the >> fixes for these tests will need to be tested on a full modules >> build. Starting from scratch, a full modules build takes me about an >> hour, whereas building langtools and using -Xbootclasspath typically >> takes under half a minute. (That's a factor of 100 or more faster.) >> > > My bad. I ought to speed up the modules build to help our > development. It's on my list to work on. Mark also has a patch to > replace cpio with rsync to improve the subsequent modules build time. Having a faster build would be nice, but it will not be an effective solution to the problem. As an analogy, look how JDK developers tend to start off with a full build, then when they are fixing a bug, they go into a particular make/XYZ directory, rebuild a few classes, then run tests using build/PLATFORM_ARCH/classes, without rebuilding images, because building images is just too slow and inconvenient. That technique will no longer work -- you'll have to do an modules build because there is no other way to update the modules that you want to test. There is no way of saying "I've rebuilt java.lang.Integer: please incrementally rebuild whatever modules it lives in". Even if I know what module it gives in, there is no way I can use the build to say "just remake the XYZ module". It's even worse for langtools developers (!!) because the langtools classes live in one repository and the make files for modules live in another repository! And no, that is not a suggestion that langtools/ should merge back into jdk/ !!! Back before the days of OpenJDK, there was a way of doing the incremental development by making a shadow copy of a JDK image using symbolic links, and replacing/adding in new classes as necessary. (And who remembers the eponymous "gafter builds"!) When we went to OpenJDK, we put effort into using -Xbootclasspath/p: and making that work as a slicker more convenient way of testing incremental builds. I guess I can see a way going forward of hacks of the form ... here's a full copy of JDK, here's a bunch of classes, please update the classes in the Foo module with any that you find in my classes directory. > >> In javac, it proved necessary to implement -Xbootclasspath options >> even in module mode (Note, I mean the javac options here, not the >> underlying JVM options). In practice, it was reasonably easy to >> augment the search for platform classes with the extra locations >> being provided by -Xbootclasspath, and -Xbootclasspath/p: in >> particular. Surely it is possible to do something similar in the >> JVM. It might be idealistic to say we should not to subvert the >> classes provided by the module system, but pragmatically, there are >> times when it will be necessary. >> > > I would hope that there is a way to address this requirement in the > modules world while not needed to keep bootclasspath :) > > Mandy > Yes. In fact, we have reached a notable milestone to be having this discussion. Up to now, we've been focussed on building the tools and infrastructure necessary to modularize the JDK. And we've achieved that! You can get a copy of the jigsaw repo, go "make modules", and a while and a few (ahem) cups of coffee later, you can have your very own jdk-module-image. Woohoo! No lack of congratulations to the team on that aspect. But, cynic that I am, this JDK is not perfect :-( and we need to be able to work in this new modular world, so that by the time we deliver JDK7 it is somewhat closer to perfection. In particular, we need to be able to fix bugs that only show up this modular world. Thus it doesn't matter that I can use -Xbootclasspath in a legacy mode. That's not what needs fixing. And *all* the old techniques have gone out the window. jdk/ developers can't use build/PLATFORM_ARCH/classes to avoid building images. langtools/ developers can't use "gafter builds", or -Xbootclasspath/p: We need a way of testing incremental updates. It needn't be specifically -Xbootclasspath/p: but that does provide an excellent model of what we should strive to achieve -- it's light weight, it's non-destructive, it doesn't require me to make a complete copy of JDK into which I can edit my classes. -- Jon From reinier at zwitserloot.com Sun Mar 28 14:42:23 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sun, 28 Mar 2010 23:42:23 +0200 Subject: -Xbootclasspath/p: (or similar) please In-Reply-To: <4BAE48F6.5000901@sun.com> References: <4BAD1A01.8090502@sun.com> <4BADA75E.4070403@sun.com> <4BAE48F6.5000901@sun.com> Message-ID: <560fb5ed1003281442i129f5b0eldfc956559d2ab339@mail.gmail.com> On apple JVMs, tools.jar is part of the *boot* classpath, and as a result you get the system javac. In projects where you want your own copy of javac, for example because you need specific features from javac7 but you don't want to demand a jvm7, then the only way to solve the problem is to add this javac.jar to the bootclasspath. Adding it to the plain classpath doesn't help because the system javac, as it is part of the boot classpath, wins out over it. This and a gazillion other situations require -Xbootclasspath(/p) to work around the issues. I get that 'classpath' isn't really a concept that remains intact in a module world, but there is an obvious analogy, isn't there? "Load these modules before any others". Also, I take it there's some sort of ordering that occurs when 2 or more modules all say they provide the same service. The analogy to bootclasspath for many many cases where its used today would be to say: *THIS* module should be used to satisfy a requirement for "a.b.c" in preference to whatever the system module registry has. --Reinier Zwitserloot On Sat, Mar 27, 2010 at 8:05 PM, Jonathan Gibbons wrote: > Mandy Chung wrote: > >> Jonathan Gibbons wrote: >> >>> What are the limitations of the bootclasspath options in the jigsaw >>> module images? >>> >>> >> bootclasspath continues to work as it is today in legacy mode. >> bootclasspath is not supported in module mode. >> >>> "java -X" says they cannot be used when executing a module. Does that >>> also apply when the module is implicit, as in tools invoked by the launcher, >>> like javac, javadoc, etc. >>> >> >> I converted most of the jdk tools to run in module mode and >> -J-Xbootclasspath can no longer be passed to those tools running in module >> mode. >> >> If a tool remains in legacy mode, it will continue to accept >> -J-Xbootclasspath option. >> >> >>> If so, can I put it a BIG request that -Xbootclasspath/p: or an >>> equivalent should be supported. I think it is CRITICALLY important for JDK >>> developers to have a way of substituting new classes for existing ones when >>> working on bug fixes and similar work. >>> >> >> Completely agree. >> > > How about if I suggest that all Jigsaw developers have to fix one or more > bugs in jdk-module-image to share the pain ;-) > > > > In the jdk/ repo I know that folk tend to do a partial build and execute >>> out of the "build/classes/" directory. This will not work in the modules >>> world. >>> >> >> We talked about implementing the "zero mod" facility (currently in javac) >> in the Java launcher so that we can say something like: >> >> $ java -L $OUTPUTDIR/modules MyUnitTest >> > > We need another name here. "ZeroMod" in javac refers to something else. It > is a private minimal alternate module system provided by javac for > bootstrapping and for testing javac functionality independently of any > external module system, such as Jigsaw. In time, ZeroMod should never be > used outside the langtools/ repository. > Separate from ZeroMod, javac can handle -L, -Xbootclasspath etc, and can > handle "module directories", meaning directory hierarchies of classes that > have not been converted into "proper modules" with jmod. This all works in > conjunction with the Jigsaw module system, and does not require the ZeroMod > module system. > > As a historical note, I had intended/expected ZeroMod to be used to > bootstrap compile parts of the jdk/ repo up to the point where the Jigsaw > module system was available. In practice, you have worked around the > bootstrapping problem a different way, by compiling most of the jdk classes > the same old way as ever (without any module info) and then compiling the > **/module-info files separately, as a much later step. This allows us to use > the Jigsaw code to handle these **/module-info files, at a cost of providing > and using javac support for -Xbootclasspath. > > >> In the langtools/ repo, we routinely compile just the langtools classes >>> and execute them with a existing JDK using -Xbootclasspath/p. The >>> limitation on -Xbootclasspath means that this too will not work in the >>> modules world. >>> >>> I would imagine that we could do something like this for langtools: >> >> $ java -L langtools/dist/modules JavacTest >> >> I haven't put much thought about this though and there are possibly issues >> to be resolved. Hmm.... javac and other jdk tools take -J and >> pass the argument to the launcher invoking the tool. I don't think we can >> prepend "-L " with -J. >> > > While -L might be the basis of a direction to take, note that we would > still need some effective way of incrementally updating a module library. > Currently all the make rules for building modules and for knowing what goes > where are all embodied in jdk/make/modules/ which is all-or-nothing and not > incremental-friendly at all. Perhaps it would help if one of the > by-products of a build was data in a form that could be given to a tool like > jmod to say, "build me a new module Foo with these classes...". > > >> Development will be severely impacted if we have to build new modules >>> images every time we want to try out a fix. >>> >>> From a testing point of view, I know that most of the langtools tests >>> exercise functionality that is independent of whether JDK is running in >>> legacy mode or modules mode. But, currently 31 tests fail in modules mode. >>> These tests do not fail in legacy mode. This means the fixes for these >>> tests will need to be tested on a full modules build. Starting from >>> scratch, a full modules build takes me about an hour, whereas building >>> langtools and using -Xbootclasspath typically takes under half a minute. >>> (That's a factor of 100 or more faster.) >>> >>> >> My bad. I ought to speed up the modules build to help our development. >> It's on my list to work on. Mark also has a patch to replace cpio with >> rsync to improve the subsequent modules build time. >> > > Having a faster build would be nice, but it will not be an effective > solution to the problem. As an analogy, look how JDK developers tend to > start off with a full build, then when they are fixing a bug, they go into a > particular make/XYZ directory, rebuild a few classes, then run tests using > build/PLATFORM_ARCH/classes, without rebuilding images, because building > images is just too slow and inconvenient. That technique will no longer work > -- you'll have to do an modules build because there is no other way to > update the modules that you want to test. There is no way of saying "I've > rebuilt java.lang.Integer: please incrementally rebuild whatever modules it > lives in". Even if I know what module it gives in, there is no way I can > use the build to say "just remake the XYZ module". It's even worse for > langtools developers (!!) because the langtools classes live in one > repository and the make files for modules live in another repository! And > no, that is not a suggestion that langtools/ should merge back into jdk/ !!! > > Back before the days of OpenJDK, there was a way of doing the incremental > development by making a shadow copy of a JDK image using symbolic links, and > replacing/adding in new classes as necessary. (And who remembers the > eponymous "gafter builds"!) When we went to OpenJDK, we put effort into > using -Xbootclasspath/p: and making that work as a slicker more convenient > way of testing incremental builds. > > I guess I can see a way going forward of hacks of the form ... here's a > full copy of JDK, here's a bunch of classes, please update the classes in > the Foo module with any that you find in my classes directory. > > > >> In javac, it proved necessary to implement -Xbootclasspath options even >>> in module mode (Note, I mean the javac options here, not the underlying JVM >>> options). In practice, it was reasonably easy to augment the search for >>> platform classes with the extra locations being provided by -Xbootclasspath, >>> and -Xbootclasspath/p: in particular. Surely it is possible to do something >>> similar in the JVM. It might be idealistic to say we should not to subvert >>> the classes provided by the module system, but pragmatically, there are >>> times when it will be necessary. >>> >>> >> I would hope that there is a way to address this requirement in the >> modules world while not needed to keep bootclasspath :) >> >> Mandy >> >> > Yes. In fact, we have reached a notable milestone to be having this > discussion. Up to now, we've been focussed on building the tools and > infrastructure necessary to modularize the JDK. And we've achieved that! > You can get a copy of the jigsaw repo, go "make modules", and a while and a > few (ahem) cups of coffee later, you can have your very own > jdk-module-image. Woohoo! No lack of congratulations to the team on that > aspect. > > But, cynic that I am, this JDK is not perfect :-( and we need to be able to > work in this new modular world, so that by the time we deliver JDK7 it is > somewhat closer to perfection. In particular, we need to be able to fix bugs > that only show up this modular world. Thus it doesn't matter that I can use > -Xbootclasspath in a legacy mode. That's not what needs fixing. > > And *all* the old techniques have gone out the window. jdk/ developers > can't use build/PLATFORM_ARCH/classes to avoid building images. langtools/ > developers can't use "gafter builds", or -Xbootclasspath/p: > > We need a way of testing incremental updates. It needn't be specifically > -Xbootclasspath/p: but that does provide an excellent model of what we > should strive to achieve -- it's light weight, it's non-destructive, it > doesn't require me to make a complete copy of JDK into which I can edit my > classes. > > -- Jon > > > From Weijun.Wang at Sun.COM Sun Mar 28 23:00:49 2010 From: Weijun.Wang at Sun.COM (Weijun Wang) Date: Mon, 29 Mar 2010 14:00:49 +0800 Subject: Creating new packages in the world of modules Message-ID: <53C308B4-C93A-4E51-80D8-F14ED95E59C1@sun.com> Hi Alan I'm planning to add two RFEs to OpenJDK: 1. A SASL mechanism called NTLM, which includes two new packages: a. com.sun.security.sasl.ntlm: The SASL provider b. com.sun.security.ntlm: The raw NTLM API Here I've created an NTLM API which I hope can also be used by sun/net/www/protocol/http/ntlm/NTLMAuthentication.java. 2. Keytool and Jarsigner as JSR 199 Tools, which include interfaces in a new package com.sun.security.tools, implementations in sun.security.tools and new META-INF/services/com.sun.security.tools.* entries. I know you have spent a lot of time recently to make sure classes to be distributed in different modules and their dependencies carefully maintained. If I want to add the new packages above, what do I need to take care of most? Any configuration files to modify? Is there a latest map showing the content of current existing modules? BTW, what is the fate of ServiceLoader in modules? Will META-INF/services entries spread into different modules and ServiceLoader tries to get a union of them for each service class? Thanks Max From jonathan.gibbons at oracle.com Mon Mar 29 08:01:20 2010 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 29 Mar 2010 08:01:20 -0700 Subject: -Xbootclasspath/p: (or similar) please In-Reply-To: <560fb5ed1003281442i129f5b0eldfc956559d2ab339@mail.gmail.com> References: <4BAD1A01.8090502@sun.com> <4BADA75E.4070403@sun.com> <4BAE48F6.5000901@sun.com> <560fb5ed1003281442i129f5b0eldfc956559d2ab339@mail.gmail.com> Message-ID: <4BB0C0C0.9090801@oracle.com> Reinier, That's an interesting and different aspect of the [problem. You'll only get the problem on a Mac if you use the current Mac JVM as the baseline. The problems I'm putting forward are for when the baseline is JDK7, so we won't see these problems on a Mac until a JDK7 Mac port is available, and then, hopefully, it'll support Jigsaw, and have classes loaded the same way as on other Jigsaw impls. The -L switch should allow the user to (create and) use a new module library instead of the standard system module library. That can be the basis of the "module version of bootclasspath". Then, the problems I've been describing boil down to being able to create a new library to use. Mandy also alluded to the minor issue of making sure tools like javac can support -J-L so that you can set -L for the JVM running tools started by the launcher. The problem with creating a new library is that currently the rules to build a library are buried in some complicated makefiles and are not currently capable of incremental building. I suspect the way forward is to figure out how to solve that problem -- by making it possible to update a JDK library without running the full set of makefiles. Note that apps running on top of JDK should not have all the same problems we're seeing with JDK right now. JDK has its own special problems caused by the complexity of modularizing the existing JDK code and by the need to be able to bootstrap-build the JDK itself. The bootstrap problem means that the classes for all the modules are currently still in a single src/*/classes hierarchy, and the build uses custom config files and class file dependency analysis to figure out which classes go in which module and which modules depend on other modules. Hopefully, apps running on top of JDK should not have these same problems. -- Jon Reinier Zwitserloot wrote: > On apple JVMs, tools.jar is part of the *boot* classpath, and as a result > you get the system javac. In projects where you want your own copy of javac, > for example because you need specific features from javac7 but you don't > want to demand a jvm7, then the only way to solve the problem is to add this > javac.jar to the bootclasspath. Adding it to the plain classpath doesn't > help because the system javac, as it is part of the boot classpath, wins out > over it. > > This and a gazillion other situations require -Xbootclasspath(/p) to work > around the issues. > > I get that 'classpath' isn't really a concept that remains intact in a > module world, but there is an obvious analogy, isn't there? "Load these > modules before any others". Also, I take it there's some sort of ordering > that occurs when 2 or more modules all say they provide the same service. > The analogy to bootclasspath for many many cases where its used today would > be to say: *THIS* module should be used to satisfy a requirement for "a.b.c" > in preference to whatever the system module registry has. > > > > --Reinier Zwitserloot > > > > On Sat, Mar 27, 2010 at 8:05 PM, Jonathan Gibbons > wrote: > > >> Mandy Chung wrote: >> >> >>> Jonathan Gibbons wrote: >>> >>> >>>> What are the limitations of the bootclasspath options in the jigsaw >>>> module images? >>>> >>>> >>>> >>> bootclasspath continues to work as it is today in legacy mode. >>> bootclasspath is not supported in module mode. >>> >>> >>>> "java -X" says they cannot be used when executing a module. Does that >>>> also apply when the module is implicit, as in tools invoked by the launcher, >>>> like javac, javadoc, etc. >>>> >>>> >>> I converted most of the jdk tools to run in module mode and >>> -J-Xbootclasspath can no longer be passed to those tools running in module >>> mode. >>> >>> If a tool remains in legacy mode, it will continue to accept >>> -J-Xbootclasspath option. >>> >>> >>> >>>> If so, can I put it a BIG request that -Xbootclasspath/p: or an >>>> equivalent should be supported. I think it is CRITICALLY important for JDK >>>> developers to have a way of substituting new classes for existing ones when >>>> working on bug fixes and similar work. >>>> >>>> >>> Completely agree. >>> >>> >> How about if I suggest that all Jigsaw developers have to fix one or more >> bugs in jdk-module-image to share the pain ;-) >> >> >> >> In the jdk/ repo I know that folk tend to do a partial build and execute >> >>>> out of the "build/classes/" directory. This will not work in the modules >>>> world. >>>> >>>> >>> We talked about implementing the "zero mod" facility (currently in javac) >>> in the Java launcher so that we can say something like: >>> >>> $ java -L $OUTPUTDIR/modules MyUnitTest >>> >>> >> We need another name here. "ZeroMod" in javac refers to something else. It >> is a private minimal alternate module system provided by javac for >> bootstrapping and for testing javac functionality independently of any >> external module system, such as Jigsaw. In time, ZeroMod should never be >> used outside the langtools/ repository. >> Separate from ZeroMod, javac can handle -L, -Xbootclasspath etc, and can >> handle "module directories", meaning directory hierarchies of classes that >> have not been converted into "proper modules" with jmod. This all works in >> conjunction with the Jigsaw module system, and does not require the ZeroMod >> module system. >> >> As a historical note, I had intended/expected ZeroMod to be used to >> bootstrap compile parts of the jdk/ repo up to the point where the Jigsaw >> module system was available. In practice, you have worked around the >> bootstrapping problem a different way, by compiling most of the jdk classes >> the same old way as ever (without any module info) and then compiling the >> **/module-info files separately, as a much later step. This allows us to use >> the Jigsaw code to handle these **/module-info files, at a cost of providing >> and using javac support for -Xbootclasspath. >> >> >> >>> In the langtools/ repo, we routinely compile just the langtools classes >>> >>>> and execute them with a existing JDK using -Xbootclasspath/p. The >>>> limitation on -Xbootclasspath means that this too will not work in the >>>> modules world. >>>> >>>> I would imagine that we could do something like this for langtools: >>>> >>> $ java -L langtools/dist/modules JavacTest >>> >>> I haven't put much thought about this though and there are possibly issues >>> to be resolved. Hmm.... javac and other jdk tools take -J and >>> pass the argument to the launcher invoking the tool. I don't think we can >>> prepend "-L " with -J. >>> >>> >> While -L might be the basis of a direction to take, note that we would >> still need some effective way of incrementally updating a module library. >> Currently all the make rules for building modules and for knowing what goes >> where are all embodied in jdk/make/modules/ which is all-or-nothing and not >> incremental-friendly at all. Perhaps it would help if one of the >> by-products of a build was data in a form that could be given to a tool like >> jmod to say, "build me a new module Foo with these classes...". >> >> >> >>> Development will be severely impacted if we have to build new modules >>> >>>> images every time we want to try out a fix. >>>> >>>> From a testing point of view, I know that most of the langtools tests >>>> exercise functionality that is independent of whether JDK is running in >>>> legacy mode or modules mode. But, currently 31 tests fail in modules mode. >>>> These tests do not fail in legacy mode. This means the fixes for these >>>> tests will need to be tested on a full modules build. Starting from >>>> scratch, a full modules build takes me about an hour, whereas building >>>> langtools and using -Xbootclasspath typically takes under half a minute. >>>> (That's a factor of 100 or more faster.) >>>> >>>> >>>> >>> My bad. I ought to speed up the modules build to help our development. >>> It's on my list to work on. Mark also has a patch to replace cpio with >>> rsync to improve the subsequent modules build time. >>> >>> >> Having a faster build would be nice, but it will not be an effective >> solution to the problem. As an analogy, look how JDK developers tend to >> start off with a full build, then when they are fixing a bug, they go into a >> particular make/XYZ directory, rebuild a few classes, then run tests using >> build/PLATFORM_ARCH/classes, without rebuilding images, because building >> images is just too slow and inconvenient. That technique will no longer work >> -- you'll have to do an modules build because there is no other way to >> update the modules that you want to test. There is no way of saying "I've >> rebuilt java.lang.Integer: please incrementally rebuild whatever modules it >> lives in". Even if I know what module it gives in, there is no way I can >> use the build to say "just remake the XYZ module". It's even worse for >> langtools developers (!!) because the langtools classes live in one >> repository and the make files for modules live in another repository! And >> no, that is not a suggestion that langtools/ should merge back into jdk/ !!! >> >> Back before the days of OpenJDK, there was a way of doing the incremental >> development by making a shadow copy of a JDK image using symbolic links, and >> replacing/adding in new classes as necessary. (And who remembers the >> eponymous "gafter builds"!) When we went to OpenJDK, we put effort into >> using -Xbootclasspath/p: and making that work as a slicker more convenient >> way of testing incremental builds. >> >> I guess I can see a way going forward of hacks of the form ... here's a >> full copy of JDK, here's a bunch of classes, please update the classes in >> the Foo module with any that you find in my classes directory. >> >> >> >> >>> In javac, it proved necessary to implement -Xbootclasspath options even >>> >>>> in module mode (Note, I mean the javac options here, not the underlying JVM >>>> options). In practice, it was reasonably easy to augment the search for >>>> platform classes with the extra locations being provided by -Xbootclasspath, >>>> and -Xbootclasspath/p: in particular. Surely it is possible to do something >>>> similar in the JVM. It might be idealistic to say we should not to subvert >>>> the classes provided by the module system, but pragmatically, there are >>>> times when it will be necessary. >>>> >>>> >>>> >>> I would hope that there is a way to address this requirement in the >>> modules world while not needed to keep bootclasspath :) >>> >>> Mandy >>> >>> >>> >> Yes. In fact, we have reached a notable milestone to be having this >> discussion. Up to now, we've been focussed on building the tools and >> infrastructure necessary to modularize the JDK. And we've achieved that! >> You can get a copy of the jigsaw repo, go "make modules", and a while and a >> few (ahem) cups of coffee later, you can have your very own >> jdk-module-image. Woohoo! No lack of congratulations to the team on that >> aspect. >> >> But, cynic that I am, this JDK is not perfect :-( and we need to be able to >> work in this new modular world, so that by the time we deliver JDK7 it is >> somewhat closer to perfection. In particular, we need to be able to fix bugs >> that only show up this modular world. Thus it doesn't matter that I can use >> -Xbootclasspath in a legacy mode. That's not what needs fixing. >> >> And *all* the old techniques have gone out the window. jdk/ developers >> can't use build/PLATFORM_ARCH/classes to avoid building images. langtools/ >> developers can't use "gafter builds", or -Xbootclasspath/p: >> >> We need a way of testing incremental updates. It needn't be specifically >> -Xbootclasspath/p: but that does provide an excellent model of what we >> should strive to achieve -- it's light weight, it's non-destructive, it >> doesn't require me to make a complete copy of JDK into which I can edit my >> classes. >> >> -- Jon >> >> >> >> From jonathan.gibbons at oracle.com Mon Mar 29 08:04:53 2010 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 29 Mar 2010 08:04:53 -0700 Subject: Creating new packages in the world of modules In-Reply-To: <53C308B4-C93A-4E51-80D8-F14ED95E59C1@sun.com> References: <53C308B4-C93A-4E51-80D8-F14ED95E59C1@sun.com> Message-ID: <4BB0C195.6060105@oracle.com> Weijun Wang wrote: > Hi Alan > > I'm planning to add two RFEs to OpenJDK: > > 1. A SASL mechanism called NTLM, which includes two new packages: > > a. com.sun.security.sasl.ntlm: The SASL provider > b. com.sun.security.ntlm: The raw NTLM API > > Here I've created an NTLM API which I hope can also be used by sun/net/www/protocol/http/ntlm/NTLMAuthentication.java. > > 2. Keytool and Jarsigner as JSR 199 Tools, which include interfaces in a new package com.sun.security.tools, implementations in sun.security.tools and new META-INF/services/com.sun.security.tools.* entries. > > I know you have spent a lot of time recently to make sure classes to be distributed in different modules and their dependencies carefully maintained. If I want to add the new packages above, what do I need to take care of most? Any configuration files to modify? Is there a latest map showing the content of current existing modules? > > BTW, what is the fate of ServiceLoader in modules? Will META-INF/services entries spread into different modules and ServiceLoader tries to get a union of them for each service class? > > Thanks > Max > > Weijun, It may help to look in jdk/make/modules/** for *.config files. The build runs as a "standard build" to begin with, then enters "jdk/make/modules" near the end of the build to actually build the modules using the info in the config files. -- Jon From sean.mullan at oracle.com Mon Mar 29 10:02:28 2010 From: sean.mullan at oracle.com (Sean Mullan) Date: Mon, 29 Mar 2010 13:02:28 -0400 Subject: How to run jigsaw unit tests ... Message-ID: <4BB0DD24.7060008@oracle.com> How do you run the jigsaw unit tests? I'm getting compilation errors with jtreg - is there a new switch I have to specify to use modules? --Sean From jonathan.gibbons at oracle.com Mon Mar 29 10:16:05 2010 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 29 Mar 2010 10:16:05 -0700 Subject: How to run jigsaw unit tests ... In-Reply-To: <4BB0DD24.7060008@oracle.com> References: <4BB0DD24.7060008@oracle.com> Message-ID: <4BB0E055.7050702@oracle.com> Sean Mullan wrote: > How do you run the jigsaw unit tests? I'm getting compilation errors > with jtreg - is there a new switch I have to specify to use modules? > > --Sean How are you trying to run the tests and what compilation errors are you getting? -- Jon From sean.mullan at oracle.com Mon Mar 29 10:26:01 2010 From: sean.mullan at oracle.com (Sean Mullan) Date: Mon, 29 Mar 2010 13:26:01 -0400 Subject: How to run jigsaw unit tests ... In-Reply-To: <4BB0E055.7050702@oracle.com> References: <4BB0DD24.7060008@oracle.com> <4BB0E055.7050702@oracle.com> Message-ID: <4BB0E2A9.5050105@oracle.com> There is nothing under the jdk/build directory and this is where the tests are looking for javac. Will try to build again first. --Sean Jonathan Gibbons wrote: > Sean Mullan wrote: >> How do you run the jigsaw unit tests? I'm getting compilation errors >> with jtreg - is there a new switch I have to specify to use modules? >> >> --Sean > How are you trying to run the tests and what compilation errors are you > getting? > > -- Jon From jonathan.gibbons at oracle.com Mon Mar 29 10:41:34 2010 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 29 Mar 2010 10:41:34 -0700 Subject: How to run jigsaw unit tests ... In-Reply-To: <4BB0E2A9.5050105@oracle.com> References: <4BB0DD24.7060008@oracle.com> <4BB0E055.7050702@oracle.com> <4BB0E2A9.5050105@oracle.com> Message-ID: <4BB0E64E.9030404@oracle.com> Sean Mullan wrote: > There is nothing under the jdk/build directory and this is where the > tests are looking for javac. > > Will try to build again first. > > --Sean > > Jonathan Gibbons wrote: >> Sean Mullan wrote: >>> How do you run the jigsaw unit tests? I'm getting compilation errors >>> with jtreg - is there a new switch I have to specify to use modules? >>> >>> --Sean >> How are you trying to run the tests and what compilation errors are >> you getting? >> >> -- Jon Are you trying a jdk-only build, or a top level build? From the top level, you want to do "make modules", and that should create images named TOP/build/PLATFORM-ARCH/*image I don't know that you can just build within jdk ("cd jdk/; make -C make") but I would image that would start running into the incremental build issues that I've been reporting in another thread here. -- Jon From Alan.Bateman at Sun.COM Mon Mar 29 11:07:38 2010 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 29 Mar 2010 19:07:38 +0100 Subject: Creating new packages in the world of modules In-Reply-To: <53C308B4-C93A-4E51-80D8-F14ED95E59C1@sun.com> References: <53C308B4-C93A-4E51-80D8-F14ED95E59C1@sun.com> Message-ID: <4BB0EC6A.10402@sun.com> Weijun Wang wrote: > Hi Alan > > I'm planning to add two RFEs to OpenJDK: > > 1. A SASL mechanism called NTLM, which includes two new packages: > > a. com.sun.security.sasl.ntlm: The SASL provider > b. com.sun.security.ntlm: The raw NTLM API > > Here I've created an NTLM API which I hope can also be used by sun/net/www/protocol/http/ntlm/NTLMAuthentication.java. > > 2. Keytool and Jarsigner as JSR 199 Tools, which include interfaces in a new package com.sun.security.tools, implementations in sun.security.tools and new META-INF/services/com.sun.security.tools.* entries. > > I know you have spent a lot of time recently to make sure classes to be distributed in different modules and their dependencies carefully maintained. If I want to add the new packages above, what do I need to take care of most? Any configuration files to modify? Is there a latest map showing the content of current existing modules? > > BTW, what is the fate of ServiceLoader in modules? Will META-INF/services entries spread into different modules and ServiceLoader tries to get a union of them for each service class? > > Thanks > Max > > It might be best to start with a "ntlm" module (the name will change in time). It could be grouped into a coarser grain module later. The NTLM authentication implementation for the http protocol handler is currently buried in the "net-compat" module. There isn't anything directly dependent on it so moving it to ntlm should be OK. Once thing to mention is that the module configurations have changed a bit in the jigsaw forest and there will probably be more changes before it is pushed to jdk7. It might be easier to just ignore the module configuration for now and we can fix it before the big push into jdk7. I assume ServiceLoader will need to be updated. Mark is the best person to answer that. -Alan From mandy.chung at oracle.com Mon Mar 29 13:12:07 2010 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 29 Mar 2010 13:12:07 -0700 Subject: Please review the change to support dual mode launcher Message-ID: <4BB10997.9000203@oracle.com> Alan, Can you review this fix? Webrev at: http://cr.openjdk.java.net/~mchung/jigsaw/dual-mode-launcher/ The language tools such as javac, javap, and javah needs to access all platform classes. In legacy world, it will access the platform classes from the bootclasspath. However, in modules world, a module can only access classes from its required modules. For example, javap won't be able to access xml module: jdk-module-image/bin/javap javax.xml.XMLConstants So this patch will fix the launcher makefile to build javah and javap to run in legacy mode like javac. When Jon pushes his patch for javac to use the jigsaw resolver to find platform classes, javac could switch to run in module mode. I added an internal switch in the launcher, -mode:legacy or -mode:module, to specify a tool to run in legacy mode or module mode when running in a modules image (e.g. jdk-module-image). This is temporary and only valid for jdk tools to use. Jon, Does javadoc need to access platform classes like javap and javah? In your patch, javac uses the jigsaw resolver to find platform classes. How about javah and javap? Thanks Mandy From Sean.Mullan at Sun.COM Mon Mar 29 13:48:44 2010 From: Sean.Mullan at Sun.COM (Sean Mullan) Date: Mon, 29 Mar 2010 16:48:44 -0400 Subject: module file reading/writing In-Reply-To: <4B86BC65.1020108@sun.com> References: <4B86BC65.1020108@sun.com> Message-ID: <4BB1122C.3030102@sun.com> Hi Dalibor, I'm wondering why you would want to extract a module-file? Wouldn't you always want to install to a library directly from a module-file? I'm also thinking about when signatures should be verified. Once you extract the module, the signature can no longer be subsequently verified (since it is over the compressed content). However, it seems that signature verification (and other security validation logic such as certificate validation, policy checks) should really be done as part of the install phase. Can you describe what the difference is between extract and install, and why you need both? Thanks, Sean Dalibor Topic wrote: > Hi, > > I've pushed an initial implementation of the reader/writer code > for the module format specified in [1]. > > It adds a new 'jmod' verb to jpkg. So if you'd want to turn a 'hello' > module at version 0.1 on disk into a jmod file, you'd do this: > > jpkg -m hello jmod hello > > and get a hello at 0.1.jmod file. > > If the module has resources, etc you can tell jpkg about the > directories to add to a module using: > > -r dir for resources > --natlib dir for native libraries > --natcmd dir for native commands > --config dir for configuration data > > In addition you can specify a directory to create the jmod file > into using -d dir option. > > Within each module file section, regular files are gzipped, while > classes are pack200 compressed. > > In order to extract a jmod file, you can use the jmod tool, for > example > > jmod extract hello at 0.1.jmod > > will extract the module file into the 'hello' directory. Sections > other then classes are extracted into subdirectories. > > There is an initial set of module format tests, with more to come, > as I hack on improving the code. Please try it out, and tell me > if it works, breaks, etc. > > cheers, > dalibor topic > > [1] http://cr.openjdk.java.net/~mr/jigsaw/notes/module-file-format/ From Alan.Bateman at Sun.COM Tue Mar 30 07:24:35 2010 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 30 Mar 2010 15:24:35 +0100 Subject: Please review the change to support dual mode launcher In-Reply-To: <4BB10997.9000203@oracle.com> References: <4BB10997.9000203@oracle.com> Message-ID: <4BB209A3.7080903@sun.com> Mandy Chung wrote: > Alan, > > Can you review this fix? > > Webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/dual-mode-launcher/ > > The language tools such as javac, javap, and javah needs to access all > platform classes. In legacy world, it will access the platform > classes from the bootclasspath. However, in modules world, a module > can only access classes from its required modules. For example, javap > won't be able to access xml module: > jdk-module-image/bin/javap javax.xml.XMLConstants > > So this patch will fix the launcher makefile to build javah and javap > to run in legacy mode like javac. When Jon pushes his patch for > javac to use the jigsaw resolver to find platform classes, javac could > switch to run in module mode. I added an internal switch in the > launcher, -mode:legacy or -mode:module, to specify a tool to run in > legacy mode or module mode when running in a modules image (e.g. > jdk-module-image). This is temporary and only valid for jdk tools to use. > > Jon, > Does javadoc need to access platform classes like javap and javah? > In your patch, javac uses the jigsaw resolver to find platform > classes. How about javah and javap? > > Thanks > Mandy The change seems reasonable to me. I realize the switch is temporary/internal only but would it be better to make it -X option? Maybe -Xlegacymode and -Xmodulesmode? -Alan. From jonathan.gibbons at oracle.com Tue Mar 30 08:44:57 2010 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 30 Mar 2010 08:44:57 -0700 Subject: Please review the change to support dual mode launcher In-Reply-To: <4BB209A3.7080903@sun.com> References: <4BB10997.9000203@oracle.com> <4BB209A3.7080903@sun.com> Message-ID: <4BB21C79.1090909@oracle.com> Alan Bateman wrote: > Mandy Chung wrote: >> Alan, >> >> Can you review this fix? >> >> Webrev at: >> http://cr.openjdk.java.net/~mchung/jigsaw/dual-mode-launcher/ >> >> The language tools such as javac, javap, and javah needs to access >> all platform classes. In legacy world, it will access the platform >> classes from the bootclasspath. However, in modules world, a module >> can only access classes from its required modules. For example, >> javap won't be able to access xml module: >> jdk-module-image/bin/javap javax.xml.XMLConstants >> >> So this patch will fix the launcher makefile to build javah and javap >> to run in legacy mode like javac. When Jon pushes his patch for >> javac to use the jigsaw resolver to find platform classes, javac >> could switch to run in module mode. I added an internal switch in >> the launcher, -mode:legacy or -mode:module, to specify a tool to run >> in legacy mode or module mode when running in a modules image (e.g. >> jdk-module-image). This is temporary and only valid for jdk tools to >> use. >> >> Jon, >> Does javadoc need to access platform classes like javap and >> javah? In your patch, javac uses the jigsaw resolver to find >> platform classes. How about javah and javap? >> >> Thanks >> Mandy > The change seems reasonable to me. I realize the switch is > temporary/internal only but would it be better to make it -X option? > Maybe -Xlegacymode and -Xmodulesmode? > > -Alan. > I think an option taking a value is better than a pair of boolean options. Separately, note that users will have to use -J to prefix the option, e.g. -J-Xmode:legacy. That's not an issue, but just remember that when trying out the option, and when documenting it. javadoc is similar to the other tools. The requirement for all of these tools is that the system property sun.boot.class.path is set to the platform class path. This is embodied in the code that handles these tools path options, Paths.java, round about line 344 http://hg.openjdk.java.net/jdk7/tl/langtools/file/de6375751eb7/src/share/classes/com/sun/tools/javac/file/Paths.java. Thus the tools could operate in "hybrid" mode, where the tool's JVM runs in module mode, but sun.boot.class.path is set to a value that would allow the tool itself to operate in legacy mode. The distinction is similar to the existing distinction between -Xbootclasspath and -J-Xbootclasspath seen by all the tools. -- Jon From Dalibor.Topic at Sun.COM Tue Mar 30 15:45:06 2010 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Wed, 31 Mar 2010 00:45:06 +0200 Subject: module file reading/writing In-Reply-To: <4BB1122C.3030102@sun.com> References: <4B86BC65.1020108@sun.com> <4BB1122C.3030102@sun.com> Message-ID: <4BB27EF2.1090103@sun.com> Sean Mullan wrote: > Hi Dalibor, > > I'm wondering why you would want to extract a module-file? I implemented it because it made testing the module format during development a bit simpler - extract, compare with originals, on to the next part in the spec. > Wouldn't you > always want to install to a library directly from a module-file? Yes. > I'm also thinking about when signatures should be verified. Once you > extract the module, the signature can no longer be subsequently verified > (since it is over the compressed content). However, it seems that > signature verification (and other security validation logic such as > certificate validation, policy checks) should really be done as part of > the install phase. Yep. > Can you describe what the difference is between extract and install, and > why you need both? Extract really just treat the module file as a container for data, which becomes problematic once we have signatures to keep for verification - so I don't think that we need both. cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der From mandy.chung at oracle.com Wed Mar 31 16:00:49 2010 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 31 Mar 2010 16:00:49 -0700 Subject: Please review the change to support dual mode launcher In-Reply-To: <4BB21C79.1090909@oracle.com> References: <4BB10997.9000203@oracle.com> <4BB209A3.7080903@sun.com> <4BB21C79.1090909@oracle.com> Message-ID: <4BB3D421.1010308@oracle.com> On 03/30/10 08:44, Jonathan Gibbons wrote: > Alan Bateman wrote: >> The change seems reasonable to me. I realize the switch is >> temporary/internal only but would it be better to make it -X option? Thanks for reminding me. Yes, I should use -X option. >> Maybe -Xlegacymode and -Xmodulesmode? >> >> -Alan. >> > I think an option taking a value is better than a pair of boolean options. > Separately, note that users will have to use -J to prefix the option, > e.g. -J-Xmode:legacy. That's not an issue, but just remember that when > trying out the option, and when documenting it. I updated it to take this new option -Xmode:legacy and -Xmode:module. > javadoc is similar to the other tools. Ok. javadoc will run in legacy mode by default until it's updated to look for the platform classes from jigsaw. > The requirement for all of these > tools is that the system property sun.boot.class.path is set to the > platform class path. This is embodied in the code that handles these > tools path options, Paths.java, round about line 344 > > http://hg.openjdk.java.net/jdk7/tl/langtools/file/de6375751eb7/src/share/classes/com/sun/tools/javac/file/Paths.java. > > > Thus the tools could operate in "hybrid" mode, where the tool's JVM runs > in module mode, but sun.boot.class.path is set to a value that would > allow the tool itself to operate in legacy mode. The distinction is > similar to the existing distinction between -Xbootclasspath and > -J-Xbootclasspath seen by all the tools. Per our discussion yesterday, we want these tools to operate in "jigsaw-enabled" mode like the new javac. The tools can then switch to run in module mode. Thanks Mandy From Mandy.Chung at Sun.COM Wed Mar 31 16:15:47 2010 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Wed, 31 Mar 2010 16:15:47 -0700 Subject: Use jdk-module-image as boot jdk (modules build + SKIP_BOOT_CYCLE=false) Message-ID: <4BB3D7A3.2090409@Sun.com> Kelly, Alan, Karen, Jon and I have been discussing the testing for the new javac change and modules image. Building JDK itself is one the the simplest and most effective tests that Jon typically uses to validate javac changes (by building JDK with SKIP_BOOT_CYCLE=false). Currently, "make SKIP_BOOT_CYCLE=false all" will first build a legacy jdk image and set it as the ALT_BOOTDIR to rebuild the jdk. As Jon suggests, for modules build, SKIP_BOOT_CYCLE=false should use the jdk-module-image as the ALT_BOOTDIR to rebuild the jdk modules. This fix makes changes in the top level repo, jdk, hotspot, and corba repository as follows: 1. http://cr.openjdk.java.net/~mchung/jigsaw/bootdir-modules-image/top-repo - Set JDK_IMAGE_NAME to j2sdk-image for legacy image build or jdk-module-image for modules image build. - not to create j2sdk-image for modules build (a bug Jon reported) 2. http://cr.openjdk.java.net/~mchung/jigsaw/bootdir-modules-image/jdk - minor cleanup such as using absolute path of the modules directory - Add OTHER_JAVAHFLAGS that will be used to set classpath to look up pkcs11 and mscapi classes from a temp directory - Add the logics to import from the modules instead of rt.jar and resources.jar Eventually we should import modules instead of copying in the classes. - eliminate the second build of the make/modules/Makefile Modules.gmk to invoke make/modules/Makefile build was needed so as to make sure the jdk modules in the "modules" directory are up-to-date. However, this is only needed for jdk developers who is doing incremental builds after the jdk is modularized. Like the images build taking the current content from the "classes" directory, it will just take the current content in the "modules" directory to build the modules images. For incremental builds, the jdk developers need to make sure the jdk modules are up-to-date. I'll also come up with some better way to make our life easier in updating the jdk-module-image. 3. http://cr.openjdk.java.net/~mchung/jigsaw/bootdir-modules-image/corba/ - corba, for some reason, links with libjvm.so and libjava.so. The fix is to include the new library path of the jdk-module-image. This will need to change when we move the native libraries to the module content. 4. http://cr.openjdk.java.net/~mchung/jigsaw/bootdir-modules-image/hotspot/ - The makefile checks if the tools.jar exists for building SA. In the modular world, it no longer has the tools.jar. An open issue: the jaxp and jaxws build may download the source bundles. With using jdk-module-image as the boot jdk, jaxp and jaxws build will set ANT_JAVA_HOME to the jdk-module-image that will fail the download as security/crypto support in the modules image is an open issue. Caused by: java.lang.SecurityException: Can not initialize cryptographic mechanism^M at javax.crypto.JceSecurity.(JceSecurity.java:86)^M I'd like to push this set of changes in jigsaw althought there is the jaxp and jaxws bundle download issue. As long as there is a local downloaded copy of the jaxp/jaxws bundle, it will build fine. Thanks Mandy